Speech keyword retrieval method, device and equipment and storage medium

Abstract

This invention provides a method, apparatus, device, and storage medium for voice keyword retrieval. The method includes: extracting features and performing cluster analysis on labeled intervals of N keyword sample audios; determining the number of states corresponding to each keyword based on the clustering results; and establishing a network model that includes keyword state sequences and non-keyword absorption states. Features are extracted from the audio to be retrieved, and the extracted features are input into the network model for decoding. Suspected keyword segments are determined based on the probability values ​​of the keyword state sequences in the decoding path. Features are extracted from the suspected keyword segments and input into a pre-trained support vector regression model. The classification score output by the support vector regression model is used to determine whether the suspected keyword segment is a trained keyword. This invention automatically determines the number of states by performing temporal cluster analysis on the features of keyword samples, making it applicable to keyword retrieval for any language and even non-language sounds, thus expanding the scope of application and improving retrieval accuracy.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and storage medium for voice keyword retrieval. Background Technology

[0002] Traditional speech keyword retrieval methods typically require prior knowledge of the language type and phoneme structure of the keywords when building the keyword model. For Mandarin keywords, initial and final vowel information needs to be annotated; for English keywords, phonetic symbol information is required; and for other languages ​​or dialects, corresponding phoneme-level annotations are also necessary. Based on this annotation information, the model can determine appropriate structural parameters and be trained. This phoneme-annotation-dependent modeling approach can achieve certain retrieval results when processing a single language.

[0003] However, in practical applications, the keywords to be retrieved may come from different languages ​​or dialects, or even be specific sounds that are not spoken. Phoneme-level annotation for each language or dialect requires not only specialized linguistic knowledge but is also costly, time-consuming, and labor-intensive. More importantly, when the language type of the keywords is unknown or they are non-spoken sounds, prior phoneme structure information is unavailable, rendering traditional methods inapplicable. This reliance on phoneme annotation limits the applicability of keyword retrieval methods and makes it difficult to meet the practical needs of cross-language and cross-dialect scenarios. Summary of the Invention

[0004] The main objective of this invention is to solve the technical problem that existing speech keyword retrieval methods rely on phoneme annotation, which limits their applicability. This invention provides a voice keyword retrieval method, characterized in that the voice keyword retrieval method includes: Feature extraction and cluster analysis are performed on the labeled intervals of N keyword sample audio uploaded by users. Based on the clustering results, the number of states corresponding to the keywords is determined, and a network model containing keyword state sequences and non-keyword absorption states is established. Feature extraction is performed on the audio to be retrieved, and the extracted features are input into the network model for decoding. Suspected keyword segments are determined based on the probability values ​​of the keyword state sequences in the decoding path. Feature extraction is performed on the suspected keyword fragments and input into a pre-trained support vector regression model. The classification score output by the support vector regression model is used to determine whether the suspected keyword fragments are training keywords.

[0005] The present invention also provides a voice keyword retrieval device, characterized in that the voice keyword retrieval device comprises: The model building module is used to extract features and perform cluster analysis on the labeled intervals of N keyword sample audio uploaded by users. Based on the clustering results, the number of states corresponding to the keywords is determined, and a network model containing keyword state sequences and non-keyword absorption states is built. The preliminary retrieval module is used to extract features from the audio to be retrieved, input the extracted features into the network model for decoding, and determine the suspected keyword segments based on the probability values ​​of the keyword state sequence in the decoding path. The verification and judgment module is used to extract features from the suspected keyword fragments and input them into a pre-trained support vector regression model, and determine whether the suspected keyword fragments are trained keywords based on the classification scores output by the support vector regression model.

[0006] The present invention also provides a voice keyword retrieval device, comprising: a memory and at least one processor, wherein the memory stores instructions, and the memory and the at least one processor are interconnected via a circuit; the at least one processor invokes the instructions in the memory to cause the voice keyword retrieval device to perform the steps of the above-described voice keyword retrieval method.

[0007] The present invention also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the steps of the above-described voice keyword retrieval method.

[0008] The aforementioned speech keyword retrieval method, apparatus, device, and storage medium extract features and perform cluster analysis on the labeled intervals of N keyword sample audios. Based on the clustering results, the number of states corresponding to each keyword is determined, and a network model containing keyword state sequences and non-keyword absorption states is established. Features are extracted from the audio to be retrieved, and the extracted features are input into the network model for decoding. Suspected keyword segments are determined based on the probability values ​​of the keyword state sequences in the decoding path. Features are extracted from the suspected keyword segments and input into a pre-trained support vector regression model. The classification score output by the support vector regression model determines whether the suspected keyword segment is a trained keyword. This invention automatically determines the number of states by performing temporal cluster analysis on the features of keyword samples, establishing a network model without manual annotation of phoneme information. This makes the method applicable to keyword retrieval of any language, dialect, and even non-speech sounds, significantly expanding the scope of application and improving retrieval accuracy.

[0009] Beneficial Effects: This invention automatically determines the number of states by clustering the features of keyword sample audio, allowing the complexity of the network model to adaptively adjust according to the actual phoneme structure of the keywords. This avoids the overfitting or insufficient expressive power problems that occur in fixed-parameter models when handling keywords of different lengths. Furthermore, a support vector regression model is used to perform secondary verification on the suspected keyword fragments obtained from the initial retrieval, further filtering out false detection fragments caused by inappropriate model adaptation. This significantly improves the retrieval accuracy while maintaining the detection rate.

[0010] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0011] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the first embodiment of the voice keyword retrieval method in this invention; Figure 2 This is a schematic diagram of a second embodiment of the voice keyword retrieval method in this invention; Figure 3 This is a schematic diagram of one embodiment of the voice keyword retrieval device in this invention; Figure 4 This is a schematic diagram of one embodiment of the voice keyword retrieval device in this invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] The terms "comprising" and "having," and any variations thereof, used in the embodiments of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0015] To facilitate understanding of this embodiment, a detailed description of the voice keyword retrieval method disclosed in this embodiment of the invention will be provided first. For example... Figure 1 As shown, this method includes the following steps: 101. Perform feature extraction and cluster analysis on the labeled intervals of N keyword sample audio uploaded by users, determine the number of states corresponding to the keywords based on the clustering results, and establish a network model that includes keyword state sequences and non-keyword absorption states. In this embodiment, the user uploads sample audio clips containing the keywords to be searched via a client. These sample audio clips contain the keyword segments that the user hopes to find in the audio clips. When uploading sample audio clips, the user needs to mark the actual time period in which the keyword appears in each sample audio clip; this time period is called the marked interval. The purpose of the marked interval is to clearly inform the system of the specific location of the keyword in the sample audio clip, enabling the system to extract the keyword's features specifically from that interval, rather than processing the entire audio clip.

[0016] The system extracts features from the labeled intervals of N sample audio files. Feature extraction refers to performing time-frequency domain analysis on the audio signal to extract acoustic feature parameters that characterize the sound properties. In this embodiment, the extracted features may include commonly used speech features such as Mel-frequency cepstral coefficients and first-order difference coefficients. Through feature extraction, the system converts the audio signal into a sequence of feature vectors, facilitating subsequent modeling and analysis.

[0017] After obtaining N sets of feature vector sequences, the system performs cluster analysis on these sequences. The purpose of cluster analysis is to identify the phoneme structure features of keywords during pronunciation and automatically determine the number of states required for modeling based on the phoneme structure. Keywords of different lengths have different numbers of phonemes. Through cluster analysis, the system can automatically identify how many phoneme units a keyword contains, thereby determining the corresponding number of states. For example, short keywords may cluster into two to three states, while long keywords may cluster into five to six states.

[0018] Based on the number of states determined by the clustering results, the system establishes a network model that includes keyword state sequences and non-keyword absorption states. The keyword state sequences describe the phoneme changes of the keywords, with each state corresponding to a phoneme unit in the keyword pronunciation process. The non-keyword absorption states model all speech content except for the keywords, including speech before and after the keywords, environmental noise, etc. The keyword state sequences and non-keyword absorption states constitute a unified network model, in which each state is modeled using a Gaussian mixture model.

[0019] The system employs the expectation-maximization algorithm to train the network model. During training, the system iteratively optimizes the state transition probability matrix and observation probability distribution parameters of the network model based on the feature vector sequences of N sample audio samples. After multiple iterations, the network model parameters gradually converge, resulting in a trained network model that accurately represents the characteristics of keywords.

[0020] 102. Extract features from the audio to be retrieved, input the extracted features into the network model for decoding, and determine the suspected keyword segments based on the probability values ​​of the keyword state sequences in the decoding path; In this embodiment, the step of extracting features from the audio to be retrieved, inputting the extracted features into the network model for decoding, and determining suspected keyword segments based on the probability values ​​of keyword state sequences in the decoding path includes: performing frame segmentation on the audio to be retrieved and extracting Mel-frequency cepstral coefficient features to obtain a feature vector sequence of the audio to be retrieved; inputting the feature vector sequence into the network model and using a dynamic programming algorithm to decode in the state space of the network model to obtain the optimal decoding path; and determining suspected keyword segments based on the time periods in the optimal decoding path where the probability value corresponding to the keyword state sequence is greater than the probability values ​​of other decoding paths.

[0021] Specifically, the audio to be searched refers to the target audio from which the user hopes to find keywords. Unlike the sample audio, the audio to be searched is usually longer, and the keywords may appear anywhere in the audio or may not appear at all. The system needs to perform a traversal search on the audio to be searched to determine whether the keywords exist and their specific locations.

[0022] The system performs frame segmentation on the audio to be retrieved. Frame segmentation is a fundamental operation in speech signal processing, and its principle is to divide a continuous audio signal into a series of short audio frames. This processing method is based on the assumption of short-time stationarity of speech signals, that is, the statistical characteristics of speech signals remain relatively stable within a short time window. Through framing, the system can extract stable acoustic features within each short time window. Frame segmentation is typically controlled by two parameters: frame length and frame shift. There is a certain overlap between adjacent frames to ensure signal continuity.

[0023] Acoustic analysis is performed on each audio frame to extract Mel-frequency cepstral coefficients (CFCs). These CFCs reflect the nonlinear perceptual characteristics of the human ear at different frequencies and can effectively characterize the spectral envelope information of the speech signal. The system arranges the extracted features in chronological order to form a feature vector sequence for the audio to be retrieved. The length of this sequence depends on the duration and framing parameters of the audio; longer audio will produce a longer feature vector sequence.

[0024] The system inputs the feature vector sequence into the trained network model for decoding. The decoding process refers to finding the state sequence most likely to have generated the observed sequence, given the observed sequence. In this embodiment, the observed sequence is the extracted feature vector sequence, and the state sequence includes the keyword state sequence and the non-keyword absorption states.

[0025] The system employs a dynamic programming algorithm for decoding. The core idea of ​​dynamic programming is to decompose a complex global optimization problem into a series of local optimization subproblems, and then solve for the global optimum recursively. During decoding, the algorithm starts from the first frame of the feature vector sequence and proceeds frame by frame. For each frame, the algorithm calculates the cumulative probability of reaching each state in the network model and records the optimal predecessor state for each state. The cumulative probability comprehensively considers the state probability of the previous time step, the transition probability between states, and the matching probability between the current frame's features and the state.

[0026] After processing all frames, the algorithm starts from the state with the highest probability in the last frame and backtracks along the recorded preceding states to obtain the optimal decoding path. This path describes the trajectory of the feature vector sequence in the network model's state space, reflecting the most likely state corresponding to each time segment of the audio to be retrieved.

[0027] Suspected keyword segments are identified based on the optimal decoding path. The system analyzes the state type of each time segment within the decoding path. When the path continuously passes through a keyword state sequence, it indicates that the audio content of the corresponding time segment matches the phoneme structure of the keyword. The system calculates the probability value corresponding to the keyword state sequence within these time segments and compares it with the probability value corresponding to the non-keyword absorption state. If the probability value of the keyword state sequence is significantly greater than the probability value of the non-keyword absorption state, it indicates that the time segment is more likely to be a keyword than other audio content, and the system marks this time segment as a suspected keyword segment.

[0028] It should be noted that because the network model learns the acoustic feature distribution patterns of keywords and non-keywords during training, the decoding process can automatically select between keyword state sequences and non-keyword absorption states. When the input audio segment is similar to keyword features, the decoding path tends to pass through the keyword state sequence; when the input audio segment is dissimilar to keyword features, the decoding path tends to remain in the non-keyword absorption state. Through this mechanism, the system can locate suspected segments that may contain keywords in long audio files.

[0029] It is understandable that the suspected keyword segments determined in this stage may contain certain false detections, that is, although some segments are similar to the keyword part in acoustic features, they are not actually the target keywords. This method adopts the principle of searching for sounds by sounds, and all segments with the same pronunciation will be retrieved. For example, "WeChat" and "Weixin" cannot be distinguished by pronunciation because they have the same pronunciation. The false detections in this stage mainly come from segments that are partially similar in pronunciation but not completely matched as a whole. Therefore, the suspected keyword segments need to go through subsequent verification steps to further confirm whether their pronunciation is exactly the same as the target keyword.

[0030] 103. Extract features from the suspected keyword segments and input them into a pre-trained support vector regression model, and judge whether the suspected keyword segments are the trained keywords according to the classification scores output by the support vector regression model.

[0031] In this embodiment, the step of extracting features from the suspected keyword segments and inputting them into a pre-trained support vector regression model, and judging whether the suspected keyword segments are the trained keywords according to the classification scores output by the support vector regression model includes: performing frame division processing on the suspected keyword segments and extracting features, determining the keyword state to which each frame belongs according to the decoding path, statistically averaging the features of each frame belonging to the same keyword state to obtain the state feature vectors corresponding to each keyword state; splicing the state feature vectors corresponding to each keyword state in the order of the state sequence to obtain the feature vector of the suspected keyword segment; inputting the feature vector of the suspected keyword segment into a pre-trained support vector regression model, calculating the distance from the feature vector to the decision hyperplane through the support vector regression model to obtain the classification score; comparing the classification score with a preset threshold, and when the classification score is greater than the preset threshold, determining that the suspected keyword segment is the trained keyword, and recording the time position information of the keyword in the audio to be retrieved.

[0032] Specifically, secondary verification is performed on the previously determined suspected keyword segments. Although dynamic programming decoding can initially screen out segments that may contain keywords according to probability values, due to the existence of some similar pronunciations, some segments with pronunciations that are partially similar to the target keyword but not completely matched as a whole may be misjudged as suspected keywords. The role of the support vector regression model is to establish a more refined discrimination boundary, and by analyzing the features of the complete pronunciation sequence, further confirm whether the pronunciation of the suspected segment is exactly the same as the target keyword, so as to distinguish the real keyword from the false detection segments with partially similar pronunciations.

[0033] The suspected keyword segments are segmented into frames and their features are extracted. This feature extraction process is consistent with the feature extraction method for the audio to be retrieved, and the extracted features include acoustic parameters such as Mel-frequency cepstral coefficients. Since suspected keyword segments are usually short, the number of frames obtained after segmentation is relatively small.

[0034] After obtaining the features of the suspected audio segments, these features need to be organized into a form suitable for processing by a support vector regression model. The key lies in how to convert audio segments of variable length into feature vectors of fixed dimensions. This embodiment uses a state-based statistical method to achieve this conversion.

[0035] Based on the decoding path obtained above, the keyword state to which each frame in the suspected fragment belongs is determined. The decoding path records the state corresponding to each frame in the network model. By querying the decoding path, it can be determined which specific state in the keyword state sequence each frame is assigned to. For example, if the keyword state sequence contains five states and the suspected fragment has twenty frames, the decoding path may show that the first four frames belong to the first state, the next five frames belong to the second state, and so on.

[0036] For all frames belonging to the same keyword state, their features are statistically averaged. This averaging operation is performed along the feature dimension, that is, the values ​​of all frames belonging to this state in each feature dimension are summed and then divided by the number of frames. Through this averaging, a representative feature of this state is obtained, called the state feature vector. The state feature vector not only contains the acoustic information of this state, but also eliminates the random fluctuations of single-frame features through the averaging operation, thus having better stability.

[0037] Perform the above averaging operation on all keyword states to obtain the state feature vector corresponding to each keyword state. Assuming the keyword state sequence contains S states, and the feature vector of each state has a dimension of D, then we will obtain S D-dimensional state feature vectors. These state feature vectors respectively characterize the acoustic properties of the keyword at different pronunciation stages.

[0038] The feature vectors of each keyword state are concatenated according to the state sequence. Concatenation involves joining multiple vectors end-to-end to form a longer vector. The concatenation order strictly follows the keyword state sequence, with the feature vector of the first state at the beginning and the feature vector of the last state at the end. The resulting vector has a dimension of D multiplied by S, and this vector represents the feature vector of the suspected keyword fragment.

[0039] This feature extraction method based on state statistics and concatenation is of great significance. On the one hand, it transforms variable-length audio segments into fixed-dimensional feature vectors, enabling support vector regression models to handle inputs of varying lengths. On the other hand, these feature vectors not only contain acoustic information but also implicitly contain phoneme sequence information, because different states correspond to different phoneme units of keywords, and the arrangement order of the state feature vectors reflects the temporal evolution of phonemes.

[0040] The feature vectors of suspected keyword fragments are input into a pre-trained support vector regression model. During training, the support vector regression model has learned the distribution patterns of true keyword and non-keyword feature vectors in the feature space and has established a decision hyperplane between the two classes of feature vectors. This hyperplane is a high-dimensional plane in the feature space used to separate the two classes of samples.

[0041] Support Vector Regression (SVR) models calculate the distance from the input feature vector to the decision hyperplane. This distance is directed, meaning it has both magnitude and direction. The distance is positive when the feature vector lies on the keyword side of the decision hyperplane, and negative when it lies on the non-keyword side. The absolute value of the distance reflects the class confidence of the feature vector; a larger absolute value indicates greater confidence that the sample belongs to the corresponding class. This distance can be converted into a similarity probability value between zero and one, which is then output as a classification score.

[0042] The classification score is compared with a preset threshold. The preset threshold is a decision boundary, usually set to a value between zero and one. When the classification score is greater than the preset threshold, the suspected segment can be determined to be a real training keyword. The temporal location information of the keyword in the audio to be retrieved, including start and end times, is recorded for user viewing and use.

[0043] When the classification score is less than or equal to a preset threshold, it indicates that the suspected fragment is more likely a false positive (non-keyword fragment), and it is filtered out and not included in the final search results. Validation using a support vector regression model effectively reduces the false positive rate and improves the accuracy of keyword retrieval.

[0044] It's important to note that the discriminative power of the Support Vector Regression (SVR) model stems from its training data, which includes both correct and incorrect keyword samples, used during the training phase. This training data allows the model to learn the subtle differences between true keywords and easily confused non-keywords. Therefore, the validation process of the SVR model is essentially a pattern recognition process based on historical experience, capable of identifying false positives missed in the initial retrieval.

[0045] Furthermore, before extracting features from the suspected keyword fragments and inputting them into the pre-trained support vector regression model, the method further includes: performing a preliminary search on a preset verification audio set to obtain preliminary search results; manually labeling the preliminary search results, classifying correctly labeled audio fragments into a positive sample set and incorrectly labeled audio fragments into a negative sample set; extracting features from the positive sample set and the negative sample set respectively to obtain a positive sample feature vector set and a negative sample feature vector set; training the support vector regression model based on the positive sample feature vector set and the negative sample feature vector set, so that the positive sample feature vector set and the negative sample feature vector set form two separable regions in the feature space, and establishing a decision hyperplane between the two regions to obtain the trained support vector regression model.

[0046] Specifically, before extracting features from the suspected keyword fragments and inputting them into the pre-trained support vector regression model, the process also includes training the support vector regression model.

[0047] In this embodiment, the support vector regression model needs to learn how to distinguish between true keyword segments and false positive segments through training. The key to training lies in obtaining sufficient positive and negative samples. Positive samples are audio segments that actually contain the target keyword, while negative samples are audio segments that do not contain the target keyword but are acoustically similar to the keyword. These negative samples are often the main source of false positives; therefore, collecting negative samples is crucial for training a high-performance validation model.

[0048] A preliminary search is performed on the pre-set verification audio set to obtain preliminary search results. The verification audio set is a pre-prepared set of audio data. These audio files can be from other audio files provided by the user or actual audio collected from the application scenario. The characteristic of the verification audio set is that it may or may not contain the target keywords, and the location and frequency of the keywords are unknown.

[0049] The pre-trained network model is used to retrieve data from the validation audio set. This retrieval process is identical to that for the audio to be retrieved, employing a dynamic programming algorithm to decode the data within the network model. Suspected keyword segments are determined based on the probability values ​​of the keyword state sequences in the decoding path. Since the network model was trained using only a limited number of audio samples at this stage, its discriminative ability is limited. Therefore, the initial retrieval results contain both genuine keyword segments and falsely detected non-keyword segments.

[0050] The initial search results are manually annotated. Annotators play or view each retrieved audio segment to determine whether the segment actually contains the target keyword. If the segment does contain the target keyword, it is marked as correct and added to the positive sample set; if the segment does not contain the target keyword, it is marked as incorrect and added to the negative sample set.

[0051] The audio clips in the positive sample set represent genuine keyword samples. Although these samples were retrieved through a network model, they can be considered reliable training data after being manually verified. Compared to the sample audio initially uploaded by the user, the samples in the positive sample set may come from different audio files, different recording environments, or different speakers, thus contributing to sample diversity.

[0052] The audio segments in the negative sample set represent difficult samples that are easily misclassified as keywords by the network model. The existence of these negative samples indicates that the network model may misidentify segments with similar pronunciations but not a complete match in some cases. By collecting these negative samples, the validation model can be trained in a targeted manner, enabling it to learn how to distinguish between the complete pronunciation of the target keyword and segments with similar pronunciations.

[0053] Features are extracted from both the positive and negative sample sets, resulting in positive and negative sample feature vector sets, respectively. The feature extraction method is identical to that used for the previously mentioned suspected segment: the audio segment is segmented into frames, acoustic features are extracted, the keyword state of each frame is determined based on the decoding path, the features of frames belonging to the same state are averaged to obtain a state feature vector, and these state feature vectors are then concatenated sequentially to form a fixed-dimensional feature vector. This unified feature extraction method ensures that both positive and negative samples are represented as feature vectors of the same dimension, facilitating subsequent training.

[0054] A support vector regression model is trained based on the sets of positive and negative sample feature vectors. The goal of training is to find an optimal decision hyperplane in the feature space that separates the positive and negative sample feature vectors as well as possible. Separation means that the positive sample feature vectors lie on one side of the decision hyperplane, the negative sample feature vectors lie on the other side, and the two classes of samples maintain the largest possible margin with respect to the decision hyperplane.

[0055] The training algorithm for Support Vector Regression (SVR) searches for the optimal hyperplane in the feature space that satisfies the above conditions. The core idea of ​​this algorithm is to maximize the classification margin, that is, to maximize the minimum distance between the two classes of samples and the decision hyperplane. During training, not all samples influence the position of the decision hyperplane; only those samples closest to the decision hyperplane, called support vectors, determine its position. These support vectors are typically the most difficult samples to distinguish, located in the boundary region between the two classes.

[0056] After training, the support vector regression model learns the distribution patterns of positive and negative feature vector sets in the feature space. Positive feature vectors form one cluster region, and negative feature vectors form another. The decision hyperplane lies between these two regions, acting as a boundary. For a newly input suspected fragment feature vector, by calculating its distance to the decision hyperplane, it can be determined whether the feature vector is closer to the positive or negative sample region, thus determining whether the suspected fragment is a real keyword or a false positive.

[0057] It's important to note that manual annotation plays a crucial role in training the support vector regression model. It is through manual annotation that true keywords and false positives in the initial search results are clearly distinguished, enabling the model to learn the discriminative features of the two types of samples. This training method based on manual annotation essentially transforms human discriminative experience into the discriminative ability of the machine learning model, allowing the subsequent automatic verification process to simulate the accuracy of human judgment.

[0058] Understandably, the performance of a support vector regression model depends on the quality and quantity of its training samples. If the negative sample set includes various types of audio segments that could lead to false positives, the trained model will be better able to handle different situations in real-world applications. Therefore, in practice, negative samples can be continuously accumulated through multiple iterations to gradually improve the performance of the validation model. After each retrieval, newly discovered false positive segments are added to the negative sample set, and the support vector regression model is retrained to continuously improve its discriminative ability.

[0059] In this embodiment, feature extraction and cluster analysis are performed on the labeled intervals of N keyword sample audios. Based on the clustering results, the number of states corresponding to each keyword is determined, and a network model containing keyword state sequences and non-keyword absorption states is established. Feature extraction is performed on the audio to be retrieved, and the extracted features are input into the network model for decoding. Suspected keyword segments are determined based on the probability values ​​of the keyword state sequences in the decoding path. Features are extracted from the suspected keyword segments and input into a pre-trained support vector regression model. The classification score output by the support vector regression model determines whether the suspected keyword segment is a trained keyword. This invention automatically determines the number of states by performing temporal cluster analysis on the features of keyword samples, establishing a network model without manual annotation of phoneme information. This makes the method applicable to keyword retrieval in any language, dialect, and even non-verbal sounds, significantly expanding the scope of application and improving retrieval accuracy.

[0060] Please see Figure 2 Another embodiment of the voice keyword retrieval method in this application includes: 201. Perform frame segmentation and feature extraction on the labeled intervals of the N keyword sample audios to obtain N sets of feature vector sequences; In this embodiment, the labeled intervals of the N keyword sample audios are processed separately. The labeled intervals have clearly defined the specific time range of the keywords in each sample audio. Frame segmentation processing is performed on these intervals to convert the continuous audio signal into a discrete audio frame sequence.

[0061] Framing is performed using a fixed window length and frame shift. The window length determines the duration of each audio frame, while the frame shift determines the time interval between adjacent frames. When the frame shift is less than the window length, adjacent frames overlap. This overlap design ensures a smooth transition of the audio signal between frames and avoids information loss caused by framing.

[0062] Feature extraction is performed on each audio frame. The process involves converting the time-domain signal of the audio frame into a frequency-domain representation and extracting parameters that characterize its acoustic properties. The extracted features include Mel-frequency cepstral coefficients and first-order difference coefficients. The Mel-frequency cepstral coefficients reflect the spectral envelope information of the audio frame, while the first-order difference coefficients reflect the trend of feature changes over time. Combining these two features provides a more comprehensive description of the dynamic characteristics of the speech signal.

[0063] The features extracted from the labeled intervals of each audio sample are arranged in chronological order to form a feature vector sequence. The length of this sequence depends on the duration of the labeled interval and the framing parameters. The labeled intervals for different audio samples may vary in length, thus the lengths of the resulting feature vector sequences may also differ. By performing the above operation on N audio samples, N sets of feature vector sequences are obtained, each corresponding to a keyword sample. It should be noted that although these N sets of feature vector sequences may have different lengths, they all describe the acoustic feature changes of the same keyword under different pronunciation instances.

[0064] 202. Perform time-series cluster analysis on the N sets of feature vector sequences, and determine the number of states in the keyword state sequence based on the clustering results; In this embodiment, the step of performing time-series cluster analysis on the N sets of feature vector sequences and determining the number of states in the keyword state sequence based on the clustering results includes: calculating the similarity between the N sets of feature vector sequences, dividing the feature vector sequences into multiple clusters based on the similarity; calculating the intra-cluster variance and inter-cluster distance of each cluster, evaluating the clustering quality based on the intra-cluster variance and inter-cluster distance; determining the optimal number of clusters based on the clustering quality, and using the optimal number of clusters as the number of states in the keyword state sequence.

[0065] Specifically, cluster analysis is performed on N sets of feature vector sequences to identify the phonemic structural features of keywords during pronunciation. The pronunciation of a keyword can be viewed as a sequence composed of several phonemic units, each corresponding to a relatively stable state of the articulatory organ. Through cluster analysis, these phonemic units can be automatically identified from the feature vector sequences, and their number can be determined.

[0066] The temporal clustering mentioned above refers to considering the temporal positional relationship of feature vectors in the sequence during the clustering process. Unlike ordinary clustering methods, temporal clustering requires that adjacent feature vectors are more likely to belong to the same cluster. This constraint conforms to the continuous characteristics of speech signals, that is, the same phoneme unit will continue for a period of time during the pronunciation process, generating several consecutive frames of features.

[0067] Calculate the similarity between N sets of feature vector sequences. The similarity calculation applies to the entire feature vector sequence, not individual feature vectors. By comparing the temporal alignment similarity of feature vector sequences between different samples, feature segments corresponding to the same phoneme unit can be identified in different samples. For example, if two samples contain the same phoneme unit, the feature segment corresponding to that phoneme unit should have a high similarity between the two samples.

[0068] The feature vector sequence is divided into multiple clusters based on similarity. The division process involves finding split points in the feature vector sequence such that the segmented fragments have high similarity among different samples. Each cluster corresponds to a phoneme unit in the keyword, and the cluster contains feature fragments from N samples at that phoneme unit location. Through this division, the phoneme structure of the keyword is revealed, and different clusters represent different phoneme units.

[0069] Calculate the intra-cluster variance and inter-cluster distance for each cluster. Intra-cluster variance measures the consistency of features within the same cluster; a smaller variance indicates more similar features within the cluster, suggesting that the pronunciation of the phoneme unit is relatively stable across different samples. Inter-cluster distance measures the discriminative power between different clusters; a larger distance indicates more significant differences in features between different phoneme units, and better clustering quality.

[0070] Clustering quality is assessed based on intra-cluster variance and inter-cluster distance. Ideally, clustering results should have low intra-cluster variance and high inter-cluster distance, meaning compact clusters and well-spaced inter-clusters. By calculating a comprehensive evaluation index of these two metrics, the merits of different clustering schemes can be quantified. Commonly used evaluation metrics include the silhouette coefficient and the Davis-Bauerding index, which comprehensively consider both intra-cluster compactness and inter-cluster segregation.

[0071] The optimal number of clusters is determined based on cluster quality. Cluster analysis typically involves trying different numbers of clusters, calculating the corresponding cluster quality index for each number, and then selecting the number of clusters that yields the best quality index as the final result. For example, one can start with a small number of clusters and gradually increase it, re-clustering and recalculating the quality index after each increase. When the quality index reaches its optimum or no longer improves significantly, the increase in the number of clusters is stopped.

[0072] The optimal number of clusters is used as the number of states in the keyword state sequence. This number reflects the number of phoneme units contained in the keyword and also corresponds to the number of states that need to be set in the keyword state sequence in the subsequent network model. By automatically determining the number of states in this way, the network model can adaptively adjust its complexity according to the actual phoneme structure of the keyword, avoiding the limitations of fixed-state-number models.

[0073] It should be noted that during the temporal clustering process, the clustering algorithm automatically identifies the phoneme boundaries of keywords, that is, determines which feature vectors belong to the first phoneme unit, which belong to the second phoneme unit, and so on. This ability to automatically identify phoneme boundaries eliminates the need for manual annotation of phoneme-level information during the modeling process; only keyword-level annotation intervals are required to determine the number of states.

[0074] Understandably, keywords of different lengths will result in different numbers of states after clustering analysis. Short keywords contain fewer phoneme units, resulting in fewer clusters and thus fewer states; long keywords contain more phoneme units, resulting in more clusters and thus more states. This adaptive state number determination mechanism allows the network model to flexibly adapt to keywords of different lengths, avoiding both overly complex modeling for short keywords due to too many states and insufficient modeling for long keywords due to too few states.

[0075] 203. Establish a network model containing keyword state sequences and non-keyword absorption states based on the number of states, wherein each state in the network model is modeled using a Gaussian mixture model; In this embodiment, establishing a network model containing keyword state sequences and non-keyword absorption states based on the number of states includes: assigning feature vectors in the N sets of feature vector sequences to corresponding states according to clustering results, establishing keyword state sequences, and establishing a single non-keyword absorption state; performing Gaussian mixture model parameter estimation on the feature vectors and non-keyword speech segments assigned to each keyword state to obtain Gaussian mixture models corresponding to each keyword state and non-keyword absorption state; and establishing a network model containing keyword state sequences and non-keyword absorption states based on the Gaussian mixture models of the keyword state sequences and non-keyword absorption states, as well as the transition relationships between states.

[0076] Specifically, the state structure of the network model is established based on the number of states determined by cluster analysis. This number of states determines how many states need to be set in the keyword state sequence, with each state corresponding to a phoneme unit in the keyword pronunciation process.

[0077] Based on the number of states and N sets of feature vector sequences, the feature vectors are assigned to corresponding states according to the clustering results. The clustering process has divided the N sets of feature vector sequences into multiple clusters, each cluster corresponding to a phoneme unit. When establishing the state structure, a state is created for each cluster, and then the feature vectors in that cluster are assigned to the corresponding state. For example, if the clustering results show that the first few frames of a sample sequence belong to the first cluster, then the feature vectors of these frames are assigned to the first state; the next few frames belong to the second cluster, then the feature vectors of these frames are assigned to the second state, and so on.

[0078] This allocation establishes a keyword state sequence. This sequence consists of multiple states arranged sequentially, reflecting the temporal order of phoneme units during keyword pronunciation. The first state corresponds to the starting phoneme of the keyword pronunciation, the last state corresponds to the ending phoneme, and the intermediate states correspond to other phoneme units during pronunciation.

[0079] A single non-keyword absorption state is established. This state models all speech content except for the keyword, including words before and after the keyword, silence, ambient noise, etc. Unlike the keyword state sequence, which contains multiple states, the non-keyword absorption state is singular. This design is based on the assumption that all non-keyword content can be represented acoustically using a single, universal state. This state does not need to describe a specific phoneme structure; it only needs to absorb the acoustic features of various non-keywords.

[0080] Gaussian mixture model parameters are estimated for the feature vectors assigned to each keyword state. Each keyword state receives feature vectors from N samples at that phoneme location. Although these feature vectors correspond to the same phoneme unit, they still exhibit certain differences due to factors such as pronunciation variations and recording conditions. The Gaussian mixture model can describe this difference, assuming that the feature vector distribution of a state can be represented by a mixture of multiple Gaussian distributions.

[0081] The parameters of a Gaussian mixture model include mixture weights, a mean vector, and a covariance matrix. The mixture weights represent the importance of each Gaussian component, the mean vector represents the center position of each Gaussian component, and the covariance matrix represents the distribution shape of each Gaussian component. Parameter estimation involves calculating the parameter values ​​that maximize the likelihood probability of the data based on the eigenvector data assigned to that state using statistical methods. A commonly used parameter estimation method is the Expectation-Maximization (EM) algorithm, which iteratively optimizes the parameters to allow the model to better fit the observed data.

[0082] Gaussian mixture model (GMM) parameters are estimated for non-keyword speech segments to obtain the corresponding GMM for the non-keyword absorption state. Non-keyword speech segments refer to the portions outside the labeled intervals in N sample audio files; these portions contain the speech content before and after the keywords. Since the labeled intervals differ among samples, the content of non-keyword speech segments also varies, potentially containing different words and sounds. After extracting features from these segments, GMM parameters are estimated to obtain model parameters that characterize the acoustic properties of non-keywords.

[0083] It should be noted that Gaussian mixture models for non-keyword absorption states typically require a larger number of Gaussian components because they need to cover a variety of different non-keyword sounds. In contrast, Gaussian mixture models for keyword states require relatively fewer Gaussian components because each state only needs to model the acoustic variation of a specific phoneme.

[0084] A network model is established based on the keyword state sequence and the Gaussian mixture model of non-keyword absorption states, as well as the transition relationships between states. The transition relationships between states define the network model's topology, i.e., which states can transition to each other. In this embodiment, the keyword state sequence adopts a left-to-right linear topology, meaning that transitions from the previous state to the current state or the next state are only allowed; jumps or backsliding are not permitted. This linear structure conforms to the temporal characteristics of keyword pronunciation.

[0085] The transition relationship between the non-keyword absorption state and the keyword state sequence is quite flexible. Before the keyword appears, the decoding path can remain in the non-keyword absorption state; when the keyword begins, the path transitions from the non-keyword absorption state to the first state of the keyword state sequence; during keyword pronunciation, the path transitions sequentially within the keyword state sequence; after the keyword ends, the path transitions back to the non-keyword absorption state from the last state of the keyword state sequence. Furthermore, the non-keyword absorption state allows for self-transition, meaning it can remain in that state continuously for multiple frames to accommodate the uncertainty of the length of non-keyword content before and after the keyword.

[0086] It's important to note that when retrieving multiple different keywords simultaneously, a corresponding keyword state sequence can be established for each keyword, and these state sequences are connected in parallel with the same non-keyword absorption state. In this structure, the decoding path can transition between the non-keyword absorption state and any keyword state sequence. For example, if retrieving two keywords simultaneously, the network model includes the state sequence of the first keyword, the state sequence of the second keyword, and a shared non-keyword absorption state. During decoding, the path can enter the first keyword state sequence from the non-keyword absorption state, return to the non-keyword absorption state after recognizing the first keyword, and then enter the second keyword state sequence when the second keyword is encountered. This parallel concatenation method allows for the simultaneous retrieval of multiple keywords in a single decoding process, eliminating the need to retrieve each keyword separately, thus improving retrieval efficiency. This structure allows for the initial establishment of a model with multiple keywords during training, or for the gradual addition of new keyword state sequences during subsequent use. Only the state transition relationships of the network model need to be expanded; retraining the existing keyword model is unnecessary, providing a flexible deployment method.

[0087] After defining the state transition relationships, it is also necessary to estimate the state transition probabilities. State transition probabilities describe the likelihood of transitioning from one state to another. These probabilities can be estimated based on the transition statistics of N sample sequences after state assignment, or they can be initially assigned using uniform initialization and then further optimized during subsequent training.

[0088] By integrating the keyword state sequence, non-keyword absorption states, Gaussian mixture models of each state, and state transition relationships, a complete network model is formed. This network model can describe the phonemic structure features of keywords, model non-keyword content, and define the rules for state evolution during the decoding process through state transition relationships.

[0089] Understandably, this unified network model structure allows the training process to simultaneously optimize keyword modeling and non-keyword modeling, with both working together to improve retrieval performance. During training, the keyword state sequence learns how to accurately represent the acoustic characteristics of the target keyword, while the non-keyword absorption state learns how to absorb various non-keyword contents, enabling automatic selection between keywords and non-keywords during decoding.

[0090] 204. The state transition probability matrix and observation probability distribution of the network model are iteratively trained using the expectation-maximization algorithm to obtain the trained network model; In this embodiment, although the aforementioned process has established the basic structure of the network model and completed preliminary parameter estimation, these parameters are only based on the initial assignment results of the feature vectors and have not yet reached their optimal values. The Expectation-Maximization algorithm, through iterative optimization, continuously adjusts the model parameters given the training data, thereby gradually improving the model's fit to the training data.

[0091] The Expectation-Maximization (EM) algorithm is an iterative algorithm for estimating parameters of probabilistic models containing latent variables. In a network model, latent variables refer to the specific paths of a sequence of feature vectors in the state space, i.e., which state each frame's feature vector corresponds to. Although initial state assignments are obtained through cluster analysis, these assignments are not uniquely determined; the same frame's feature vector may correspond to multiple states, albeit with varying probabilities. The purpose of the Expectation-Maximization algorithm is to find the model parameters that maximize the probability of the observed data occurring, considering all possible paths.

[0092] The Expectation-Maximization (EM) algorithm consists of two alternating steps, called the expectation step and the maximization step. In the expectation step, based on the current model parameters, the posterior probability of each possible state path is calculated. The posterior probability refers to the probability that a certain state path will occur given a known sequence of observed feature vectors. Calculating the posterior probability requires considering both state transition probabilities and observation probabilities; that is, a path not only requires reasonable transitions between states but also a high probability that each state will generate a corresponding feature vector.

[0093] The expectation step calculation requires traversing all possible state paths. Since the product of the number of states and the sequence length can be very large, directly traversing all paths is computationally infeasible. Therefore, the expectation step employs a forward-backward algorithm for efficient computation. The forward algorithm calculates from the beginning of the sequence towards the end, accumulating the probability of reaching each state frame by frame; the backward algorithm calculates from the end of the sequence towards the beginning, accumulating the probability of reaching the end of the sequence frame by frame from each state. Combining the forward and backward probabilities yields the posterior probability of each state at each time step, as well as the posterior probability of the transition between each pair of adjacent states.

[0094] In the maximization step, the model parameters are re-estimated based on the posterior probabilities calculated in the expectation step. The principle of parameter estimation is to maximize the expected log-likelihood of the observed data under the new parameters. Specifically, the update of the state transition probabilities is based on the posterior probability statistics of the state transitions, that is, the new probability of a certain state transition is equal to the posterior expected frequency of that transition in all sample sequences divided by the posterior expected frequency of the initial state. The update of the observation probability distribution is then based on the posterior expected statistics of the eigenvectors in each state, re-estimated for the mean vector, covariance matrix, and mixture weights of the Gaussian mixture model.

[0095] By alternating between expectation steps and maximization steps, the model parameters improve after each iteration. The standard for improvement is a continuous increase in the log-likelihood of the training data, meaning the model fits the training data better and better. The iteration process stops when the increment of the log-likelihood is less than a preset convergence threshold, or when the preset maximum number of iterations is reached. The model parameters obtained at this point are the parameters after training is complete.

[0096] It's important to note that a key characteristic of the Expectation-Maximization (EM) algorithm is that it guarantees the monotonically non-decreasing nature of the log-likelihood. This means that after each iteration, the model's fit to the training data will not worsen; it will only remain constant or improve. This property ensures the algorithm's stability, allowing the training process to reliably converge to a local optimum.

[0097] The trained network model has an optimized state transition probability matrix and observation probability distribution. The state transition probability matrix describes the tendency of transitions between states; for example, in the keyword state sequence, the transition probability between adjacent states is high, and the probability of skipping transitions is low, reflecting the continuity of keyword pronunciation. The observation probability distribution accurately describes the statistical regularity of the feature vectors generated by each state, enabling accurate modeling of each phoneme unit of the keyword.

[0098] The trained network model is capable of effectively decoding the audio to be retrieved. The decoding process utilizes the state transition probabilities and observation probabilities obtained during training to find the optimal path in the network model's state space, thereby achieving accurate keyword retrieval. Since the model parameters have been fully optimized during training, retrieval accuracy is guaranteed.

[0099] 205. Extract features from the audio to be retrieved, input the extracted features into the network model for decoding, and determine the suspected keyword segments based on the probability values ​​of the keyword state sequences in the decoding path; 206. Extract features from the suspected keyword fragment and input them into a pre-trained support vector regression model. Determine whether the suspected keyword fragment is a trained keyword based on the classification score output by the support vector regression model.

[0100] In this embodiment, steps 205-206 are similar to steps 102-103 in the first embodiment, and will not be described again here.

[0101] In this embodiment, feature extraction and cluster analysis are performed on the labeled intervals of N keyword sample audios. Based on the clustering results, the number of states corresponding to each keyword is determined, and a network model containing keyword state sequences and non-keyword absorption states is established. Feature extraction is performed on the audio to be retrieved, and the extracted features are input into the network model for decoding. Suspected keyword segments are determined based on the probability values ​​of the keyword state sequences in the decoding path. Features are extracted from the suspected keyword segments and input into a pre-trained support vector regression model. The classification score output by the support vector regression model determines whether the suspected keyword segment is a trained keyword. This invention automatically determines the number of states by performing temporal cluster analysis on the features of keyword samples, establishing a network model without manual annotation of phoneme information. This makes the method applicable to keyword retrieval in any language, dialect, and even non-verbal sounds, significantly expanding the scope of application and improving retrieval accuracy.

[0102] The above describes the voice keyword retrieval method in the embodiments of the present invention. The following describes the voice keyword retrieval device in the embodiments of the present invention. Please refer to [link to device description]. Figure 3 One embodiment of the voice keyword retrieval device in this invention includes: The model building module 301 is used to extract features and perform cluster analysis on the labeled intervals of N keyword sample audio uploaded by users, determine the number of states corresponding to the keywords based on the clustering results, and build a network model that includes keyword state sequences and non-keyword absorption states. The preliminary retrieval module 302 is used to extract features from the audio to be retrieved, input the extracted features into the network model for decoding, and determine the suspected keyword segments based on the probability values ​​of the keyword state sequence in the decoding path. The verification and judgment module 303 is used to extract features from the suspected keyword fragment and input them into a pre-trained support vector regression model, and to determine whether the suspected keyword fragment is a trained keyword based on the classification score output by the support vector regression model.

[0103] In this embodiment of the invention, the voice keyword retrieval device operates the aforementioned voice keyword retrieval method. The device performs feature extraction and cluster analysis on the labeled intervals of N keyword sample audios, determines the number of states corresponding to each keyword based on the clustering results, and establishes a network model containing keyword state sequences and non-keyword absorption states. Features are extracted from the audio to be retrieved, and the extracted features are input into the network model for decoding. Suspected keyword segments are determined based on the probability values ​​of the keyword state sequences in the decoding path. Features are extracted from the suspected keyword segments and input into a pre-trained support vector regression model. The classification score output by the support vector regression model determines whether the suspected keyword segment is a trained keyword. This invention automatically determines the number of states by performing temporal cluster analysis on the features of keyword samples, establishing a network model without manual annotation of phoneme information. This makes the method applicable to keyword retrieval in any language, dialect, or even non-verbal sounds, significantly expanding the scope of application and improving retrieval accuracy.

[0104] above Figure 3 The voice keyword retrieval device in this embodiment of the invention will be described in detail from the perspective of unitized functional entities. The voice keyword retrieval device in this embodiment of the invention will be described in detail from the perspective of hardware processing.

[0105] Figure 4This is a schematic diagram of the structure of a voice keyword retrieval device 400 provided in an embodiment of the present invention. The voice keyword retrieval device 400 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 410 (e.g., one or more processors) and a memory 420, and one or more storage media 430 (e.g., one or more mass storage devices) for storing application programs 433 or data 432. The memory 420 and storage media 430 can be temporary or persistent storage. The program stored in the storage media 430 may include one or more units (not shown in the diagram), each unit may include a series of instruction operations on the voice keyword retrieval device 400. Furthermore, the processor 410 may be configured to communicate with the storage media 430 and execute the series of instruction operations in the storage media 430 on the voice keyword retrieval device 400 to implement the steps of the aforementioned voice keyword retrieval method.

[0106] The voice keyword retrieval device 400 may also include one or more power supplies 440, one or more wired or wireless network interfaces 450, one or more input / output interfaces 460, and / or one or more operating systems 431, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 4 The illustrated structure of the voice keyword retrieval device does not constitute a limitation on the voice keyword retrieval device provided by the present invention. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0107] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the voice keyword retrieval method.

[0108] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0109] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A voice keyword retrieval method, characterized in that, The voice keyword retrieval method includes: Feature extraction and cluster analysis are performed on the labeled intervals of N keyword sample audio uploaded by users. Based on the clustering results, the number of states corresponding to the keywords is determined, and a network model containing keyword state sequences and non-keyword absorption states is established. Feature extraction is performed on the audio to be retrieved, and the extracted features are input into the network model for decoding. Suspected keyword segments are determined based on the probability values ​​of the keyword state sequences in the decoding path. Feature extraction is performed on the suspected keyword fragments and input into a pre-trained support vector regression model. The classification score output by the support vector regression model is used to determine whether the suspected keyword fragments are training keywords.

2. The voice keyword retrieval method according to claim 1, characterized in that, The process of extracting features and performing cluster analysis on the labeled intervals of N keyword sample audios uploaded by users, determining the number of states corresponding to each keyword based on the clustering results, and establishing a network model that includes keyword state sequences and non-keyword absorption states includes: The labeled intervals of the N keyword sample audios are segmented into frames and features are extracted to obtain N sets of feature vector sequences; Perform time-series cluster analysis on the N sets of feature vector sequences, and determine the number of states in the keyword state sequence based on the clustering results; Based on the number of states, a network model is established that includes a sequence of keyword states and non-keyword absorption states, wherein each state in the network model is modeled using a Gaussian mixture model. The state transition probability matrix and observation probability distribution of the network model are iteratively trained using the expectation-maximization algorithm to obtain the trained network model.

3. The voice keyword retrieval method according to claim 2, characterized in that, The step of performing time-series cluster analysis on the N sets of feature vector sequences and determining the number of states in the keyword state sequence based on the clustering results includes: Calculate the similarity between the N sets of feature vector sequences, and divide the feature vector sequences into multiple clusters based on the similarity; Calculate the intra-cluster variance and inter-cluster distance for each cluster, and evaluate the clustering quality based on the intra-cluster variance and inter-cluster distance; The optimal number of clusters is determined based on the clustering quality, and the optimal number of clusters is used as the number of states in the keyword state sequence.

4. The voice keyword retrieval method according to claim 2, characterized in that, The step of establishing a network model containing keyword state sequences and non-keyword absorption states based on the number of states includes: Based on the number of states and the N sets of feature vector sequences, the feature vectors in the N sets of feature vector sequences are assigned to the corresponding states according to the clustering results, a keyword state sequence is established, and a single non-keyword absorption state is established. Gaussian mixture model parameters are estimated for the feature vectors assigned to each keyword state and the non-keyword speech segments respectively, to obtain the Gaussian mixture models corresponding to each keyword state and non-keyword absorption state. Based on the Gaussian mixture model of the keyword state sequence and the non-keyword absorption state, and the transition relationship between states, a network model containing the keyword state sequence and the non-keyword absorption state is established.

5. The voice keyword retrieval method according to claim 1, characterized in that, The process of extracting features from the audio to be retrieved, inputting the extracted features into the network model for decoding, and determining suspected keyword segments based on the probability values ​​of keyword state sequences in the decoding path includes: The audio to be retrieved is processed by frame segmentation and the Mel frequency cepstral coefficient features are extracted to obtain the feature vector sequence of the audio to be retrieved. The feature vector sequence is input into the network model, and a dynamic programming algorithm is used to decode it in the state space of the network model to obtain the optimal decoding path. Suspected keyword segments are determined based on the time periods during which the probability value of the keyword state sequence in the optimal decoding path is greater than the probability value of other decoding paths.

6. The voice keyword retrieval method according to claim 1, characterized in that, The step of extracting features from the suspected keyword fragments and inputting them into a pre-trained support vector regression model, and determining whether the suspected keyword fragments are training keywords based on the classification scores output by the support vector regression model, includes: The suspected keyword fragments are segmented into frames and features are extracted. The keyword state of each frame is determined according to the decoding path. The features of each frame belonging to the same keyword state are statistically averaged to obtain the state feature vector corresponding to each keyword state. The feature vectors corresponding to each keyword state are concatenated in the order of the state sequence to obtain the feature vector of the suspected keyword fragment. The feature vector of the suspected keyword fragment is input into a pre-trained support vector regression model. The distance from the feature vector to the decision hyperplane is calculated by the support vector regression model to obtain the classification score. The classification score is compared with a preset threshold. When the classification score is greater than the preset threshold, the suspected keyword fragment is determined to be a training keyword, and the time position information of the keyword in the audio to be retrieved is recorded.

7. The voice keyword retrieval method according to claim 1, characterized in that, Before extracting features from the suspected keyword fragments and inputting them into a pre-trained support vector regression model, the method further includes: A preliminary search is performed on the preset verification audio set to obtain preliminary search results; The preliminary search results are manually labeled, with correctly labeled audio segments categorized into a positive sample set and incorrectly labeled audio segments categorized into a negative sample set. Features are extracted from the positive sample set and the negative sample set respectively to obtain a positive sample feature vector set and a negative sample feature vector set; A support vector regression model is trained based on the set of positive sample feature vectors and the set of negative sample feature vectors, so that the set of positive sample feature vectors and the set of negative sample feature vectors form two separable regions in the feature space, and a decision hyperplane is established between the two regions to obtain the trained support vector regression model.

8. A voice keyword retrieval device, characterized in that, The voice keyword retrieval device includes: The model building module is used to extract features and perform cluster analysis on the labeled intervals of N keyword sample audio uploaded by users. Based on the clustering results, the number of states corresponding to the keywords is determined, and a network model containing keyword state sequences and non-keyword absorption states is built. The preliminary retrieval module is used to extract features from the audio to be retrieved, input the extracted features into the network model for decoding, and determine the suspected keyword segments based on the probability values ​​of the keyword state sequence in the decoding path. The verification and judgment module is used to extract features from the suspected keyword fragments and input them into a pre-trained support vector regression model, and determine whether the suspected keyword fragments are trained keywords based on the classification scores output by the support vector regression model.

9. A voice keyword retrieval device, characterized in that, The voice keyword retrieval device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the voice keyword retrieval device to perform the steps of the voice keyword retrieval method as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instruction is executed by the processor, it implements the steps of the voice keyword retrieval method as described in any one of claims 1-7.

Images