A hierarchical forced alignment method and system for incomplete transcribed speech

By employing a hierarchical forced alignment method, combined with CTC posterior and Viterbi decoding, the problem of word-level and phoneme-level temporal boundary localization under conditions of incomplete long-term speech transcription is solved, achieving accurate speech signal alignment.

CN123072399APending Publication Date: 2026-11-10UNIV OF SCI & TECH OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611465160.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-20
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing forced alignment techniques struggle to accurately locate word-level and phoneme-level time boundaries in long conversations due to incomplete transcribed text. This is especially true when there are missing words or weak local speech-text correspondences, leading to severe time boundary errors in traditional methods.

Method used

A hierarchical forced alignment method is adopted, which connects the macro-localization of temporal classification and the local alignment stage of boundary awareness. By using the CTC posterior matrix and Viterbi decoding, combined with frame-level phoneme posterior and target-aware boundary comparison terms, word-level and phoneme-level temporal boundaries are precisely located.

Benefits of technology

It limits the propagation of long-distance boundary errors, improves the local boundary localization capability, reduces training label noise, accommodates local silence and untranscribed speech, and ensures consistency between the global timeline and word order.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123072399A_ABST
    Figure CN123072399A_ABST
Patent Text Reader

Abstract

This invention discloses a hierarchical forced alignment method and system for incomplete transcribed speech, belonging to the field of speech signal processing and speech data annotation. The method includes: Step S1, receiving the speech signal to be aligned, the transcribed text, and a pronunciation dictionary, and converting the transcribed words into a fixed target phoneme sequence through the pronunciation dictionary; Step S2, connecting the temporal classification macro-localization stage to obtain each local speech block; Step S3, obtaining stable local word-level and phoneme-level time boundaries within each local speech block through a boundary-aware local alignment stage; Step S4, adding the start time of the corresponding local speech block to the local word-level and phoneme-level time boundaries within each local speech block, merging them according to the original recording time order; filling empty markers for time intervals not covered by local speech blocks, and outputting the word-level and phoneme-level time boundaries on the complete recording timeline to complete the hierarchical forced alignment. This method can obtain word-level and phoneme-level time boundaries even under incomplete speech transcription conditions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of speech signal processing and speech data annotation technology, and in particular to a hierarchical forced alignment method and system for incomplete transcribed speech. Background Technology

[0002] Forced alignment, given a speech signal and its transcribed text, determines the temporal boundaries of linguistic units such as transcribed words and phonemes within the speech signal. It can be used in scenarios such as speech corpus annotation, phonetics research, and speech model training. Existing forced alignment techniques typically assume that the transcribed text is relatively complete and that the transcribed word sequence closely matches the speech content in the recording. However, in actual recordings such as long conversations, the transcribed text is often incomplete and may contain missing words, especially non-fluent components in the speech signal, such as repetitions, corrections, and filler pauses, which are often difficult to transcribe effectively. There may also be discrepancies between the start and end positions of the transcription and the start and end positions of the recording, gaps in local annotations, or weak transcription-speech correspondences. Traditional forced alignment techniques employ acoustic modeling and decoding based on Hidden Markov Models (HMMs). Transcribed words are converted into a phoneme sequence arranged in a predetermined order using a pronunciation dictionary. The optimal alignment path is then found in the decoding graph constrained by this phoneme sequence using Viterbi decoding, thus determining the temporal boundaries of each phoneme. Since the decoding graph is typically constructed based on the provided transcribed text, when acoustic events in the speech signal are not represented in the transcribed text, the corresponding acoustic frames are difficult to find a natural correspondence in the decoding graph. Traditional forced alignment forcibly assigns these acoustic frames to adjacent transcribed words or phonemes, causing local mismatches and potentially leading to temporal boundary drift. In long recordings or when the transcribed text only sparsely describes the speech content, this problem can manifest as severe long-tailed temporal errors. Existing processing methods for imperfect speech-text correspondences mostly focus on coarse-grained time synchronization or transcription repair, still struggling to simultaneously ensure reliable text-speech localization and fine-grained word-level and phoneme-level boundary determination under incomplete transcription conditions. The connectionist temporal classification (CTC) method, widely used in current automatic speech recognition models, can establish a monotonic correspondence between acoustic frame sequences and transcribed symbol sequences without relying on frame-level supervised labels, thus providing temporal correspondence information between transcribed text and speech signals. However, CTC posterior typically exhibits sparsity and peaking characteristics: most frames correspond to blank symbols, while the target symbol only produces non-blank emissions in a few frames. While non-blank emission moments can be used to identify the approximate location of transcribed text in the speech signal, they cannot directly characterize the complete duration of words or phonemes, nor should they be directly used as the final word-level or phoneme-level temporal boundaries. Therefore, relying solely on CTC emission information usually makes it difficult to simultaneously achieve reliable macroscopic localization and fine-grained local boundary determination in long-term speech. Therefore, the inventors discovered that, under conditions of incomplete transcription of long-term speech and weak local speech-text correspondence, a key technical problem to be solved is how to limit the propagation of global temporal boundary errors caused by untranscribed acoustic regions and improve the localization capability of word-level and phoneme-level temporal boundaries within speech regions reliably supported by the transcribed text. This technical problem comprises two interrelated aspects: first, reliably locating local speech regions corresponding to the transcribed text within complete recordings; and second, precisely locating the temporal boundaries between adjacent words and phonemes within the determined local speech regions using fine-grained acoustic evidence. In view of this, the present invention is hereby proposed. Summary of the Invention

[0003] The purpose of this invention is to provide a hierarchical forced alignment method and system for incomplete transcribed speech. Under the condition of incomplete speech transcription, it can perform hierarchical forced alignment of transcribed words in Mandarin or English by connecting the CTC macro-localization stage and the boundary-aware local alignment stage, thereby obtaining word-level and phoneme-level time boundaries and solving the above-mentioned technical problems existing in the prior art. The objective of this invention is achieved through the following technical solution: A hierarchical forced alignment method for incomplete transcribed speech includes: Step S1: Receive the speech signal to be aligned, the transcribed text, and the pronunciation dictionary; perform normalization on the transcribed text and check whether the transcribed words in the transcribed text exist in the pronunciation dictionary; convert the transcribed words into a fixed target phoneme sequence through the pronunciation dictionary. Step S2, connecting the temporal classification macro-localization stage: The frame-level log-posterior matrix of connection-time classification is obtained by using a connection-time classification phoneme recognition model that matches complete speech signals to the corresponding language. Using the fixed target phoneme sequence obtained in step S1 as the fixed target, a connection temporal classification dynamic programming framework is constructed. The connection temporal classification dynamic programming framework is calculated based on the connection temporal classification frame-level logarithmic posterior matrix, and the optimal framework path is backtracked. The non-blank emission time of each target phoneme is obtained based on the optimal framework path. According to the correspondence between the transcribed word and the target phoneme, the non-blank emission time is assigned to the corresponding transcribed word. The non-blank emission times of the first and last target phonemes of each transcribed word are extended to both sides to form word-level anchor intervals. Then, the local speech blocks are sequentially merged according to the time interval between adjacent word-level anchor intervals to obtain each local speech block, and a continuous transcribed word subsequence corresponding to each local speech block is obtained. Step S3, Boundary-Aware Local Alignment Stage: Multiple local speech blocks are extracted from the complete speech signal. Each local speech block is input into a pre-trained frame-level phoneme classification model, and the local frame-level phoneme posterior matrix is ​​output. The continuous transcribed word subsequences corresponding to each local speech block obtained in step S2 are expanded into local target phoneme sequences according to the pronunciation dictionary, and the word phoneme states are connected according to the transcribed word order to form a constrained pronunciation map; Viterbi decoding is performed on the obtained constrained phoneme map. Based on the local frame-level phoneme posterior matrix, the candidate path score is calculated by combining the state dwell probability, state transition probability, auxiliary non-lexical state cost, and target perception boundary comparison term. For candidate transitions from source phoneme state to target phoneme state, the average log posterior of the local windows before and after the candidate boundary is calculated respectively. When the source phoneme evidence before the boundary is superior to the target phoneme, and the target phoneme evidence after the boundary is superior to the source phoneme, the candidate path score is increased to obtain the first frame-level state path that maintains the order of the transcribed words. The first frame-level state path is folded into continuous state segments. Short silence states or short residual speech states located within the continuous state segments and whose duration does not exceed a predetermined threshold are identified. The identified short auxiliary states are deleted, but all lexical phoneme states are retained. A fixed linear state sequence is constructed based on the remaining lexical phoneme states and the retained auxiliary non-lexical states. Viterbi decoding is performed again under the condition that no new auxiliary states are allowed to be added. The state duration and adjacent boundaries are re-estimated to obtain stable local word-level and phoneme-level time boundaries within each local speech block. Step S4: Add the start time of the corresponding local speech block to the local word-level and phoneme-level time boundaries of each local speech block obtained in step S3, and merge them according to the original recording time order; fill the time intervals not covered by the local speech blocks with empty markers, and verify the consistency between the output transcribed word sequence and the input transcribed word sequence, and finally output the word-level and phoneme-level time boundaries on the complete recording time axis, that is, complete the hierarchical forced alignment of the speech signal to be aligned. A hierarchical forced alignment system for incomplete transcribed speech, used to implement the method described in this invention, includes: The system comprises an input preprocessing module, a connection-based temporal classification macroscopic localization processing module, a boundary-aware local alignment processing module, and a global temporal recovery processing module; among which... The input preprocessing module can receive the speech signal to be aligned, the transcribed text, and the pronunciation dictionary; it performs normalization on the transcribed text and checks whether the transcribed words in the transcribed text exist in the pronunciation dictionary; it converts the transcribed words into a fixed target phoneme sequence through the pronunciation dictionary and outputs it. The connection-time classification macro-localization processing module is communicatively connected to the input preprocessing module and can obtain the connection-time classification frame-level log posterior matrix by using the connection-time classification phoneme recognition model that matches the complete speech signal in the corresponding language. Using the fixed target phoneme sequence output by the input preprocessing module as the fixed target, a connection temporal classification dynamic programming framework is constructed. The connection temporal classification dynamic programming framework is calculated based on the connection temporal classification frame-level logarithmic posterior matrix, and the optimal framework path is backtracked. The set of non-blank emission times for each target phoneme is obtained based on the optimal framework path. According to the correspondence between the transcribed word and the target phoneme, the non-blank emission times in the set of non-blank emission times for each target phoneme are assigned to the corresponding transcribed word. The non-blank emission times of the first and last target phonemes of each transcribed word are extended to both sides to form word-level anchor intervals. Then, the local speech blocks are sequentially merged according to the time interval between adjacent word-level anchor intervals to obtain each local speech block. Each local speech block corresponds to a continuous transcribed word subsequence, and each local speech block is output. The boundary-aware local alignment processing module is communicatively connected to the connection-temporal classification macro-localization processing module. It can extract multiple local speech blocks from the complete speech signal, input each local speech block into a pre-trained frame-level phoneme classification model, and output a local frame-level phoneme posterior matrix. The continuous transcribed word subsequences corresponding to each local speech block output by the connection time-series classification macro-localization processing module are expanded into local target phoneme sequences according to the pronunciation dictionary, and the word phoneme states are connected according to the transcribed word order to form a constrained pronunciation map; Viterbi decoding is performed on the obtained constrained phoneme map. Based on the local frame-level phoneme posterior matrix, the candidate path score is calculated by combining the state dwell probability, state transition probability, auxiliary non-lexical state cost, and target perception boundary comparison term. For candidate transitions from source phoneme state to target phoneme state, the average log posterior of the local windows before and after the candidate boundary is calculated respectively. When the source phoneme evidence before the boundary is superior to the target phoneme, and the target phoneme evidence after the boundary is superior to the source phoneme, the candidate path score is increased to obtain the first frame-level state path that maintains the order of the transcribed words. The first frame-level state path is folded into continuous state segments. Short silence states or short residual speech states located within the continuous state segments and whose duration does not exceed a predetermined threshold are identified. The identified short auxiliary states are deleted, but all lexical phoneme states are retained. A fixed linear state sequence is constructed based on the remaining lexical phoneme states and the retained auxiliary non-lexical states. Viterbi decoding is performed again under the condition that no new auxiliary states are allowed. The state duration and adjacent boundaries are re-estimated, and stable local word-level and phoneme-level time boundaries within each local speech block are obtained and output. The global time recovery processing module is communicatively connected to the boundary-aware local alignment processing module. It can add the start time of the corresponding local speech block to the local word-level and phoneme-level time boundaries of each local speech block output by the boundary-aware local alignment processing module, and merge them according to the original recording time order. It fills empty markers for time intervals not covered by local speech blocks, and verifies the consistency between the output transcribed word sequence and the input transcribed word sequence. Finally, it outputs the word-level and phoneme-level time boundaries on the complete recording timeline, completing the hierarchical forced alignment of the speech signal to be aligned. Compared with existing technologies, the layered forced alignment method and system for incomplete transcribed speech provided by this invention have the following advantages: (1) Limiting the propagation of long-distance boundary errors: The first stage uses macroscopic localization of CTC with transcription constraints to form local speech blocks, which limits the fine boundary search in the second stage to the local range corresponding to consecutive transcribed words, thereby reducing the possibility of local mismatches propagating to the entire recording. (2) Improve local boundary positioning capability: In the second stage, within a local speech block, frame-level phoneme posterior, target-aware boundary comparison term, and constraint sequence decoding are used to determine word-level and phoneme-level temporal boundaries by integrating phoneme evidence on both sides of the candidate boundary. (3) Reduce training label boundary noise: During training, frames that are automatically aligned to both sides of the boundary are set as exclusion labels, and the loss is calculated only on non-exclusion frames, so that potential boundary time biases do not directly participate in model training. (4) It can accommodate partial silence and a small amount of untranscribed speech: The constrained phonetic map allows silence and residual speech states to absorb pauses, fillers, and small amounts of acoustic material not covered by transcription, while maintaining the order constraints of transcribed words and lexical phonemes; this mechanism is not equivalent to automatic completion or recovery of missing text. (5) Suppress local instability caused by short auxiliary states: After the first decoding pass, short auxiliary states are cleaned up, and a second decoding pass is performed on a fixed sequence without adding new auxiliary states to realign the sequences, in order to reduce the impact of transient auxiliary state insertions on local boundaries. (6) Maintain consistency between the global timeline and word order: The local boundaries are restored to the original recording timeline, and time sequence, gap filling, and transcription word order consistency checks are performed to obtain the word-level and phoneme-level time boundaries on the complete recording. Attached Figure Description To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 A flowchart illustrating a layered forced alignment method for incomplete transcribed speech provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the construction of a local voice block based on the CTC transmission anchor point in the method provided in the embodiment of the present invention. Figure 3 This is a flowchart illustrating the training process of the frame-level phoneme classification model in the method provided in this embodiment of the invention; wherein, Indicates exclusion of tags, Indicates a silent state. Indicates the residual speech state. Figure 4 Step S32 of the method provided in the embodiments of the present invention includes a schematic diagram of a constraint pronunciation map for auxiliary non-lexical states; wherein, For the phonemes to be evaluated, Indicates a silent state. Indicates the residual speech state. Figure 5 This is a schematic diagram of the target perception boundary comparison scoring in step S33 of the method provided in the embodiment of the present invention; wherein, the solid line in the logarithmic verification data represents the source phoneme and the dashed line represents the target phoneme. Figure 6 The flowchart shows the short auxiliary state cleanup and second fixed sequence realignment in step S34 of the method provided in the embodiment of the present invention. Figure 7This is a block diagram of a hierarchical forced alignment system for incomplete transcribed speech provided in an embodiment of the present invention. Detailed Implementation The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them, and do not constitute a limitation on the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention. First, the following explanations are provided for the terms that may be used in this article: The term "and / or" means that either or both can be achieved simultaneously. For example, X and / or Y means that it includes both "X" or "Y" as well as the three cases of "X and Y". The terms "comprising," "including," "containing," "having," or other similar semantic descriptions should be interpreted as non-exclusive inclusion. For example, including a technical feature element (such as raw material, component, ingredient, carrier, dosage form, material, size, part, component, mechanism, device, step, process, method, reaction conditions, processing conditions, parameter, algorithm, signal, data, product or article of manufacture, etc.) should be interpreted as including not only the expressly listed technical feature element, but also other technical feature elements that are not expressly listed and are well-known in the art. The term "composed of" excludes any technical features not expressly listed. When used in a claim, it closes the claim to exclude all technical features other than those expressly listed, except for associated conventional impurities. If the term appears only in a clause of a claim, it limits the claim to the elements expressly listed in that clause; elements recited in other clauses are not excluded from the overall claim. Unless otherwise explicitly specified or limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this document according to the specific circumstances. The terms “center,” “longitudinal,” “lateral,” “length,” “width,” “thickness,” “up,” “down,” “front,” “back,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” “outer,” “clockwise,” and “counterclockwise” indicate the current orientation or positional relationship, and are only for the convenience and simplification of description, and do not explicitly or implicitly suggest that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this document. The technical solution provided by this invention will be described in detail below. Contents not described in detail in the embodiments of this invention are prior art known to those skilled in the art. Where specific conditions are not specified in the embodiments of this invention, they shall be performed according to conventional conditions in the art or conditions recommended by the manufacturer. Reagents or instruments used in the embodiments of this invention whose manufacturers are not specified are all conventional products that can be purchased commercially. like Figure 1 As shown, this invention provides a layered forced alignment method for incomplete transcribed speech, comprising the following steps: Step S1: Receive the speech signal to be aligned, the transcribed text, and the pronunciation dictionary; perform normalization on the transcribed text and check whether the transcribed words in the transcribed text exist in the pronunciation dictionary; convert the transcribed words into a fixed target phoneme sequence through the pronunciation dictionary. Step S2, connecting the temporal classification macro-localization stage: The frame-level log-posterior matrix of connection-time classification is obtained by using a connection-time classification phoneme recognition model that matches complete speech signals to the corresponding language. Using the fixed target phoneme sequence obtained in step S1 as the fixed target, a connection temporal classification dynamic programming framework is constructed. The connection temporal classification dynamic programming framework is calculated based on the connection temporal classification frame-level logarithmic posterior matrix, and the optimal framework path is backtracked. The non-blank emission time of each target phoneme is obtained based on the optimal framework path. According to the correspondence between the transcribed word and the target phoneme, the non-blank emission time is assigned to the corresponding transcribed word. The non-blank emission times of the first and last target phonemes of each transcribed word are extended to both sides to form word-level anchor intervals. Then, the local speech blocks are sequentially merged according to the time interval between adjacent word-level anchor intervals to obtain each local speech block, and a continuous transcribed word subsequence corresponding to each local speech block is obtained. Step S3, Boundary-Aware Local Alignment Stage: Multiple local speech blocks are extracted from the complete speech signal. Each local speech block is input into a pre-trained frame-level phoneme classification model, and the local frame-level phoneme posterior matrix is ​​output. The continuous transcribed word subsequences corresponding to each local speech block obtained in step S2 are expanded into local target phoneme sequences according to the pronunciation dictionary, and the word phoneme states are connected according to the transcribed word order to form a constrained pronunciation map; Viterbi decoding is performed on the obtained constrained phoneme map. Based on the local frame-level phoneme posterior matrix, the candidate path score is calculated by combining the state dwell probability, state transition probability, auxiliary non-lexical state cost, and target perception boundary comparison term. For candidate transitions from source phoneme state to target phoneme state, the average log posterior of the local windows before and after the candidate boundary is calculated respectively. When the source phoneme evidence before the boundary is superior to the target phoneme, and the target phoneme evidence after the boundary is superior to the source phoneme, the candidate path score is increased to obtain the first frame-level state path that maintains the order of the transcribed words. The first frame-level state path is folded into continuous state segments. Short silence states or short residual speech states located within the continuous state segments and whose duration does not exceed a predetermined threshold are identified. The identified short auxiliary states are deleted, but all lexical phoneme states are retained. A fixed linear state sequence is constructed based on the remaining lexical phoneme states and the retained auxiliary non-lexical states. Viterbi decoding is performed again under the condition that no new auxiliary states are allowed to be added. The state duration and adjacent boundaries are re-estimated to obtain stable local word-level and phoneme-level time boundaries within each local speech block. Step S4: Add the start time of the corresponding local speech block to the local word-level and phoneme-level time boundaries of each local speech block obtained in step S3, and merge them according to the original recording time order; fill the time intervals not covered by the local speech blocks with empty markers, and verify the consistency between the output transcribed word sequence and the input transcribed word sequence, and finally output the word-level and phoneme-level time boundaries on the complete recording time axis, that is, complete the hierarchical forced alignment of the speech signal to be aligned. Preferably, in step S1 of the above method, the transcribed word is converted into a fixed sequence of target phonemes arranged in a predetermined order using a pronunciation dictionary, including: For each transcribed word, the first candidate pronunciation in the pronunciation dictionary is selected and expanded according to the order of the transcribed word to obtain a fixed target phoneme sequence; for Mandarin and English, text processing rules, pronunciation dictionaries and phoneme sets matched to the corresponding languages ​​are used respectively. Preferably, step S2 of the above method specifically includes: Step S21, obtain the frame-level acoustic posterior matrix for connection-time classification: The complete speech signal is input into the connection-temporal classification phoneme recognition model that matches the corresponding language. The logarithmic posterior of each acoustic frame relative to the phoneme set and blank symbols is calculated to form the connection-temporal classification frame-level logarithmic posterior matrix for macroscopic localization of word-level or phoneme-level time boundaries. Step S22, perform connection-time classification grid alignment of transcription constraints: Using the fixed target phoneme sequence obtained in step S1 as the fixed target, a connection temporal classification dynamic programming framework is constructed. The optimal framework path is calculated and backtracked based on the connection temporal classification frame-level logarithmic posterior matrix to obtain the set of non-blank emission times for each target phoneme. Then, based on the correspondence between the transcribed word and the target phoneme, the non-blank emission times in the set of non-blank emission times for each target phoneme are assigned to the corresponding transcribed word. Step S23: Construct word-level anchors and merge local speech blocks: For each transcribed word, based on the non-blank emission times of the first and last target phonemes of the transcribed word, the interval is expanded to both sides with an expansion radius of 300ms to form a word-level anchor interval; then, the intervals between adjacent word-level anchor intervals are sequentially merged to obtain each local speech block, and each local speech block corresponds to a continuous transcribed word subsequence; The threshold for merging adjacent word-level anchor point intervals is 200ms. Merging is performed when the difference between the start point of the next anchor point and the end point of the previous anchor point is less than or equal to this merging threshold. Preferably, in step S22 of the above method, the optimal grid path is calculated and backtracked based on the connection time-series classification frame-level logarithmic posterior matrix to obtain the set of non-blank transmission times for each target phoneme, which is: (1); In the formula, Indicates the first The set of non-blank emission moments of the target phoneme corresponding to each transcribed word; Indicates the first The first of the 1 transcribed words Non-blank emission moments of each target phoneme; In step S23, based on the non-blank emission times of the first and last target phonemes of the transcribed word, the range is expanded to both sides with an expansion radius of 300ms to form a word-level anchor point interval, which is: (2); In the formula, Indicates the first The word-level anchor point range of each transcribed word. =[ , ], and These are the start and end points of the anchor point interval for that word level, respectively; () represents the function that takes the maximum value; () represents the function that takes the minimum value; Indicates the radius of expansion. =300ms; This indicates the total duration of the complete speech signal input in step S21; In step S23, the condition for sequentially merging adjacent word-level anchor point intervals according to the time interval is as follows: (3); In the formula, This marks the starting point of the next word-level anchor point interval; This is the end point of the previous word-level anchor point interval; This represents the merging threshold for adjacent word-level anchor point intervals. =200ms. In step S23, the set of local speech blocks obtained by sequentially merging adjacent word-level anchor point intervals is as follows: (4); In the formula, Indicates by A collection of local speech blocks; , The first The start and end points of a local speech block; For the first A sequence of consecutive transcribed words corresponding to a local speech block. Preferably, step 3 of the above method specifically includes: Step S31, obtain the local frame-level phoneme posterior matrix: Local speech blocks are extracted from the complete speech signal. Each local speech block is input into a pre-trained frame-level phoneme classification model. The local frame-level phoneme posterior matrix of the word phoneme state, silence state and residual speech state is calculated by outputting the frame shift with a length of 10ms. Step S32, construct a constrained pronunciation graph that includes auxiliary non-lexical states: The continuous transcribed word subsequences corresponding to each local speech block are expanded into local target phoneme sequences according to the pronunciation dictionary. The lexical phoneme states are connected according to the transcribed word order to form a constrained pronunciation graph. When there are multiple candidate pronunciations in the pronunciation dictionary, candidate pronunciation branches are formed in the constrained pronunciation graph, but the order of transcribed words is not changed. Optional silence states and optional residual speech states are set between words, at the start and end points of local speech blocks, and paths that skip auxiliary non-lexical states are allowed. State-specific entry penalties and state biases for auxiliary non-lexical states are set. Step S33, perform the first pass of boundary-aware decoding: Viterbi decoding or its beam search implementation is performed on the constrained phoneme map obtained in step S32. The candidate path score is calculated by combining frame-level phoneme posterior, state dwell probability, state transition probability, auxiliary non-lexical state cost, and target perception boundary comparison term. For candidate transitions from source phoneme state to target phoneme state, the average log posterior of the local windows before and after the candidate boundary is calculated respectively. When the source phoneme evidence in the 3 frames before the boundary is superior to the target phoneme, and the target phoneme evidence in the 3 frames after the boundary is superior to the source phoneme, the candidate path score is increased, and the boundary comparison weight is 200. Step S34: Clean up the short auxiliary states and perform a second pass of fixed sequence realignment: The first frame-level state path executed in step S33 is folded into continuous state segments. Short silence states or short residual speech states located inside the path and whose duration does not exceed a predetermined threshold of 60ms are identified. The identified short auxiliary states are deleted, but all word phoneme states are retained. A fixed linear state sequence is constructed based on the remaining lexical phoneme states and the retained auxiliary non-lexical states. Viterbi decoding is then performed again under the condition that no new auxiliary states are allowed to be added, so as to re-estimate the state duration and adjacent boundaries, and obtain stable local word-level and phoneme-level time boundaries within each local speech block. Preferably, in step S31 of the above method, the frame-level phoneme classification model is pre-trained in the following manner, including: Step T1, construct the initial frame-level phoneme tags: Acquire the phoneme temporal boundaries from the training speech, training transcribed text, and automatic forced aligner, and project the phoneme temporal boundaries onto a fixed frame sequence to form initial frame-level phoneme labels. Step T2, construct boundary frame exclusion labels: Centered on the time boundary of each phoneme, one frame to the left and one frame to the right of the boundary are set as exclusion tags respectively. When the frame shift is 10 ms, 10 ms are excluded on each side, and the total of the left and right sides is 20 ms. Step T3, construct the residual speech category: Randomly select 5% of the training frames that are not silent and have not been set to exclusion labels and relabel them as the category corresponding to the residual speech state; Step T4: Construct a frame-level phoneme classification model: Construct a frame-level phoneme classification model that includes a speech representation encoder and a frame-level classification head, and make the output temporal resolution of the frame-level phoneme classification model suitable for boundary localization. Step T5, perform masked cross-entropy training: The training speech and the supervised sequence containing exclusion labels, lexical phonemes, silence states and residual speech states are input into the frame-level phoneme classification model. Cross-entropy loss is calculated only for frames that are not set as exclusion labels to obtain the second-stage frame-level phoneme classification model for use in step S31. The frame-level phoneme classification model outputs the frame-level posterior of lexical phonemes, silence states and residual speech states. Preferably, in step T5 of the above method, the cross-entropy loss is calculated. ,for: (5); In the formula, To exclude labels; For the first The target category of each training frame; For frame-level phoneme classification models, the target category The given posterior prediction; The set of training frame indices that were not assigned exclusion labels; This is the number of frames in the training frame index set. Preferably, in step S31 of the above method, the frame-level phoneme classification model outputs a frame shift of 10ms to calculate the local frame-level phoneme posterior matrix of the lexical phoneme state, silence state, and residual speech state, which is: (6); In the formula, () represents the set of lexical phonemes; represents the output category of the frame-level phoneme classification model; represents the acoustic feature of the frame; represents the set of phonemes; represents the set of auxiliary non-lexical states; represents the silence state; represents the residual speech state; In step S32, the continuous transcribed word subsequences corresponding to each local speech block are expanded into local target phoneme sequences according to the pronunciation dictionary, as follows: (7); In the formula, represents the local target phoneme sequence corresponding to the i-th local speech block; is the number of target phonemes contained in the local target phoneme sequence corresponding to the i-th local speech block; is the i-th target phoneme in the local target phoneme sequence corresponding to the i-th local speech block; In step S33, the target perception boundary comparison term is: (8); In the formula, The comparison value of the target perception boundary comparison term at the candidate boundary; For the first frame; The source phoneme before the candidate boundary; The target phoneme after the candidate boundary; For the number of boundary context frames, =3 frames, that is, 30ms before and after the boundary; In step S33, the average log-posterior mean of the local windows before and after the candidate boundary is calculated as follows: (9); In the formula, [ , () represents a local frame interval; Phonemes to be evaluated; For the first Frame acoustic characteristics; For the first The frame belongs to the phoneme to be evaluated The posterior probability; In steps S33 and S34, the Viterbi objective function for performing Viterbi decoding is: (10); In the formula, The boundary frame sequence of the local target phoneme sequence. =( 1,…, , Mk+1 ); For the first The initial boundary frame of the target phoneme state; For the first The number of frames in which the state of a target phoneme lasts. = - ; The probability of staying in the Viterbi state; For boundary comparison weights, =200; The boundary sequence that maximizes the objective function. This invention also provides a hierarchical forced alignment system for incomplete transcribed speech to implement the above method, comprising: The system comprises an input preprocessing module, a connection-based temporal classification macroscopic localization processing module, a boundary-aware local alignment processing module, and a global temporal recovery processing module; among which... The input preprocessing module can receive the speech signal to be aligned, the transcribed text, and the pronunciation dictionary; it performs normalization on the transcribed text and checks whether the transcribed words in the transcribed text exist in the pronunciation dictionary; it converts the transcribed words into a fixed target phoneme sequence through the pronunciation dictionary and outputs it. The connection-time classification macro-localization processing module is communicatively connected to the input preprocessing module and can obtain the connection-time classification frame-level log posterior matrix by using the connection-time classification phoneme recognition model that matches the complete speech signal in the corresponding language. Using the fixed target phoneme sequence output by the input preprocessing module as the fixed target, a connection temporal classification dynamic programming framework is constructed. The connection temporal classification dynamic programming framework is calculated based on the connection temporal classification frame-level logarithmic posterior matrix, and the optimal framework path is backtracked. The set of non-blank emission times for each target phoneme is obtained based on the optimal framework path. According to the correspondence between the transcribed word and the target phoneme, the non-blank emission times in the set of non-blank emission times for each target phoneme are assigned to the corresponding transcribed word. The non-blank emission times of the first and last target phonemes of each transcribed word are extended to both sides to form word-level anchor intervals. Then, the local speech blocks are sequentially merged according to the time interval between adjacent word-level anchor intervals to obtain each local speech block. Each local speech block corresponds to a continuous transcribed word subsequence, and each local speech block is output. The boundary-aware local alignment processing module is communicatively connected to the connection-temporal classification macro-localization processing module. It can extract multiple local speech blocks from the complete speech signal, input each local speech block into a pre-trained frame-level phoneme classification model, and output a local frame-level phoneme posterior matrix. The continuous transcribed word subsequences corresponding to each local speech block output by the connection time-series classification macro-localization processing module are expanded into local target phoneme sequences according to the pronunciation dictionary, and the word phoneme states are connected according to the transcribed word order to form a constrained pronunciation map; Viterbi decoding is performed on the obtained constrained phoneme map. Based on the local frame-level phoneme posterior matrix, the candidate path score is calculated by combining the state dwell probability, state transition probability, auxiliary non-lexical state cost, and target perception boundary comparison term. For candidate transitions from source phoneme state to target phoneme state, the average log posterior of the local windows before and after the candidate boundary is calculated respectively. When the source phoneme evidence before the boundary is superior to the target phoneme, and the target phoneme evidence after the boundary is superior to the source phoneme, the candidate path score is increased to obtain the first frame-level state path that maintains the order of the transcribed words. The first frame-level state path is folded into continuous state segments. Short silence states or short residual speech states located within the continuous state segments and whose duration does not exceed a predetermined threshold are identified. The identified short auxiliary states are deleted, but all lexical phoneme states are retained. A fixed linear state sequence is constructed based on the remaining lexical phoneme states and the retained auxiliary non-lexical states. Viterbi decoding is performed again under the condition that no new auxiliary states are allowed. The state duration and adjacent boundaries are re-estimated, and stable local word-level and phoneme-level time boundaries within each local speech block are obtained and output. The global time recovery processing module is communicatively connected to the boundary-aware local alignment processing module. It can add the start time of the corresponding local speech block to the local word-level and phoneme-level time boundaries of each local speech block output by the boundary-aware local alignment processing module, and merge them according to the original recording time order. It fills empty markers for time intervals not covered by local speech blocks, and verifies the consistency between the output transcribed word sequence and the input transcribed word sequence. Finally, it outputs the word-level and phoneme-level time boundaries on the complete recording timeline, completing the hierarchical forced alignment of the speech signal to be aligned. Preferably, in the above system, the boundary-aware local alignment processing module includes: The system comprises a local posterior generation submodule, a constraint graph construction submodule, a boundary-aware decoding submodule, and a fixed sequence realignment submodule; among which... The local posterior generation submodule can extract multiple local speech blocks from the complete speech signal, input each local speech block into a pre-trained frame-level phoneme classification model, and output a local frame-level phoneme posterior matrix. The constraint graph construction submodule is communicatively connected to the connection temporal classification macro-localization processing module. It can expand the continuous transcribed word subsequences corresponding to each local speech block output by the connection temporal classification macro-localization processing module into local target phoneme sequences according to the pronunciation dictionary, and connect the word phoneme states according to the transcribed word order to form a constraint pronunciation graph. The boundary-aware decoding submodule is communicatively connected to the constraint graph construction submodule and the local posterior generation submodule, respectively. It can perform Viterbi decoding on the constraint phonetic graph output by the constraint graph construction submodule. Based on the local frame-level phoneme posterior matrix output by the local posterior generation submodule, it calculates the candidate path score by integrating the state dwell probability, state transition probability, auxiliary non-lexical state cost, and target-aware boundary comparison term. For candidate transitions from the source phoneme state to the target phoneme state, it calculates the average logarithmic posterior of the local windows before and after the candidate boundary. When the source phoneme evidence before the boundary is superior to the target phoneme, and the target phoneme evidence after the boundary is superior to the source phoneme, the candidate path score is increased, and the first frame-level state path that maintains the order of the transcribed words is obtained and output. The fixed sequence realignment submodule is communicatively connected to the boundary-aware decoding submodule. It can fold the first frame-level state path output by the boundary-aware decoding submodule into continuous state segments, identify short silence states or short residual speech states located within the continuous state segments and whose duration does not exceed a predetermined threshold, delete the identified short auxiliary states but retain all lexical phoneme states, construct a fixed linear state sequence based on the remaining lexical phoneme states and the retained auxiliary non-lexical states, and perform Viterbi decoding again under the condition that no new auxiliary states are allowed to be added, re-estimate the state duration and adjacent boundaries, and obtain and output stable local word-level and phoneme-level time boundaries within each local speech block. The system further includes: a model training module, capable of offline training and outputting a frame-level phoneme classification model according to the following steps: Step T1, construct the initial frame-level phoneme tags: Acquire the phoneme temporal boundaries from the training speech, training transcribed text, and automatic forced aligner, and project the phoneme temporal boundaries onto a fixed frame sequence to form initial frame-level phoneme labels. Step T2, construct boundary frame exclusion labels: Centered on the time boundary of each phoneme, set the 1 frame to the left and 1 frame to the right of the boundary as exclusion labels respectively. When the frame shift is 10ms, 10ms is excluded on each side, and the total of the left and right sides is 20ms. Step T3, construct the residual speech category: Randomly select 5% of the training frames that are not silent and have not been set to exclusion labels and relabel them as the category corresponding to the residual speech state; Step T4: Construct a frame-level phoneme classification model: Construct a frame-level phoneme classification model that includes a speech representation encoder and a frame-level classification head, and make the output temporal resolution of the frame-level phoneme classification model suitable for boundary localization. Step T5, perform masked cross-entropy training: The training speech and the supervised sequence containing exclusion labels, lexical phonemes, silence states and residual speech states are input into the frame-level phoneme classification model. Cross-entropy loss is calculated only for frames that are not set as exclusion labels to obtain the second-stage frame-level phoneme classification model for use by the boundary-aware local alignment processing module. The frame-level phoneme classification model outputs the frame-level posterior of lexical phonemes, silence states and residual speech states. In summary, the method and system of the present invention can perform hierarchical forced alignment of Mandarin or English transcribed words by connecting the macro-localization stage of CTC (Connected Temporal Classification) and the boundary-aware local alignment stage under the condition of incomplete speech transcription, thereby obtaining word-level and phoneme-level time boundaries. To more clearly demonstrate the technical solution and its effects provided by this invention, specific embodiments are described in detail below. These embodiments are used to illustrate the two-stage inference process and output format of this invention and are not intended to limit the scope of protection. The models, parameters, and labels used in the embodiments are all verifiable implementations; except for explicitly given single-sample comparisons, they are not used to infer boundary accuracy at the dataset level. Example 1 like Figures 1 to 6 As shown, this embodiment provides a hierarchical forced alignment method for incomplete transcribed speech. This method addresses the problems of easy propagation of global alignment errors and difficulty in accurately determining local phoneme boundaries under incomplete transcription conditions. First, it uses CTC grid alignment with transcription constraints to obtain local speech blocks corresponding to the transcribed text. Then, it uses a frame-level phoneme classification model trained with boundary noise suppression, combined with a constraint pronunciation map including auxiliary non-lexical states and target-aware boundary decoding, to determine the word-level and phoneme-level time boundaries in the local speech blocks. Finally, it generates word-level and phoneme-level time boundaries on the complete recording timeline through short auxiliary state cleanup and second fixed sequence realignment. This invention specifically employs a two-stage hierarchical forced alignment architecture, linking the macro-level text-to-speech (CTC) localization stage on the complete recording with the boundary-aware local alignment stage within local speech blocks, thus enabling the division of labor between macro-level text-to-speech localization and fine-grained boundary determination; wherein... In the first stage, the transcribed word is expanded into a fixed target phoneme sequence according to the first candidate pronunciation in the pronunciation dictionary, CTC grid alignment of the transcription constraint is performed, and sparse non-blank emission moments are expanded and merged into local speech blocks corresponding to the subsequence of the transcribed word. In the second stage of training, one frame to the left and one frame to the right of the phoneme boundary obtained by automatic forced alignment are set as exclusion labels, and the frame-level phoneme classification model is trained using masked cross-entropy loss. At the same time, 5% of the non-silent and non-exclusion frames are relabeled as residual speech categories, so that the model can learn and output the posterior probability of non-target speech to support the processing of untranscribed speech and other content during the decoding process. In the second stage of reasoning, a constrained articulation map is constructed that preserves the order of transcribed words and phonemes while allowing silence and residual speech states to occupy local acoustic frames, in order to absorb pauses, noise, and a small amount of untranscribed speech without generating or recovering missing text. In the first pass of boundary-aware decoding, the target-aware boundary comparison term is added to the Viterbi score. The comparison changes in the posterior probabilities of adjacent phonemes before and after the candidate boundary are used to enhance the boundary evidence of phoneme transfer. That is, the posterior probability of the phoneme before the boundary is higher and the posterior probability of the phoneme after the boundary is higher. Clean up short auxiliary states in the first pass path that do not exceed a predetermined threshold, and perform a second pass realignment on a fixed state sequence where no new auxiliary states are allowed, in order to suppress local boundary instability caused by transient auxiliary insertion. Add the corresponding block start time to the boundaries within each local speech block, merge them into the complete recording timeline, verify the word order, and output word-level and phoneme-level time boundaries. The above technical features form a complete input-processing-output chain: the first stage limits the propagation range of long-distance mismatches; the second stage uses fine-grained posterior and structured decoding to determine local time boundaries within the local speech region supported by transcription; the second realignment further stabilizes local segments; and finally, through global time recovery, word-level and phoneme-level time boundaries on the complete recording timeline are generated. The inputs to the method in this embodiment include: a speech signal to be aligned; a transcribed text corresponding to the speech signal but possibly incomplete; a pronunciation dictionary recording the correspondence between transcribed words and the pronunciation of one or more phonemes; a first-stage CTC phoneme recognition model matched to the language; and a second-stage frame-level phoneme classification model matched to the language. Incomplete transcription includes, but is not limited to: missing words or parts of untranscribed speech; hesitation, repetition, and other speech disfluency; non-lexical acoustic events such as laughter and noise; background speech or other speech not included in the transcription; the start and end positions of the transcription not being completely consistent with the start and end positions of the recording; and gaps in local annotations or intervals with weak transcription-speech correspondence. This method is used for time alignment of provided transcribed words and does not require automatic identification, completion, or recovery of all missing text as a necessary function. The method specifically includes the following steps: Step S1, Receive and preprocess the input: The system receives the speech signal to be aligned, the transcribed text, and a pronunciation dictionary. It performs normalization on the transcribed text and checks if the transcribed words exist in the pronunciation dictionary. For each transcribed word, the first candidate pronunciation in the pronunciation dictionary is consistently selected, and the sequence is expanded according to the transcribed word order to obtain a fixed target phoneme sequence. For Mandarin and English, text processing rules, pronunciation dictionaries, and phoneme sets matched to the respective languages ​​are used. Step S2, connecting the temporal classification macro-localization stage: Step S21, obtain the first-stage CTC frame-level acoustic posterior matrix: The complete speech signal is input into the CTC phoneme recognition model matched with the corresponding language. The logarithmic posterior of each acoustic frame relative to the phoneme set and blank symbols is calculated to form the CTC frame-level logarithmic posterior matrix. This matrix is ​​used for macroscopic localization and does not directly provide the final word-level or phoneme-level time boundaries. Step S22, perform CTC grid alignment of transcription constraints: Using the fixed target phoneme sequence obtained in step S21 as the fixed target, a CTC dynamic programming framework is constructed. The optimal framework path is calculated and backtracked based on the CTC frame-level logarithmic posterior matrix to obtain the set of non-blank emission times for each target phoneme. Then, based on the correspondence between words and target phonemes, the non-blank emission times in the set are assigned to the corresponding transcribed words. Step S23: Construct word-level anchors and merge local speech blocks: For each transcribed word, word-level anchor intervals are formed by expanding outwards based on the non-blank emission times of the first and last target phonemes of the transcribed word. These intervals are then sequentially merged according to the time intervals between adjacent word-level anchor intervals to obtain local speech blocks, each corresponding to a continuous sequence of transcribed words. In a preferred embodiment, the anchor extension radius is 300ms, and the merging threshold for adjacent word-level anchor intervals is 200ms. Merging occurs when the difference between the starting point of the next anchor and the ending point of the previous anchor is less than or equal to this threshold. Word-level anchor intervals and local speech blocks only limit the subsequent local search range and are not considered as final word or phoneme boundaries. Step S3, Boundary-Aware Local Alignment Stage: Step S31, obtain the local frame-level phoneme posterior matrix: Each local speech block is cropped from the complete speech signal and input into a pre-trained frame-level phoneme classification model. The local frame-level phoneme posterior matrix of the word phoneme state, silence state and residual speech state is calculated with a 10ms output frame shift. Step S32, construct a constrained pronunciation graph that includes auxiliary non-lexical states: The continuous sequence of transcribed words corresponding to a local speech block is expanded into a local target phoneme sequence according to the pronunciation dictionary. Lexical phoneme states are connected in the order of the transcribed words. When multiple candidate pronunciations exist in the pronunciation dictionary, candidate pronunciation branches can be formed in the constrained pronunciation graph in the second stage, without changing the order of the transcribed words. Optional silence states and optional residual speech states are set between words, at the start and end points of local speech blocks, and paths that skip auxiliary non-lexical states are allowed. State-specific entry penalties and per-frame state biases are set for auxiliary non-lexical states to limit their overuse. Step S33, perform the first pass of boundary-aware decoding: Viterbi decoding is performed on the constrained phoneme map obtained in step S32 for beam search. Based on the local frame-level phoneme posterior matrix, the candidate path score is calculated by integrating the state dwell probability, state transition probability, auxiliary non-lexical state cost, and target perception boundary contrast term. For candidate transitions from the source phoneme state to the target phoneme state, the average log-posterior of the local windows before and after the candidate boundary is calculated. When the source phoneme evidence before the boundary is superior to the target phoneme evidence, and the target phoneme evidence after the boundary is superior to the source phoneme evidence, the candidate path score is increased. In a preferred embodiment, a boundary context window of 3 frames (30ms) is used before and after the boundary, and the boundary contrast weight is 200. This yields the first frame-level state path that maintains the transcribed word order. Step S34: Clean up the short auxiliary states and perform a second pass of fixed sequence realignment: The first frame-level state path executed in step S33 is folded into continuous state segments. Short silence states or short residual speech states located within the path and whose duration does not exceed a predetermined threshold are identified. The identified short auxiliary states are deleted, but all lexical phoneme states are retained. A fixed linear state sequence is constructed based on the remaining lexical phoneme states and the retained auxiliary non-lexical states. Viterbi decoding is performed again under the condition that no new auxiliary states are allowed to be added, so as to re-estimate the state duration and adjacent boundaries. In a preferred embodiment, the short auxiliary state threshold is 60ms. This results in stable local word-level and phoneme-level time boundaries. Step S4: Restore the global timeline and output word-level and phoneme-level time boundaries: The word-level and phoneme-level time boundaries within each local speech block are added with the start time of the corresponding local speech block, and then merged according to the original recording time order. Time intervals not covered by local speech blocks are filled with empty markers, and the consistency between the output transcribed word sequence and the input transcribed word sequence is verified. Finally, the word-level and phoneme-level time boundaries on the complete recording timeline are output, and can be written to a time alignment annotation file containing word and phoneme layers. The time alignment annotation file can be in TextGrid format. In the above method, the following unified notation is used: the input speech signal is denoted as x(1:T); the N transcribed words arranged in order are denoted as W = (w1,…,w i ,w N ); No. Each transliterated word is denoted as w i The word is denoted as M based on the number of phonemes after the expansion of the first candidate pronunciation. i ;No. The word in the first The CTC non-blank emission time of each target phoneme is denoted as The total duration of the input speech is denoted as D; the word-level anchor point expansion radius is denoted as... The threshold for merging adjacent word-level anchor point intervals is denoted as g. In step S22 above, the set of non-blank emission times of the target phoneme is: (1); In the formula, Indicates the first The set of non-blank emission times of the target phonemes corresponding to each transcribed word. This set is obtained by the optimal path of the CTC lattice in step S21, providing sparse acoustic timing basis for constructing word-level anchors in step S22. In step S23 above, the resulting word-level anchor point interval is: (2); In the formula, =[ , ] indicates the first The word-level anchor point range of each transcribed word. and These represent the start and end points of the word-level anchor point interval, respectively. =300ms. This method extends sparse CTC emission to potentially cover a conservative local search interval for word acoustic implementations, rather than a final boundary estimate. In step S23 above, the condition for sequentially merging according to the time interval between adjacent word-level anchor point intervals is as follows: (3); In the formula, This marks the starting point of the next word-level anchor point interval; This is the end point of the previous word-level anchor point interval; This represents the merging threshold for adjacent word-level anchor point intervals. =200ms. Adjacent word-level anchor intervals that meet this condition are merged into the same local speech block to merge fluent continuous words and retain larger intervals that may correspond to pauses or untranscribed speech. In step S23 above, the set of local speech blocks obtained by sequentially merging adjacent word-level anchor point intervals is as follows: (4); In the formula, Indicates by A collection of local speech blocks; , The first The start and end points of a local speech block; For the first A sequence of consecutive transcribed words corresponding to a local speech block. In step S31 above, the frame-level phoneme classification model outputs a frame shift of 10ms to calculate the local frame-level phoneme posterior matrix for the lexical phoneme state, silence state, and residual speech state, which is: (6); In the formula, () represents the set of lexical phonemes; The output category of the frame-level phoneme classification model; For the first Frame acoustic characteristics; Represents a set of phonemes; To assist the set of non-lexical states; Indicates a silent state; This represents the residual speech state. This posterior provides acoustic evidence for local boundary decoding. In step S32 above, the continuous transcribed word subsequences corresponding to each local speech block are expanded into local target phoneme sequences according to the pronunciation dictionary, as follows: (7); In the formula, Indicates the first The local target phoneme sequence corresponding to each local speech block; For the first The number of target phonemes contained in the local target phoneme sequence corresponding to each local speech block; For the first The local target phoneme sequence corresponding to the local speech block is the first one. One target phoneme. This sequence is used to define the order of lexical phonemes in the second stage. In step S33 above, the target perception boundary comparison term is: (8); In the formula, The comparison value of the target perception boundary comparison term at the candidate boundary; For the first frame; The source phoneme before the candidate boundary; The target phoneme after the candidate boundary; For the number of boundary context frames, =3 frames, i.e., 30ms before and 30ms after the boundary. This formula rewards candidate transitions where the source phoneme is dominant before the boundary and the target phoneme is dominant after the boundary. In step S33 above, the average log-posterior mean of the local windows before and after the candidate boundary is calculated as follows: (9); In the formula, [ , () represents a local frame interval; Phonemes to be evaluated; For the first Frame acoustic characteristics; For the first The frame belongs to the phoneme to be evaluated The posterior probability. In steps S33 and S34 above, the Viterbi objective function for performing Viterbi decoding is: (10); In the formula, The boundary frame sequence of the local target phoneme sequence. =( 1,…, , Mk+1 ); For the first The initial boundary frame of the target phoneme state; For the first The number of frames in which the state of a target phoneme lasts. = - ; The probability of staying in the Viterbi state; For boundary comparison weights, =200; This is the boundary sequence that maximizes the objective function. The formula represents the decoding objective for the lexical phoneme portion; the actual first decoding pass also incorporates entry penalties for silence and residual speech states, as well as state biases for each frame, on the extended constrained phonetic map. In step S3 of the above method, the frame-level phoneme classification model is trained in the following manner, including: Step T1, construct the initial frame-level phoneme tags: The phoneme temporal boundaries generated from training speech, training transcription, and the automatic forced aligner are obtained. These phoneme temporal boundaries are projected onto a fixed frame sequence to form initial frame-level phoneme labels. Mandarin and English employ transcription rules, pronunciation dictionaries, and phoneme sets matched to their respective languages. Mandarin can use an initial consonant-final phoneme organization method, with finals not carrying tone marks. Step T2, construct boundary frame exclusion labels: Centered on each phoneme boundary, set the one frame to the left and one frame to the right of the boundary as exclusion tags respectively. When the frame shift is 10ms, 10ms is excluded from each side, totaling 20ms (0.02s). This exclusion only occurs at the training supervision and loss calculation levels; audio frames are not deleted. Step T3, construct the residual speech category: A subset of frames is randomly selected from the non-silent training frames that are not set as exclusion labels and relabeled with the category corresponding to the residual speech state (sph). In a preferred embodiment, the relabeling ratio is 5%. This category is used to enable the model to learn the residual acoustic material of the speech class, rather than learning the text of the specific words that were omitted. T4 constructs a frame-level phoneme classification model: A frame-level phoneme classification model is constructed, comprising a speech representation encoder and a frame-level classification head, with its output temporal resolution suitable for boundary localization. In a preferred embodiment, the output frame shift of the second-stage model is 10 ms. The speech representation encoder uses wav2vec2-Large. T5 performs masked cross-entropy training: The training speech and the supervised sequence containing exclusion labels, lexical phonemes, silence states, and residual speech states are input into the frame-level phoneme classification model. Cross-entropy loss is calculated only for frames not set as exclusion labels, resulting in the second-stage frame-level phoneme classification model used in step S5. This model outputs the frame-level posterior of lexical phonemes, silence states, and residual speech states. In step T5 above, the cross-entropy loss is calculated. ,for: (5); In the formula, To exclude labels; For the first The target category of each training frame; For frame-level phoneme classification models, the target category The given posterior prediction; The set of training frame indices that were not assigned exclusion labels; This is the number of frames in the training frame index set. The loss is computed only on non-excluded frames, ensuring that potential boundary time noise does not participate in the training loss. This embodiment involves Figures 1-6 middle, Figure 1 The overall flowchart of the hierarchical forced alignment method is shown below: Step S1: Acquire and preprocess the speech signal, transcribed text, and pronunciation dictionary; Step S21: Obtain the first-stage CTC frame-level acoustic posterior; Step S22: Perform CTC grid alignment with transcription constraints; Step S23: Construct word-level anchors and merge them into local speech blocks; Step S31: Obtain the local frame-level phoneme posterior; Step S32: Construct a constrained pronunciation map containing auxiliary non-lexical states; Step S33: Perform the first pass of boundary-aware decoding; Step S34: Clean up short auxiliary states and perform the second pass of fixed sequence realignment; Step S4: Restore the global time axis and output word-level and phoneme-level time boundaries. Figure 2This diagram illustrates the construction of local speech blocks based on CTC (Central Transmission Point) anchors. It shows the complete recording timeline, transcribed words, untranscribed speech or pause areas, non-blank emission times of target phonemes within each transcribed word, word-level anchor intervals extending to both sides, merging relationships between adjacent word-level anchor intervals, and the resulting multiple local speech blocks. The anchor boundaries and local speech block boundaries in the diagram do not represent the final word or phoneme boundaries; larger untranscribed areas do not proceed to the second stage. Figure 3 The flowchart for training the second-stage frame-level phoneme classification model shows the following steps: Step T1: Construct initial frame-level phoneme labels based on the automatic forced alignment results; Step T2: Set one frame to the left and one frame to the right of each phoneme boundary as exclusion labels; Step T3: Relabel 5% of non-silent frames as residual speech categories; Step T4: Construct a frame-level phoneme classification model with an output frame shift of 10ms; Step T5: Train the model using masked cross-entropy loss; and finally, output the second-stage frame-level phoneme classification model. Figure 4 This is a schematic diagram of a constrained phoneme graph including auxiliary non-lexical states. It shows the start node, end node, lexical phoneme states arranged in the order of the transcribed words, candidate phoneme branches in the phoneme dictionary, optional silence states and residual speech states at the start and end positions of words and blocks, and paths that skip auxiliary states. All valid paths maintain the order of the transcribed words and lexical phonemes. Figure 5 This diagram illustrates the target perception boundary comparison scoring, showing the candidate boundary, the context window of the three frames before and three frames after the boundary, the posterior changes of the source phoneme and the target phoneme, and the relationship between the posterior advantage of the source phoneme before the boundary and the posterior advantage of the target phoneme after the boundary, which together form the target perception boundary comparison term. Figure 6 The flowchart for short auxiliary state cleanup and second fixed sequence realignment shows the following steps: obtaining the first decoding state path, folding it into continuous state segments, determining whether the duration of the internal auxiliary state does not exceed 60 ms, deleting short auxiliary states and retaining all word phoneme states, constructing a fixed linear state sequence, performing the second fixed sequence realignment without allowing the addition of new auxiliary states, and obtaining local word-level and phoneme-level time boundaries. Example 2 like Figure 7 As shown, this embodiment provides a hierarchical forced alignment system for incomplete transcribed speech to implement the method of Embodiment 1, comprising: The system comprises an input preprocessing module, a connection-based temporal classification macroscopic localization processing module, a boundary-aware local alignment processing module, and a global temporal recovery processing module; among which... The input preprocessing module can receive the speech signal to be aligned, the transcribed text, and the pronunciation dictionary; it performs normalization on the transcribed text and checks whether the transcribed words in the transcribed text exist in the pronunciation dictionary; it converts the transcribed words into a fixed target phoneme sequence through the pronunciation dictionary and outputs it. The connection-time classification macro-localization processing module is communicatively connected to the input preprocessing module and can obtain the connection-time classification frame-level log posterior matrix by using the connection-time classification phoneme recognition model that matches the complete speech signal in the corresponding language. Using the fixed target phoneme sequence output by the input preprocessing module as the fixed target, a connection temporal classification dynamic programming framework is constructed. The connection temporal classification dynamic programming framework is calculated based on the connection temporal classification frame-level logarithmic posterior matrix, and the optimal framework path is backtracked. The non-blank emission time of each target phoneme is obtained based on the optimal framework path. According to the correspondence between the transcribed word and the target phoneme, the non-blank emission time is assigned to the corresponding transcribed word. The non-blank emission times of the first and last target phonemes of each transcribed word are extended to both sides to form word-level anchor intervals. Then, the local speech blocks are sequentially merged according to the time interval between adjacent word-level anchor intervals to obtain each local speech block, so that each local speech block corresponds to a continuous transcribed word subsequence, and each local speech block is output. The boundary-aware local alignment processing module is communicatively connected to the connection-temporal classification macro-localization processing module. It can extract multiple local speech blocks from the complete speech signal, input each local speech block into a pre-trained frame-level phoneme classification model, and output a local frame-level phoneme posterior matrix. The continuous transcribed word subsequences corresponding to each local speech block output by the connection time-series classification macro-localization processing module are expanded into local target phoneme sequences according to the pronunciation dictionary, and the word phoneme states are connected according to the transcribed word order to form a constrained pronunciation map; Viterbi decoding or its beam search implementation is performed on the obtained constrained phoneme map. The candidate path score is calculated by combining the local frame-level phoneme posterior matrix, state dwell probability, state transition probability, auxiliary non-lexical state cost, and target perception boundary contrast term. For candidate transitions from source phoneme state to target phoneme state, the average log posterior of the local windows before and after the candidate boundary is calculated respectively. When the source phoneme evidence before the boundary is superior to the target phoneme evidence, and the target phoneme evidence after the boundary is superior to the source phoneme evidence, the candidate path score is increased to obtain the first frame-level state path that maintains the order of transcribed words. The first frame-level state path is folded into continuous state segments. Short silence states or short residual speech states located within the continuous state segments and whose duration does not exceed a predetermined threshold are identified. The identified short auxiliary states are deleted, but all lexical phoneme states are retained. A fixed linear state sequence is constructed based on the remaining lexical phoneme states and the retained auxiliary non-lexical states. Viterbi decoding is performed again under the condition that no new auxiliary states are allowed. The state duration and adjacent boundaries are re-estimated, and stable local word-level and phoneme-level time boundaries within each local speech block are obtained and output. The global time recovery processing module is communicatively connected to the boundary-aware local alignment processing module. It can add the start time of the corresponding local speech block to the local word-level and phoneme-level time boundaries of each local speech block output by the boundary-aware local alignment processing module, and merge them according to the original recording time order. It fills empty markers for time intervals not covered by local speech blocks, and verifies the consistency between the output transcribed word sequence and the input transcribed word sequence. Finally, it outputs the word-level and phoneme-level time boundaries on the complete recording timeline, completing the hierarchical forced alignment of the speech signal to be aligned. Furthermore, the boundary-aware local alignment processing module in the above system includes: The system comprises a local posterior generation submodule, a constraint graph construction submodule, a boundary-aware decoding submodule, and a fixed sequence realignment submodule; among which... The local posterior generation submodule can extract multiple local speech blocks from the complete speech signal, input each local speech block into a pre-trained frame-level phoneme classification model, and output a local frame-level phoneme posterior matrix; The constraint graph construction submodule is communicatively connected to the connection temporal classification macro-localization processing module. It can expand the continuous transcribed word subsequences corresponding to each local speech block output by the connection temporal classification macro-localization processing module into local target phoneme sequences according to the pronunciation dictionary, and connect the word phoneme states according to the transcribed word order to form a constraint pronunciation graph. The boundary-aware decoding submodule is communicatively connected to the constraint graph construction submodule and the local posterior generation submodule, respectively. It can perform Viterbi decoding or its beam search implementation on the constraint phonetic graph output by the constraint graph construction submodule. It calculates the candidate path score by combining the local frame-level phoneme posterior matrix, state dwell probability, state transition probability, auxiliary non-lexical state cost, and target-aware boundary comparison term output by the local posterior generation submodule. For candidate transitions from source phoneme state to target phoneme state, it calculates the average logarithmic posterior of the local windows before and after the candidate boundary. When the source phoneme evidence before the boundary is superior to the target phoneme, and the target phoneme evidence after the boundary is superior to the source phoneme, the candidate path score is increased, and the first frame-level state path that maintains the order of transcribed words is obtained and output. The fixed sequence realignment submodule is communicatively connected to the boundary-aware decoding submodule. It can fold the first frame-level state path output by the boundary-aware decoding submodule into continuous state segments, identify short silence states or short residual speech states located inside the path and whose duration does not exceed a predetermined threshold, delete the identified short auxiliary states but retain all lexical phoneme states, construct a fixed linear state sequence based on the remaining lexical phoneme states and the retained auxiliary non-lexical states, and perform Viterbi decoding again under the condition that no new auxiliary states are allowed to be added, re-estimate the state duration and adjacent boundaries, and obtain and output stable local word-level and phoneme-level time boundaries within each local speech block. Furthermore, the system also includes a model training module, which can train offline and output a frame-level phoneme classification model according to steps T1-T5 of Embodiment 1. Each module in this embodiment can be implemented by the processor executing program instructions in the memory. The modules pass fixed target phoneme sequences, local speech blocks, local frame-level phoneme posteriors, constrained articulation maps, state paths, and time boundary data according to the input-output relationship of the corresponding steps. The method and system of this invention are limited to Mandarin Chinese and English. Each language employs its own language-specific transcription text processing rules, pronunciation dictionaries, phoneme sets, a first-stage CTC phoneme recognition model, and a second-stage frame-level phoneme classification model. Mandarin Chinese may use an initial consonant-final vowel phoneme organization method, with finals not carrying tone marks; English uses a phoneme set matching its pronunciation dictionary. If different phoneme symbols or stress representations are used between the two stages, clear mapping rules should be established. The specific implementation information described above is used to support the embodiments and does not limit a particular model name, network structure, vocabulary, or parameters to a unique implementation. The advantages and positive effects of this invention are as follows: (1) Limiting the propagation of long-distance boundary errors: In the first stage, the macroscopic localization of CTC with transcription constraints is used to form local speech blocks, which limits the fine boundary search in the second stage to the local range corresponding to the continuous transcribed words, thereby reducing the possibility of local mismatches propagating to the whole recording from a mechanism perspective. (2) Improving local boundary localization capability: In the second stage, the frame-level phoneme posterior, target-aware boundary comparison term and constraint sequence decoding are used in the local speech block to determine the word-level and phoneme-level time boundaries by comprehensively considering the phoneme evidence on both sides of the candidate boundary. (3) Reducing training label boundary noise: During training, the frames on both sides of the automatically aligned boundary are set as exclusion labels, and the loss is calculated only on non-exclusion frames, so that potential boundary time deviations do not directly participate in model training. (4) Accommodating local silence and a small amount of untranscribed speech: The constrained articulation map allows the silent state and residual speech state to absorb pauses, filling areas and a small amount of acoustic material not covered by transcription, while maintaining the order constraints of transcribed words and lexical phonemes; this mechanism is not equivalent to automatically filling in or restoring missing text. (5) Suppress local instability caused by short auxiliary states: After the first decoding pass, short auxiliary states are cleared, and a second realignment is performed on a fixed sequence without adding new auxiliary states to reduce the impact of transient auxiliary state insertion on local boundaries. (6) Maintain consistency between the global timeline and word order: The local boundaries are restored to the original recording timeline, and time order, gap filling, and transcription word order consistency checks are performed to obtain word-level and phoneme-level time boundaries on the complete recording. Example 3 This embodiment provides a hierarchical forced alignment method for incomplete transcribed speech, specifically a two-stage forced alignment method for incomplete English speech transcription, including: Step S1, Input and Text - Pronunciation Processing: The input was a 60.000s English speech recording in 16kHz mono, 16-bit PCM format. The accompanying transcription dictionary contained 186 ordered words, of which 115 had different word forms, and the pronunciation dictionary covered all 115 word forms. The transcription ended with "in" and included colloquial phenomena such as "um," "uh," and repeated words. It did not require coverage of all audible content in the recording, thus constituting what this application calls an incomplete transcription. The system only aligned the given transcription and did not automatically identify or add new words. In step S1, the transcription is segmented and normalized by whitespace characters. In the first stage, the first candidate pronunciation from the pronunciation dictionary is selected for each word, and the stress numbers after English vowel phonemes are removed, forming a fixed target phoneme sequence. In the second stage, when constructing the constrained pronunciation map, stressed phonemes and multiple candidate pronunciation branches can be retained, but all valid paths maintain the given word order. For example, "so" uses "S OW", "that's" uses "DH AE TS", and the first candidate pronunciation of "fifteen" becomes "FIH FT IY N" after destressing. Step S2, the first stage of connection time series classification macro-localization stage: Processed according to steps S21-S23 of the method in Example 1, in steps S21-S23, the first-stage model outputs the CTC frame-level posterior with a 20ms frame shift, constructs the CTC lattice based on the fixed target phoneme sequence, and backtracks the non-blank emission times of the target phonemes. A word-level emission interval is formed using the emission times of the first and last target phonemes of each word, extending 300ms to the left and right; adjacent word-level anchor point intervals are merged when the interval is less than 200ms. This input yields 10 local speech blocks, and the continuous word subsequences within each block collectively cover all 186 transcribed words. Table 1 shows the correspondence between local speech blocks and transcribed words. . Step S3, the second stage of fine alignment and two-pass decoding: Processed according to steps S31-S34 of the method in Example 1, in steps S31-S34, the second-stage model outputs local frame-level phoneme posteriors with a 10 ms frame shift. A constrained phonemic map is constructed according to the local transcription word order, allowing silence states (sil) or residual speech states (sph) at word and block boundaries, but auxiliary states must not generate new words. The first pass of Viterbi decoding uses a beamwidth of 400, a state dwell probability of 0.92, a boundary contrast weight of 200, and a 30 ms boundary context; the frame bias and entry cost for silence states are both -0.5, and the frame bias for residual speech states is -2.0, and the entry cost is -3.0. The first pass of the state path is folded into continuous state segments, and short internal auxiliary states are deleted. Then, a fixed linear sequence is constructed using the remaining states, ensuring no new auxiliary states are added, and a second pass of realignment is performed. In the current operating parameters, the silence threshold is 60 ms, the residual speech threshold is 60 ms, and the judgment is strictly less than the calculated frame number. With a 10 ms frame shift, the actual deleted segments are internal silence segments not exceeding 60 ms and internal residual speech segments not exceeding 40 ms, respectively. For the first 4.6s verification segment, the first pass yields 69 state segments. After deleting 2 short silence segments and 3 short residual speech segments, the second pass of realignment is completed with 64 fixed states. Step S4, Global Time Recovery and Output: Processing according to step S4 of the method in Example 1, in step S4, the start time of the corresponding local speech block is added to each local boundary to restore the original 60s time axis; the uncovered intervals caused by the block outside and the frame grid tail difference are explicitly represented by NULL. Finally, the TextGrid contains two interval layers, phones and words, both of which continuously cover 0 to 60s without overlap, reversal, or unrepresented gaps. Table 2 shows the alignment results. . For example, in the first local speech block, the word-level interval of "so" is 0.440–0.560s, corresponding to phonemes S (0.440–0.510s) and OW1 (0.510–0.560s); the word-level interval of "that's" is 0.560–0.740s, corresponding to phonemes DH (0.560–0.590s), AE1 (0.590–0.670s), T (0.670–0.710s), and S (0.710–0.740s) respectively. The local acoustic material from 3.770–4.100s is absorbed by the residual speech state and is displayed as [missing] at the word level. Example 4 This embodiment provides a hierarchical forced alignment method for incomplete transcribed speech. It is a two-stage forced alignment method for incomplete transcribed Mandarin speech, including: Step S1, Input and Mandarin Pronunciation Processing: The input was a 4.203938s long, 16 kHz, mono, 16-bit PCM format Mandarin speech, transcribed into the phrase "even the situation where transactions almost stopped," consisting of seven ordered words. The pronunciation dictionary used toneless initial-final phonemes: even / sh en zh ix, appear / ch ux ian, transaction / j iao iy i, almost / j ih u, stop / t ing zh ix, of / de, situation / q ing k uang. Each of the seven words had a dictionary entry, which, when concatenated, yielded 26 target phonemes. An existing tonal reference label writes the apical vowel in "transaction" as ii. However, the two-stage model vocabulary used in this embodiment does not contain ii but does contain iy. Therefore, the symbol mapping from ii to iy is only performed on the model input side, while the original reference label remains unchanged. This processing is used to coordinate the dictionary and model output categories without changing the transcribing word order. Step S2, the first stage of macro-positioning: Processed according to steps S21-S23 of the method in Example 1, in steps S21-S23, the first stage processes the complete speech with a 20 ms frame shift, constructing a transcription constraint CTC lattice for 26 fixed target phonemes. After extending the anchor points by 300 ms on each side and merging them with a 200 ms interval threshold, a local speech block of 0.263–3.760 s is obtained, which contains all 7 transcribed words. Word-by-word CTC emission information is only used as macroscopic localization evidence; for example, the CTC emission interval for "transaction" is 1.5890–2.1120 s, and the geometric mean of the target phoneme emission probability is 0.009275, but this low value is not directly used as the final boundary, which is still determined by the second stage. Step S3, the second stage of fine alignment and output: Processed according to steps S31-S34 of the method in Example 1, in steps S31-S34, the second stage outputs the local frame-level phoneme posterior with a 10ms frame shift, and constructs a constraint pronunciation map according to the order of 7 words and 26 phonemes. The Mandarin fine alignment model in this example includes the silence state sil, but does not include the output categories of the residual speech state sph. Therefore, the constraint map retains the optional sil and disables the optional sph, without manually adding categories without model weight support. The first decoding yields 28 state segments, without deleting short silence segments; the second decoding completes the realignment using a fixed linear sequence composed of all 28 states. After adding a 0.263s block start time to the local boundaries and filling the intervals outside the blocks with NULL, the `phones` layer contains 30 intervals, and the `words` layer contains 11 intervals, including 26 word phoneme intervals, 2 `sil` intervals (total 0.220s), and 2 NULL intervals (total 0.723938s). Both layers continuously cover 0–4.203938s, with 7 words for both input and output in the exact same order. The total CPU single-threaded execution time is 7.473s. Table 3 shows the correspondence between local speech blocks and transcribed words. . The output 26 word phonemes are sh en zh ix ch ux ian j iao iy ijihu ting zh ix deq ing k uang, which are completely consistent with the input dictionary. The word label and phoneme label sequences are identical; the mean absolute difference between word interval endpoints is 10.143 ms, and the mean absolute difference between phoneme interval endpoints is 16.250 ms. Since the tools used to generate the existing reference annotations, their manual review status, and the ground truth attributes have not yet been confirmed, this comparison is only used for structure and endpoint verification in this example and should not be considered as a conclusion regarding the generalization performance of the Mandarin dataset. To facilitate a clear understanding of the proposed solution, the terms used in the above methods are explained as follows: (1) Forced alignment: Given a speech signal and its transcribed text, determine the temporal boundaries of the transcribed language units in the speech. (2) Incomplete transcription: Transcription text that does not fully record all audible language content or non-lexical acoustic events in the speech signal. (3) Layered forced alignment: A forced alignment method that links CTC macro positioning with boundary perception local alignment. (4) CTC macro-positioning stage: The first stage of obtaining phoneme emission anchors and constructing local speech blocks by using a fixed target CTC grid on the complete recording. (5) Boundary-aware local alignment stage: The second stage in which fine boundaries are determined in local speech blocks using frame-level phoneme posterior and boundary-aware decoding. (6) Transcribed text: A sequence of text words that corresponds to the speech signal but may be incomplete. (7) Pronunciation dictionary: Data that records the correspondence between transcribed words and the pronunciation of one or more phonemes. (8) Target phoneme sequence: A phoneme sequence generated according to the transliteration word order and pronunciation dictionary for constrained alignment. (9) Connection Temporal Classification (CTC): A sequence modeling method that establishes a monotonic correspondence between acoustic frame sequences and target symbol sequences. (10) Blank symbol: A special symbol used in the CTC output space to represent a target symbol that has not been emitted; the blank symbol is not equivalent to the mute state. (11) Non-blank emission time: The time and position at which the target phoneme symbol is emitted in the optimal grid path of CTC; this time is only used to form the anchor point and is not directly used as the final phoneme boundary. (12) Word-level anchor interval: a conservative time interval obtained by expanding to both sides based on the emission times of the first and last target phonemes of a word. (13) Local speech block: a local speech interval corresponding to a continuous transcribed word subsequence, obtained by merging one or more adjacent word-level anchor intervals; the boundary of the local speech block is not the final word or phoneme boundary. (14) Frame-level phoneme classification model: A model that outputs lexical phonemes and auxiliary non-lexical state posterior probabilities for local speech blocks frame by frame. (15) Frame-level phoneme posterior: The posterior probability given by the frame-level phoneme classification model for each phoneme or auxiliary state. (16) Lexical phoneme state: The phoneme decoding state determined by the pronunciation sequence of the transcribed word. (17) Auxiliary non-lexical states: Optional decoding states used to indicate silence or untranscribed residual speech acoustic material. (18) Mute state: An auxiliary non-lexical state used to absorb pauses or mute acoustic frames; the mute state is not equivalent to the CTC blank symbol. (19) Residual speech state: An auxiliary non-lexical state used to absorb residual acoustic material of speech that has not been transcribed, and is not used for automatic completion or recovery of missing words. (20) Exclusion label: Assign a supervision label to the frames adjacent to the phoneme boundary so that the corresponding frames do not participate in the training loss; setting an exclusion label does not mean deleting the speech frame. (21) Mask cross-entropy loss: Frame-level classification cross-entropy loss calculated only on non-excluded tag frames. (22) Constrained phonetic graph: Preserves the phoneme order of transliterated words and vocabulary, and allows for decoding graphs in non-vocabulary states. (23) Target perception boundary comparison term: The transfer additional score is calculated based on the posterior changes of the source phoneme and the target phoneme before and after the candidate boundary; this term belongs to the inference decoding score and is not part of the training loss. (24) Boundary context window: The local frame interval before and after the candidate boundary when calculating the target perception boundary contrast term; this window is not the same as the boundary frame exclusion range during the training phase. (25) First pass of boundary-aware decoding: Viterbi decoding performed on a constrained phonetic graph containing optional auxiliary non-lexical states. (26) Short auxiliary state: Internal silence or residual speech state segments whose duration does not exceed a predetermined threshold during the first decoding pass, excluding lexical phoneme states. (27) Second pass fixed sequence realignment: Viterbi decoding is performed again on the fixed state sequence after cleaning up short auxiliary states, where no new auxiliary states are allowed. (28) Global time recovery: Add the start time of the local speech block to the local boundary and merge it into the original recording timeline. (29) Word-level and phoneme-level time boundaries: The start and end times of each transcribed word and each phoneme on the original recording timeline are the specific technical outputs of this invention. (30) Time alignment annotation file: a structured file that records word-level and phoneme-level time intervals; TextGrid is only one implementation format. (31) Mandarin: one of the languages ​​used in this invention, employing transcription processing rules, pronunciation dictionaries, phoneme sets and acoustic models that match Mandarin. (32) English: One of the languages ​​used in this invention, employing transcription processing rules, pronunciation dictionaries, phoneme sets and acoustic models that match English. In summary, this invention, while maintaining a given transcription word order and without automatic word completion, first performs macroscopic localization of long speech, then performs fine alignment within local speech blocks with boundary comparison and short auxiliary state cleanup, ultimately outputting word-level and phoneme-level time boundaries on a unified time axis. Specific languages, phoneme symbols, model structures, and parameters can be adjusted without deviating from the above technical concept. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. A layered forced alignment method for incomplete transcribed speech, characterized in that, include: Step S1: Receive the speech signal to be aligned, the transcribed text, and the pronunciation dictionary; perform normalization on the transcribed text and check whether the transcribed words in the transcribed text exist in the pronunciation dictionary; convert the transcribed words into a fixed target phoneme sequence through the pronunciation dictionary. Step S2, connecting the temporal classification macro-localization stage: The frame-level log-posterior matrix of connection-time classification is obtained by using a connection-time classification phoneme recognition model that matches complete speech signals to the corresponding language. Using the fixed target phoneme sequence obtained in step S1 as the fixed target, a connection temporal classification dynamic programming framework is constructed. The connection temporal classification dynamic programming framework is calculated based on the connection temporal classification frame-level logarithmic posterior matrix, and the optimal framework path is backtracked. The non-blank emission time of each target phoneme is obtained based on the optimal framework path. According to the correspondence between the transcribed word and the target phoneme, the non-blank emission time is assigned to the corresponding transcribed word. The non-blank emission times of the first and last target phonemes of each transcribed word are extended to both sides to form word-level anchor intervals. Then, the local speech blocks are sequentially merged according to the time interval between adjacent word-level anchor intervals to obtain each local speech block, and a continuous transcribed word subsequence corresponding to each local speech block is obtained. Step S3, Boundary-Aware Local Alignment Stage: Multiple local speech blocks are extracted from the complete speech signal. Each local speech block is input into a pre-trained frame-level phoneme classification model, and the local frame-level phoneme posterior matrix is ​​output. The continuous transcribed word subsequences corresponding to each local speech block obtained in step S2 are expanded into local target phoneme sequences according to the pronunciation dictionary, and the word phoneme states are connected according to the transcribed word order to form a constrained pronunciation map; Viterbi decoding is performed on the obtained constrained phoneme map. Based on the local frame-level phoneme posterior matrix, the candidate path score is calculated by combining the state dwell probability, state transition probability, auxiliary non-lexical state cost, and target perception boundary comparison term. For candidate transitions from source phoneme state to target phoneme state, the average log posterior of the local windows before and after the candidate boundary is calculated respectively. When the source phoneme evidence before the boundary is superior to the target phoneme, and the target phoneme evidence after the boundary is superior to the source phoneme, the candidate path score is increased to obtain the first frame-level state path that maintains the order of the transcribed words. The first frame-level state path is folded into continuous state segments. Short silence states or short residual speech states located within the continuous state segments and whose duration does not exceed a predetermined threshold are identified. The identified short auxiliary states are deleted, but all lexical phoneme states are retained. A fixed linear state sequence is constructed based on the remaining lexical phoneme states and the retained auxiliary non-lexical states. Viterbi decoding is performed again under the condition that no new auxiliary states are allowed to be added. The state duration and adjacent boundaries are re-estimated to obtain stable local word-level and phoneme-level time boundaries within each local speech block. Step S4: Add the start time of the corresponding local speech block to the local word-level and phoneme-level time boundaries of each local speech block obtained in step S3, and merge them according to the original recording time order; fill the time intervals not covered by the local speech blocks with empty markers, and verify the consistency between the output transcribed word sequence and the input transcribed word sequence, and finally output the word-level and phoneme-level time boundaries on the complete recording time axis, that is, complete the hierarchical forced alignment of the speech signal to be aligned.

2. The layered forced alignment method for incomplete transcribed speech according to claim 1, characterized in that, In step S1, the transcribed word is converted into a fixed sequence of target phonemes arranged in a predetermined order using a pronunciation dictionary, as follows: For each transcribed word, the first candidate pronunciation in the pronunciation dictionary is selected and expanded according to the order of the transcribed word to obtain a fixed target phoneme sequence; for Mandarin and English, text processing rules, pronunciation dictionaries and phoneme sets matched to the corresponding languages ​​are used respectively.

3. The layered forced alignment method for incomplete transcribed speech according to claim 1 or 2, characterized in that, Step S2 specifically includes: Step S21, obtain the frame-level acoustic posterior matrix for connection-time classification: The complete speech signal is input into the connection-temporal classification phoneme recognition model that matches the corresponding language. The logarithmic posterior of each acoustic frame relative to the phoneme set and blank symbols is calculated to form the connection-temporal classification frame-level logarithmic posterior matrix for macroscopic localization of word-level or phoneme-level time boundaries. Step S22, perform connection-time classification grid alignment of transcription constraints: Using the fixed target phoneme sequence obtained in step S1 as the fixed target, a connection temporal classification dynamic programming framework is constructed. The optimal framework path is calculated and backtracked based on the connection temporal classification frame-level logarithmic posterior matrix to obtain the set of non-blank emission times for each target phoneme. Then, based on the correspondence between the transcribed word and the target phoneme, the non-blank emission times in the set of non-blank emission times for each target phoneme are assigned to the corresponding transcribed word. Step S23: Construct word-level anchors and merge local speech blocks: For each transcribed word, based on the non-blank emission times of the first and last target phonemes of the transcribed word, the interval is expanded to both sides with an expansion radius of 300ms to form a word-level anchor interval; then, the intervals between adjacent word-level anchor intervals are sequentially merged to obtain each local speech block, and each local speech block corresponds to a continuous transcribed word subsequence; The threshold for merging adjacent word-level anchor point intervals is 200ms. Merging is performed when the difference between the start point of the next anchor point and the end point of the previous anchor point is less than or equal to this merging threshold.

4. The layered forced alignment method for incomplete transcribed speech according to claim 3, characterized in that, In step S22, the optimal grid path is calculated and backtracked based on the connection time-series classification frame-level logarithmic posterior matrix to obtain the set of non-blank transmission times for each target phoneme, as follows: (1); In the formula, Indicates the first The set of non-blank emission moments of the target phoneme corresponding to each transcribed word; Indicates the first The first of the 1 transcribed words Non-blank emission moments of each target phoneme; In step S23, based on the non-blank emission times of the first and last target phonemes of the transcribed word, the range is expanded to both sides with an expansion radius of 300ms to form a word-level anchor point interval. ,for: (2); In the formula, Indicates the first The word-level anchor point range of each transcribed word. =[ , ], and These are the start and end points of the anchor point interval for that word level, respectively; () represents the function that takes the maximum value; () represents the function that takes the minimum value; Indicates the radius of expansion. =300ms; This indicates the total duration of the complete speech signal input in step S21; In step S23, the condition for sequentially merging adjacent word-level anchor point intervals according to the time interval is as follows: (3); In the formula, This marks the starting point of the next word-level anchor point interval; This is the end point of the previous word-level anchor point interval; This represents the merging threshold for adjacent word-level anchor point intervals. =200ms. In step S23, the set of local speech blocks obtained by sequentially merging adjacent word-level anchor point intervals is as follows: (4); In the formula, Indicates by A collection of local speech blocks; , The first The start and end points of a local speech block; For the first A sequence of consecutive transcribed words corresponding to a local speech block.

5. The layered forced alignment method for incomplete transcribed speech according to claim 1 or 2, characterized in that, Step 3 specifically includes: Step S31, obtain the local frame-level phoneme posterior matrix: Local speech blocks are extracted from the complete speech signal. Each local speech block is input into a pre-trained frame-level phoneme classification model. The local frame-level phoneme posterior matrix of the word phoneme state, silence state and residual speech state is calculated by outputting the frame shift with a length of 10ms. Step S32, construct a constrained pronunciation graph that includes auxiliary non-lexical states: The continuous transcribed word subsequences corresponding to each local speech block are expanded into local target phoneme sequences according to the pronunciation dictionary. The lexical phoneme states are connected according to the transcribed word order to form a constrained pronunciation graph. When there are multiple candidate pronunciations in the pronunciation dictionary, candidate pronunciation branches are formed in the constrained pronunciation graph, but the order of transcribed words is not changed. Optional silence states and optional residual speech states are set between words, at the start and end points of local speech blocks, and paths that skip auxiliary non-lexical states are allowed. State-specific entry penalties and state biases for auxiliary non-lexical states are set. Step S33, perform the first pass of boundary-aware decoding: Viterbi decoding or its beam search implementation is performed on the constrained phoneme map obtained in step S32. The candidate path score is calculated by combining frame-level phoneme posterior, state dwell probability, state transition probability, auxiliary non-lexical state cost, and target perception boundary comparison term. For candidate transitions from source phoneme state to target phoneme state, the average log posterior of the local windows before and after the candidate boundary is calculated respectively. When the source phoneme evidence in the 3 frames before the boundary is superior to the target phoneme, and the target phoneme evidence in the 3 frames after the boundary is superior to the source phoneme, the candidate path score is increased, and the boundary comparison weight is 200. Step S34: Clean up the short auxiliary states and perform a second pass of fixed sequence realignment: The first frame-level state path executed in step S33 is folded into continuous state segments. Short silence states or short residual speech states located inside the path and whose duration does not exceed a predetermined threshold of 60ms are identified. The identified short auxiliary states are deleted, but all word phoneme states are retained. A fixed linear state sequence is constructed based on the remaining lexical phoneme states and the retained auxiliary non-lexical states. Viterbi decoding is then performed again under the condition that no new auxiliary states are allowed to be added, so as to re-estimate the state duration and adjacent boundaries, and obtain stable local word-level and phoneme-level time boundaries within each local speech block.

6. The layered forced alignment method for incomplete transcribed speech according to claim 5, characterized in that, In step S31, the frame-level phoneme classification model is pre-trained in the following manner, including: Step T1, construct the initial frame-level phoneme tags: Acquire the phoneme temporal boundaries from the training speech, training transcribed text, and automatic forced aligner, and project the phoneme temporal boundaries onto a fixed frame sequence to form initial frame-level phoneme labels. Step T2, construct boundary frame exclusion labels: Centered on the time boundary of each phoneme, one frame to the left and one frame to the right of the boundary are set as exclusion tags respectively. When the frame shift is 10 ms, 10 ms are excluded on each side, and the total of the left and right sides is 20 ms. Step T3, construct the residual speech category: Randomly select 5% of the training frames that are not silent and have not been set to exclusion labels and relabel them as the category corresponding to the residual speech state; Step T4: Construct a frame-level phoneme classification model: Construct a frame-level phoneme classification model that includes a speech representation encoder and a frame-level classification head, and make the output temporal resolution of the frame-level phoneme classification model suitable for boundary localization. Step T5, perform masked cross-entropy training: The training speech and the supervised sequence containing exclusion labels, lexical phonemes, silence states and residual speech states are input into the frame-level phoneme classification model. Cross-entropy loss is calculated only for frames that are not set as exclusion labels to obtain the second-stage frame-level phoneme classification model for use in step S31. The frame-level phoneme classification model outputs the frame-level posterior of lexical phonemes, silence states and residual speech states.

7. The layered forced alignment method for incomplete transcribed speech according to claim 5, characterized in that, In step T5, the cross-entropy loss is calculated. ,for: (5); In the formula, To exclude labels; For the first The target category of each training frame; For frame-level phoneme classification models, the target category The given posterior prediction; The set of training frame indices that were not assigned exclusion labels; This is the number of frames in the training frame index set.

8. The layered forced alignment method for incomplete transcribed speech according to claim 5, characterized in that, In step S31, the frame-level phoneme classification model outputs a frame shift of 10ms to calculate the local frame-level phoneme posterior matrix for the lexical phoneme state, silence state, and residual speech state, which is: (6); In the formula, () represents the set of lexical phonemes; The output category of the frame-level phoneme classification model; For the first Frame acoustic characteristics; Represents a set of phonemes; To assist the set of non-lexical states; Indicates a silent state; Indicates the residual speech state; In step S32, the continuous transcribed word subsequences corresponding to each local speech block are expanded into local target phoneme sequences according to the pronunciation dictionary. ,for: (7); In the formula, Indicates the first The local target phoneme sequence corresponding to each local speech block; For the first The number of target phonemes contained in the local target phoneme sequence corresponding to each local speech block; For the first The local target phoneme sequence corresponding to the local speech block is the first one. One target phoneme; In step S33, the target perception boundary comparison term is: (8); In the formula, The comparison value of the target perception boundary comparison term at the candidate boundary; For the first frame; The source phoneme before the candidate boundary; The target phoneme after the candidate boundary; For the number of boundary context frames, =3 frames, that is, 30ms before and after the boundary; In step S33, the average log-posterior mean of the local windows before and after the candidate boundary is calculated as follows: (9); In the formula, [ , () represents a local frame interval; Phonemes to be evaluated; For the first Frame acoustic characteristics; For the first The frame belongs to the phoneme to be evaluated The posterior probability; In steps S33 and S34, the Viterbi objective function for performing Viterbi decoding is: (10); In the formula, The boundary sequence that maximizes the objective function; {} indicates a function that returns the input parameter that makes the function achieve its maximum value; The boundary frame sequence of the local target phoneme sequence. =( 1,…, , Mk+1 ); For the first The initial boundary frame of the target phoneme state; For the first The number of frames in which the state of a target phoneme lasts. = - ; The probability of staying in the Viterbi state; For boundary comparison weights, =200.

9. A hierarchical forced alignment system for incomplete transcribed speech, used to implement the method of any one of claims 1-8, characterized in that, include: The system comprises an input preprocessing module, a connection-based temporal classification macroscopic localization processing module, a boundary-aware local alignment processing module, and a global temporal recovery processing module; among which... The input preprocessing module can receive the speech signal to be aligned, the transcribed text, and the pronunciation dictionary; it performs normalization on the transcribed text and checks whether the transcribed words in the transcribed text exist in the pronunciation dictionary; it converts the transcribed words into a fixed target phoneme sequence through the pronunciation dictionary and outputs it. The connection-time classification macro-localization processing module is communicatively connected to the input preprocessing module and can obtain the connection-time classification frame-level log posterior matrix by using the connection-time classification phoneme recognition model that matches the complete speech signal in the corresponding language. Using the fixed target phoneme sequence output by the input preprocessing module as the fixed target, a connection temporal classification dynamic programming framework is constructed. The connection temporal classification dynamic programming framework is calculated based on the connection temporal classification frame-level logarithmic posterior matrix, and the optimal framework path is backtracked. The set of non-blank emission times for each target phoneme is obtained based on the optimal framework path. According to the correspondence between the transcribed word and the target phoneme, the non-blank emission times in the set of non-blank emission times for each target phoneme are assigned to the corresponding transcribed word. The non-blank emission times of the first and last target phonemes of each transcribed word are extended to both sides to form word-level anchor intervals. Then, the local speech blocks are sequentially merged according to the time interval between adjacent word-level anchor intervals to obtain each local speech block. Each local speech block corresponds to a continuous transcribed word subsequence, and each local speech block is output. The boundary-aware local alignment processing module is communicatively connected to the connection-temporal classification macro-localization processing module. It can extract multiple local speech blocks from the complete speech signal, input each local speech block into a pre-trained frame-level phoneme classification model, and output a local frame-level phoneme posterior matrix. The continuous transcribed word subsequences corresponding to each local speech block output by the connection time-series classification macro-localization processing module are expanded into local target phoneme sequences according to the pronunciation dictionary, and the word phoneme states are connected according to the transcribed word order to form a constrained pronunciation map; Viterbi decoding is performed on the obtained constrained phoneme map. Based on the local frame-level phoneme posterior matrix, the candidate path score is calculated by combining the state dwell probability, state transition probability, auxiliary non-lexical state cost, and target perception boundary comparison term. For candidate transitions from source phoneme state to target phoneme state, the average log posterior of the local windows before and after the candidate boundary is calculated respectively. When the source phoneme evidence before the boundary is superior to the target phoneme, and the target phoneme evidence after the boundary is superior to the source phoneme, the candidate path score is increased to obtain the first frame-level state path that maintains the order of the transcribed words. The first frame-level state path is folded into continuous state segments. Short silence states or short residual speech states located within the continuous state segments and whose duration does not exceed a predetermined threshold are identified. The identified short auxiliary states are deleted, but all lexical phoneme states are retained. A fixed linear state sequence is constructed based on the remaining lexical phoneme states and the retained auxiliary non-lexical states. Viterbi decoding is performed again under the condition that no new auxiliary states are allowed. The state duration and adjacent boundaries are re-estimated, and stable local word-level and phoneme-level time boundaries within each local speech block are obtained and output. The global time recovery processing module is communicatively connected to the boundary-aware local alignment processing module. It can add the start time of the corresponding local speech block to the local word-level and phoneme-level time boundaries of each local speech block output by the boundary-aware local alignment processing module, and merge them according to the original recording time order. It fills empty markers for time intervals not covered by local speech blocks, and verifies the consistency between the output transcribed word sequence and the input transcribed word sequence. Finally, it outputs the word-level and phoneme-level time boundaries on the complete recording timeline, completing the hierarchical forced alignment of the speech signal to be aligned.

10. The hierarchical forced alignment system for incomplete transcribed speech according to claim 9, characterized in that, The boundary-aware local alignment processing module includes: The system comprises a local posterior generation submodule, a constraint graph construction submodule, a boundary-aware decoding submodule, and a fixed sequence realignment submodule; among which... The local posterior generation submodule can extract multiple local speech blocks from the complete speech signal, input each local speech block into a pre-trained frame-level phoneme classification model, and output a local frame-level phoneme posterior matrix; The constraint graph construction submodule is communicatively connected to the connection temporal classification macro-localization processing module. It can expand the continuous transcribed word subsequences corresponding to each local speech block output by the connection temporal classification macro-localization processing module into local target phoneme sequences according to the pronunciation dictionary, and connect the word phoneme states according to the transcribed word order to form a constraint pronunciation graph. The boundary-aware decoding submodule is communicatively connected to the constraint graph construction submodule and the local posterior generation submodule, respectively. It can perform Viterbi decoding on the constraint phonetic graph output by the constraint graph construction submodule. Based on the local frame-level phoneme posterior matrix output by the local posterior generation submodule, it calculates the candidate path score by integrating the state dwell probability, state transition probability, auxiliary non-lexical state cost, and target-aware boundary comparison term. For candidate transitions from the source phoneme state to the target phoneme state, it calculates the average logarithmic posterior of the local windows before and after the candidate boundary. When the source phoneme evidence before the boundary is superior to the target phoneme, and the target phoneme evidence after the boundary is superior to the source phoneme, the candidate path score is increased, and the first frame-level state path that maintains the order of the transcribed words is obtained and output. The fixed sequence realignment submodule is communicatively connected to the boundary-aware decoding submodule. It can fold the first frame-level state path output by the boundary-aware decoding submodule into continuous state segments, identify short silence states or short residual speech states located within the continuous state segments and whose duration does not exceed a predetermined threshold, delete the identified short auxiliary states but retain all lexical phoneme states, construct a fixed linear state sequence based on the remaining lexical phoneme states and the retained auxiliary non-lexical states, and perform Viterbi decoding again under the condition that no new auxiliary states are allowed to be added, re-estimate the state duration and adjacent boundaries, and obtain and output stable local word-level and phoneme-level time boundaries within each local speech block. The system further includes: a model training module, capable of offline training and outputting a frame-level phoneme classification model according to the following steps: Step T1, construct the initial frame-level phoneme tags: Acquire the phoneme temporal boundaries from the training speech, training transcribed text, and automatic forced aligner, and project the phoneme temporal boundaries onto a fixed frame sequence to form initial frame-level phoneme labels. Step T2, construct boundary frame exclusion labels: Centered on the time boundary of each phoneme, one frame to the left and one frame to the right of the boundary are set as exclusion tags respectively. When the frame shift is 10 ms, 10 ms are excluded on each side, and the total of the left and right sides is 20 ms. Step T3, construct the residual speech category: Randomly select 5% of the training frames that are not silent and have not been set to exclusion labels and relabel them as the category corresponding to the residual speech state; Step T4: Construct a frame-level phoneme classification model: Construct a frame-level phoneme classification model that includes a speech representation encoder and a frame-level classification head, and make the output temporal resolution of the frame-level phoneme classification model suitable for boundary localization. Step T5, perform masked cross-entropy training: The training speech and the supervised sequence containing exclusion labels, lexical phonemes, silence states and residual speech states are input into the frame-level phoneme classification model. Cross-entropy loss is calculated only for frames that are not set as exclusion labels to obtain the second-stage frame-level phoneme classification model for use by the boundary-aware local alignment processing module. The frame-level phoneme classification model outputs the frame-level posterior of lexical phonemes, silence states and residual speech states.