Viterbi algorithm searching method and device, medium and program product

Abstract

The invention relates to the field of algorithms, in particular to a Viterbi algorithm searching method and device, a medium and a program product, and the method comprises the steps: respectively calculating the cost from each state at an n moment to a state 1 at an (n + 1) moment, and obtaining an optimal preorder state; calculating the cost from the state 1 at the n moment to the state 2 to k at the n + 1 moment; respectively calculating the cost from the state 2 at the n moment to the state 2 at the k to the n + 1 moment to obtain an optimal preorder state; respectively calculating the cost from the state 2 at the n moment to the state 3 at the n + 1 moment to the k; and so on, the optimal preorder state of all the states at the n + 1 moment is found. According to the search method, the number of times of search calculation is not increased, but the memory access mode and the cache efficiency are greatly improved, and an experimental structure shows that the calculation time of a new search order is reduced by 30%.

Description

Technical Field

[0001] The present invention relates to the field of algorithms, and in particular to a Viterbi algorithm search method, device, medium, and program product. Background Art

[0002] The Viterbi algorithm is a dynamic programming algorithm used to find the most likely state sequence in a hidden Markov model (HMM). Given a series of observations, it finds the most likely hidden state path that leads to those observations. The algorithm avoids the computational cost of exhaustively enumerating all possible state sequences by recursively calculating the optimal state at each moment and gradually storing the results.

[0003] The core idea of the Viterbi algorithm is to decompose the problem into multiple subproblems, retaining only the most likely state under the current observation at each step, thereby effectively reducing computational complexity. It primarily solves the following problem: given a sequence of observations, find the most likely sequence of hidden states (i.e., the optimal path).

[0004] The Viterbi algorithm is commonly used to solve problems involving time series and state inference, particularly in scenarios using HMM models. It has a wide range of applications, particularly in fields involving sequence data and latent state inference, including: natural language processing (NLP): part-of-speech tagging and speech recognition; bioinformatics: gene sequence alignment and DNA sequence analysis; communications: channel coding and decoding; image and video processing: handwriting recognition; trellis code quantization and optimization; and finance: market state prediction.

[0005] Traditional Viterbi search relies on fixed transition and emission probabilities of a hidden Markov model (HMM). All state transition and emission probabilities are pre-trained and do not dynamically adjust based on context or the current state. This approach offers the advantage of computational simplicity and the efficient calculation of the optimal path using dynamic programming. However, it ignores dynamic changes in context and does not update the probability model in real time as the observation sequence changes.

[0006] However, in practical applications, context affects the probability model. Taking speech recognition as an example, context is extremely important. The current pronunciation, vocabulary, grammar, and even the semantics of the sentence will affect the subsequent speech information. In a fixed HMM model, the emission probability and transition probability are static, which has certain limitations. After considering the dynamic changes in context, the adjustment of the probability model will make the decoding process more accurate. Therefore, based on the traditional Viterbi search algorithm, the Viterbi algorithm was developed to dynamically update the probability model.

[0007] For example, several enhanced decoding techniques used in modern speech recognition can be regarded as extensions of the traditional Viterbi algorithm, including: Adaptive Viterbi Search; Context-Dependent Viterbi Search; Neural Network Decoding with Viterbi Search; Beam Search with Dynamic Model Updates.

[0008] For example, in video coding, the Rate Distortion Optimization Quantization (RDOQ) technology widely used in AV1 and VVC also dynamically updates the probability model based on changes in context.

[0009] These methods all introduce context-dependent or dynamic model adjustments. Essentially, they all involve some kind of update to the probabilistic model, differing only in the specific techniques. The evolution from the traditional Viterbi algorithm, which doesn't update the probabilistic model, to one that dynamically updates the probabilistic model based on contextual information has led to significant memory usage in both software and hardware implementations.

[0010] Dynamically updating the probability model usually means that we need to store not only the search path, but also the context-related information of each node, the dynamic probability update status and the candidate path status.

[0011] Taking speech recognition as an example, voice is an increasingly important medium for human-computer interaction, and voice interfaces using automatic speech recognition (ASR) are now commonplace in a wide range of applications. Currently, the most widely used technology for large-vocabulary ASR is cloud-based speech recognition. This involves uploading extracted speech samples or features to a cloud-based computing server, where computationally intensive algorithms convert the speech into text, and the resulting words are streamed back to the user's device. This requires a stable, low-latency, and high-bandwidth internet connection. However, in applications such as ATMs or kiosks, maintaining a high-speed, low-latency internet connection simply for decoding speech is difficult. This can also lead to significant battery drain on portable devices, especially in areas with poor network connectivity, requiring analog radio chains to operate at higher power levels. Finally, the security of uploaded voice data is a concern, particularly for medical and military applications.

[0012] Another approach is fully offline recognition, where the entire computation is performed on the edge device. For large-vocabulary continuous speech recognition systems, the models are often too large to fit in the embedded device's memory, necessitating the use of DRAM, which can accommodate large amounts of data but has higher latency. On-chip memory, however, offers very low latency, but its low storage capacity makes it unsuitable for storing complete language models.

[0013] In addition to storing the model, LVCSR systems typically employ the Viterbi algorithm. This algorithm operates by searching a large state space, evolving this list with each successive frame by maintaining a list of currently active states and updating state probabilities. This list of active states can often grow to thousands of elements and requires periodic pruning. This typically involves processing all current states to obtain candidate states for the next frame, then using some beamwidth parameter to retain the best N states.

[0014] From the perspective of hardware accelerators for embedded applications, the context-dependent Viterbi algorithm presents several new problems. For example, we have the following:

[0015] Large target vocabularies may involve over a million states and may result in thousands of intermediate active states being computed before pruning is applied.

[0016] If we try to set a hard upper bound on the number of states (storing the top N states instead of the best N states) to fit within fixed hardware resources, it may lead to a drastic drop in recognition accuracy (as discussed later in Section 5).

[0017] The energy cost of DRAM access is much higher than on-chip SRAM.

[0018] This incurs high energy and latency costs since speech models in DRAM are accessed in unpredictable patterns based on input.

[0019] Having a large active state list size results in acoustic evaluation of many intermediate states, leading to high computational cost.

[0020] From the above observations, we can see that memory management and the ability to decode speech using limited memory are key issues that need to be addressed in resource-constrained automatic speech recognition implementations.

[0021] In order to better illustrate the problems existing in the prior art, the following first describes in detail a Viterbi search process and the problems encountered during the search process.

[0022] like Figure 1As shown in the figure, for any moment n and the next moment n+1 after n, there are 4 states. State1 to 4 in the figure represent states 1 to 4 respectively; the red blocks represent the cost corresponding to each state; the white blocks represent the states passed by the previous path, and the solid color blocks above the white blocks, such as yellow, blue, and green blocks, represent the context information corresponding to the current state. Figure 1 As can be seen from the example, the context information corresponding to each state may be different. Each time the optimal path for each state at time n+1 is calculated, the corresponding context information needs to be updated, which involves a large amount of context data copying. Since the Viterbi search process is called repeatedly, when the number of states is large, this data copying will occupy a large amount of memory bandwidth and consume power. Usually, separate space is allocated for all states at time n and time n+1 to store context information. The search order is as follows: Figure 2 As shown in the figure; first, the calculation of state 1 at time n+1 is performed. The green line in the figure represents a calculation. The cost of transferring from various states to state 1 at time n+1 is calculated, all costs are compared, and the smallest one is found. Then, as shown by the red line, the calculation is switched to state 2 at time n+1, and so on, until the last state is calculated; after the search is completed, the context information is updated; after the update is completed, the memory space used to store the context information and the previous path at time n is reused to store the information at the next time n+2.

[0023] Then we can see that the total storage space requirement is:

[0024]

[0025] in,

[0026] Indicates the total storage space requirement;

[0027] Indicates the total number of states at each moment.

[0028] Represents the preceding path storage space of each state at each moment;

[0029] Represents the optimal cost storage space for each state at each moment;

[0030] A storage space for contextual information representing each state at each moment.

[0031] The reason for multiplying by 2 is that the search requires storage space for two adjacent moments in the search.

[0032] This search method has the problem of low cache efficiency for the context-dependent Viterbi algorithm, because every time a state at time n+1 is calculated, such as the jth state, all the dependent context information must be used again. For applications such as speech recognition, since there are many states and the amount of dependent context information is large, cache thrashing will occur, reducing efficiency. Moreover, since the context information must also be updated one by one, for example, at time n+1, if the best path for state 1 is to transfer from state 2 at time n, then the context information of state 1 at time n+1 needs to be inherited from state 2 at time n and added with its own new information. If new space is allocated to store context information for each state at time n+1, then an extra memory space is required (this is why it is multiplied by 2 in the formula). This space is very large. If the storage space of the previous moment is reused, the context information at time n cannot be updated to the new context information at time n+1 before the state transfer at time n+1 is calculated. This can be solved by Figure 1 Take this as an example to illustrate this problem; Figure 1 In the example, states 2 and 3 at time n+1 are both derived from state 2 at time n. After calculating state 2 at time n+1, since its best predecessor state is state 2 at time n, the context information storage space of state 2 at time n cannot be immediately reused to update the context information of state 2 at time n+1 and its own information. This is because the next calculation of the best predecessor states of states 3 and 4 at time n+1 also requires the context information of state 2 at time n. In other words, the state transition calculation at time n+1 will form a dependency on the state at time n. Therefore, data update can only be performed after these dependencies are removed. Summary of the Invention

[0033] The present invention addresses the above-described problem by starting with the idea of localizing data as much as possible, dividing the state and changing the search order to achieve the purpose of reducing memory requirements and improving memory access efficiency.

[0034] A Viterbi algorithm search method, when searching between any time n and the next time n+1 after n, includes the following steps:

[0035] S1: Find the optimal preceding state of state 1 at time n+1:

[0036] Calculate the cost of each state at time n to state 1 at time n+1, select the smallest cost as the current optimal cost of state 1 at time n+1, and the corresponding state as the optimal predecessor state of state 1 at time n+1;

[0037] Calculate the cost from state 1 at time n to state 2 to k at time n+1 as the current optimal cost of state 2 to k at time n+1. Correspondingly, state 1 is also used as the optimal previous state of state 2 to k at time n+1.

[0038] S2: Find the optimal preceding state of state 2 at time n+1:

[0039] Calculate the cost of state 2 at time n to state 2 at time k to n+1 respectively. If it is less than the current optimal cost of state 2 at time n+1, update the current optimal cost and optimal previous state of state 2 at time n+1.

[0040] Calculate the cost of state 2 at time n to state 3 to k at time n+1 respectively. If it is less than the current optimal cost of the corresponding state at time n+1, update the current optimal cost and optimal previous state of the corresponding state at time n+1.

[0041] S3: And so on, find the optimal preceding state of all states at time n+1;

[0042] State i (i∈[1,K]) represents the i-th state, Indicates the total number of states at each moment.

[0043] Furthermore, the Viterbi algorithm is a context-dependent Viterbi algorithm;

[0044] S1 also includes: allocating space for all states at time n:

[0045] The space allocated for state i includes:

[0046] Nq, C, C1, C2, C3;

[0047] in,

[0048] Represents the preceding path storage space of each state at each moment;

[0049] Represents the optimal cost storage space for each state at each moment;

[0050] Represents the optimal previous state storage space for each state at each moment;

[0051] A storage space for dependency counters representing each state at each moment;

[0052] A storage space for contextual information representing each state at each moment;

[0053] Set a dependency counter for each state at time n. The dependency counter of state i is D(i), which means that state i at time n is dependent on several states at time n+1. The initial value of D(i) is K, which means that state i at time n is initially dependent on all states at time n+1.

[0054] Assign C1 and C2 to each state at time n+1;

[0055] When calculating the cost from each state at time n to state 1 at time n+1, if each state at time n is not the optimal predecessor state of state 1 at time n+1, then its corresponding D(i) is subtracted by 1;

[0056] If the optimal preceding state of any state at the subsequent n+1 time is replaced by state 1 to another state, then D(1) is reduced by 1;

[0057] S2 also includes:

[0058] Calculate the cost of states 2 to k at time n to state 2 at time n+1 respectively, and when updating, if states 2 to k at time n are not the best predecessor states of state 1 at time n+1, then the corresponding D(i) is subtracted by 1;

[0059] Calculate the cost of state 2 at time n to state 3 to k at time n+1 respectively, and when updating, if there is no update after each comparison, then D(2) is reduced by 1; in the subsequent process, if the best previous state of any state at time n+1 is replaced by state 2 to other states, then D(2) is reduced by 1;

[0060] Whenever D(i) = 0 at state i at time n, C, Nq, and C3 at state i at time n are released for use by state allocation C, Nq, and C3 at time n+1;

[0061] Whenever a state at time n+1 is assigned to C, Nq, or C3, the corresponding data is updated. After the update is completed, D(i) of state i at time n corresponding to the optimal predecessor state of the state is reduced by 1.

[0062] The present invention also provides an electronic device,

[0063] include:

[0064] Memory for storing computer programs;

[0065] The processor is configured to implement the above method steps when executing the program stored in the memory.

[0066] The present invention also provides a computer-readable storage medium,

[0067] The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above method steps are implemented.

[0068] The present invention also provides a computer program product, comprising a computer program, which implements the above method when executed by a processor.

[0069] Beneficial Effects: The search method of the present invention does not increase the number of search calculations, but significantly improves memory access patterns and cache efficiency. Experimental results show that the new search order reduces computation time by 30%. The above search method can be applied to all Viterbi algorithm scenarios. In further context-dependent Viterbi algorithms, combined with the memory allocation method of the present invention, memory usage can be significantly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 Memory allocation diagram for traditional Viterbi algorithm search;

[0071] Figure 2 This is the search order diagram for the traditional Viterbi algorithm search;

[0072] Figure 3 This is the first step of the search method embodiment of the present invention;

[0073] Figure 4 This is the second step of the search method embodiment of the present invention;

[0074] Figure 5 This is the third step of the search method embodiment of the present invention;

[0075] Figure 6 This is the fourth step of the search method embodiment of the present invention;

[0076] Figure 7 This is the fifth step of the search method embodiment of the present invention. DETAILED DESCRIPTION

[0077] In order to make the purposes, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0078] A Viterbi algorithm search method, when searching between any time n and the next time n+1 after n, includes the following steps:

[0079] S1: Allocate space for all states at time n:

[0080] The space allocated to state i (i∈[1,K]) includes:

[0081] Nq, C, C1, C2, C3;

[0082] Among them, state i represents the i-th state;

[0083] Indicates the total number of states at each moment;

[0084] Represents the preceding path storage space of each state at each moment;

[0085] Represents the optimal cost storage space for each state at each moment;

[0086] Represents the optimal previous state storage space for each state at each moment;

[0087] A storage space for dependency counters representing each state at each moment;

[0088] A storage space for contextual information representing each state at each moment.

[0089] Set a dependency counter for each state at time n. The dependency counter of state i is D(i), which means that state i at time n is dependent on several states at time n+1. The initial value of D(i) is K, which means that state i at time n is initially dependent on all states at time n+1.

[0090] Find the optimal preceding state of state 1 at time n+1:

[0091] Assign C1 and C2 to each state at time n+1;

[0092] Calculate the cost of each state at time n to state 1 at time n+1 respectively, select the smallest cost as the current optimal cost of state 1 at time n+1, and the corresponding state as the optimal predecessor state of state 1 at time n+1; in this process, if each state at time n is not the optimal predecessor state of state 1 at time n+1, then its corresponding D(i) is subtracted by 1;

[0093] Calculate the cost from state 1 at time n to state 2 to k at time n+1 as the current optimal cost of state 2 to k at time n+1 (this is different from the traditional algorithm). Correspondingly, state 1 is also used as the optimal predecessor state of state 2 to k at time n+1. If the optimal predecessor state of any state at the subsequent n+1 time is replaced by state 1, then D(1) is reduced by 1.

[0094] S2: Find the optimal preceding state of state 2 at time n+1:

[0095] Calculate the cost of state 2 to k at time n to state 2 at time n+1 respectively. If it is less than the current optimal cost of state 2 at time n+1, update the current optimal cost and optimal predecessor state of state 2 at time n+1. In this process, if state 2 to k at time n is not the optimal predecessor state of state 1 at time n+1, then subtract 1 from its corresponding D(i).

[0096] Calculate the cost of state 2 at time n to state 3 to k at time n+1 respectively. If it is smaller than the current optimal cost of the corresponding state at time n+1, update the current optimal cost and optimal previous state of the corresponding state at time n+1. In this process, if there is no update after each comparison, D(2) is reduced by 1. In the subsequent process, if the optimal previous state of any state at time n+1 is replaced by state 2 to other states, D(2) is reduced by 1.

[0097] S3: And so on, find the optimal preceding state of all states at time n+1;

[0098] Whenever D(i) = 0 at state i at time n, C, Nq, and C3 at state i at time n are released for use by state allocation C, Nq, and C3 at time n+1;

[0099] Whenever a state at time n+1 is assigned to C, Nq, or C3, the corresponding data is updated. After the update is completed, D(i) of state i at time n corresponding to the optimal predecessor state of the state is reduced by 1.

[0100] In the above embodiment, simply using the above search method does not increase the number of search calculations, but significantly improves memory access patterns and cache efficiency. Experimental results show that the new search order reduces calculation time by 30%. The above search method can be applied to all Viterbi algorithm scenarios.

[0101] Furthermore, in a context-dependent Viterbi algorithm, combined with the aforementioned memory allocation method, memory usage can be significantly reduced. This is because, compared to the memory allocation method of the traditional context-dependent Viterbi algorithm, the present invention allocates two additional spaces, C2 and C3, for each state to store the optimal cost and optimal sequence. However, the state at time n+1 no longer requires separate allocations of C and Nq; it can simply be reused after the memory is freed up at time n. Furthermore, the space required for C and Nq is typically much larger than that required for C2 and C3, thus significantly saving memory space.

[0102] like Figures 3 to 7 As shown, taking K as 5 as an example, the whole process is described:

[0103] like Figure 3 As shown in the figure, the first step searches for the optimal predecessor state for state 1 at time n+1. This requires information about every link from state 1 to state 5 at time n. After this step, the cost of transitioning from the predecessor state 1 to states 2 through 5 at time n+1 is immediately calculated (since the cost of transitioning from state 1 at time n to state 1 at time n+1 has already been calculated). The minimum cost and the best predecessor state to date are recorded, respectively. The calculations in this step are all represented by green lines. After the calculations are complete, the search switches to the optimal predecessor state for state 2 at time n+1.

[0104] like Figure 4 As shown in the figure, the second step searches for the optimal predecessor state for state 2 at time n+1. This requires information about every connection from state 2 to state 5 at time n. Note that the cost from state 1 at time n to state 2 at time n+1 has already been calculated in the previous step and stored in the minimum cost of state 2 at time n+1. During the second step, this cost is compared with the minimum cost of state 2 at time n+1. If the minimum cost is smaller, the cost is updated immediately. After the calculation is complete, the cost from the previous state 2 to each state at time n+1 is immediately calculated (note: the cost from state 2 to state 1 at time n+1 has already been calculated and does not need to be calculated). The minimum cost and the optimal predecessor state to date are recorded for each transition from state 2 to state 5 at time n+1. The calculations in this step are all represented by red lines. The green dashed line in the figure represents the previously calculated cost, so it does not need to be recalculated. After the calculations represented by the red line are completed, the transition to state 3 at time n+1 is completed.

[0105] The contents of the third, fourth, and fifth steps are similar and will not be repeated here.

[0106] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A Viterbi algorithm search method, characterized in that: When searching between any moment n and the next moment n+1, The following steps are involved: S1: Find the optimal preceding state of state 1 at time n+1: Calculate the cost of each state at time n to state 1 at time n+1 respectively, select the smallest cost as the current optimal cost of state 1 at time n+1, and the corresponding state as the optimal predecessor state of state 1 at time n+1; Calculate the cost from state 1 at time n to state 2 to k at time n+1 as the current optimal cost of state 2 to k at time n+1. Correspondingly, state 1 is also used as the optimal previous state of state 2 to k at time n+1. S2: Find the optimal preceding state of state 2 at time n+1: Calculate the cost of state 2 at time n to state 2 at time k to n+1 respectively. If it is less than the current optimal cost of state 2 at time n+1, update the current optimal cost and optimal previous state of state 2 at time n+1. Calculate the cost of state 2 at time n to state 3 to k at time n+1 respectively. If it is less than the current optimal cost of the corresponding state at time n+1, update the current optimal cost and optimal previous state of the corresponding state at time n+1. S3: And so on, find the optimal preceding state of all states at time n+1; State i (i∈[1,K]) represents the i-th state, Indicates the total number of states at each moment.

2. The Viterbi algorithm search method according to claim 1, wherein: The Viterbi algorithm is a context-dependent Viterbi algorithm; S1 also includes: allocating space for all states at time n: The space allocated to state i (i∈[1,K]) includes: Nq, C, C1, C2, C3; in, Represents the preceding path storage space of each state at each moment; Represents the optimal cost storage space for each state at each moment; Represents the optimal previous state storage space for each state at each moment; A storage space for dependency counters representing each state at each moment; A storage space for contextual information representing each state at each moment; Set a dependency counter for each state at time n. The dependency counter of state i is D(i), which means that state i at time n is dependent on several states at time n+1. The initial value of D(i) is K, which means that state i at time n is initially dependent on all states at time n+1. Assign C1 and C2 to each state at time n+1; When calculating the cost from each state at time n to state 1 at time n+1, if each state at time n is not the optimal predecessor state of state 1 at time n+1, then its corresponding D(i) is subtracted by 1; If the optimal preceding state of any state at the subsequent n+1 time is replaced by state 1 to another state, then D(1) is reduced by 1; S2 also includes: Calculate the cost of states 2 to k at time n to state 2 at time n+1 respectively, and when updating, if states 2 to k at time n are not the best predecessor states of state 1 at time n+1, then the corresponding D(i) is subtracted by 1; Calculate the cost of state 2 at time n to state 3 to k at time n+1 respectively, and when updating, if there is no update after each comparison, then D(2) is reduced by 1; in the subsequent process, if the best previous state of any state at time n+1 is replaced by state 2 to other states, then D(2) is reduced by 1; Whenever D(i) = 0 at state i at time n, C, Nq, and C3 at state i at time n are released for use by state allocation C, Nq, and C3 at time n+1; Whenever a state at time n+1 is assigned to C, Nq, or C3, the corresponding data is updated. After the update is completed, D(i) of state i at time n corresponding to the optimal predecessor state of the state is reduced by 1.

3. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the method steps described in any one of claims 1-2 when executing a program stored in a memory.

4. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps according to any one of claims 1 to 2 are implemented.

5. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 2 is implemented.

Images