Low-delay cluster voice scheduling method based on voice recognition

Abstract

The invention discloses a low-delay cluster voice scheduling method based on voice recognition, and the method comprises the steps: 1, collecting voice stream data, and segmenting the voice stream data into a plurality of voice segments; 2, performing node distribution processing to form a scheduling state vector; 3, calculating a queuing state parameter and a reasoning time delay parameter, and constructing a queuing state index; 4, processing future speech features by using an improved CUSIDE model, and generating simulated future context features constrained by a queuing state; 5, inputting a streaming speech recognition model, executing parameter training updating, and generating a model instance; 6, in the online reasoning stage, recognition processing is executed on the input voice fragments, and voice recognition marks are output; and step 7, executing voice channel distribution, speaking right switching or broadcast route adjustment operation. According to the invention, collaborative optimization of voice recognition and scheduling decision is realized.

Description

Technical Field

[0001] This invention relates to the field of automatic speech recognition and cluster scheduling technology, and in particular to a low-latency cluster speech scheduling method based on speech recognition. Background Technology

[0002] With the widespread deployment of cloud computing platforms and edge computing nodes in voice interaction systems, cluster-based speech recognition services are gradually becoming a key foundational capability in scenarios such as intelligent customer service, emergency command and dispatch, vehicle communication systems, and large-scale voice conferencing. Speech streams typically need to be distributed and processed across multiple computing nodes, completing recognition and subsequent scheduling decisions under strict latency constraints. In existing technologies, speech recognition models mostly adopt streaming processing architectures, improving recognition accuracy through limited future context. Cluster scheduling systems, on the other hand, typically allocate tasks based on node load, queue length, or network latency. These two types of systems are often designed and run independently, lacking a unified coordination mechanism, making it difficult to form a closed-loop linkage between model inference strategies and cluster operating status.

[0003] Existing scheduling methods based on load awareness or queuing theory typically only model node states at the task allocation level, failing to influence how the speech recognition model uses future context internally. When cluster load surges or network fluctuations occur, the model still executes inference according to a fixed look-ahead strategy, easily leading to uncontrollable end-to-end latency. On the other hand, the training phase of existing streaming speech recognition models usually ignores cluster queuing states and node heterogeneity, only optimizing parameters for offline corpora, making it difficult for model instances to adapt to latency constraints under different operating states. Furthermore, current technologies rarely incorporate the queuing state evolution process into the model structure for modeling, lack a mechanism to integrate cluster state information into the simulation of future context generation, and have not established a complete closed-loop control process that links model instance selection and voice channel scheduling based on operating state indicators. This leads to problems such as delayed scheduling decisions, untimely switching of voice rights, or unstable broadcast routing adjustments in multi-node concurrent scenarios.

[0004] Therefore, how to provide a low-latency cluster voice scheduling method based on speech recognition is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a low-latency cluster voice scheduling method based on speech recognition. This invention embeds the queuing state index constructed from the scheduling state vector into the simulated future context generation path of the improved CUSIDE model, thereby forming a structural coupling between the speech recognition look-ahead strategy and the cluster operating state. In the training phase, multiple sets of model instances are constructed, and in the online phase, model selection is performed based on queuing state and latency parameters. This further links the execution of voice channel allocation, talk rights switching, and broadcast route adjustment, thereby reducing end-to-end latency, suppressing queuing accumulation, and improving the overall throughput of the system in multi-node concurrent scenarios.

[0006] A low-latency trunking voice scheduling method based on speech recognition according to an embodiment of the present invention includes the following steps: Step 1: Collect the voice stream data output by the terminal device and divide it into multiple voice segments according to a preset time window; Step 2: Perform node allocation processing on the multiple voice segments to form a scheduling state vector; Step 3: Calculate the queuing state parameters and inference delay parameters based on the scheduling state vector, and construct the queuing state index; Step 4: Process future speech features using the improved CUSIDE model. The improved CUSIDE model sets up a cluster queuing state embedding path in the simulated future context generation path, and generates simulated future context features constrained by the queuing state based on the queuing state index. Step 5: Input the speech segments and the simulated future context features into the streaming speech recognition model, perform parameter training and update, and generate model instances corresponding to different future context configuration parameters; Step 6: During the online inference phase, based on the real-time collected queuing state parameters and inference delay parameters, select model instances corresponding to the queuing state parameters and inference delay parameters from the pre-trained model set, perform recognition processing on the input speech segments, and output speech recognition tags; Step 7: Perform voice channel allocation, voice right switching, or broadcast routing adjustment operations based on the voice recognition markers and preset time threshold parameters.

[0007] Optionally, step one specifically includes: The analog speech signal output from the microphone acquisition channel of the terminal device is processed by analog-to-digital conversion to form pulse code modulation speech stream data. The pulse code modulation speech stream data adopts the linear pulse code modulation format and carries sampling rate parameters and quantization bit depth parameters. Write the pulse code modulated speech stream data into a circular buffer and write a sampling timestamp for each sampling point in the circular buffer; Read the preset time window parameters, which include at least the time window duration parameter and the time window step parameter; Based on the sampling rate parameter, the time window duration parameter is converted into the number of time window sampling points, and the time window step parameter is converted into the number of step sampling points; Using the number of step sampling points as the sliding step size, a sequence of sampling points of consecutive time window sampling points is extracted from the circular buffer in chronological order to form multiple speech segments; Each speech segment is constructed into a data structure containing a continuous sequence of sampling points, a segment start timestamp, a segment end timestamp, and a segment sequence number; Amplitude normalization and DC component removal are performed on multiple voice segments, and the processed voice segments are written into the segment queue.

[0008] Optionally, step two specifically involves: After receiving multiple voice segments, the compute node identifier is read from the compute node registry and a voice recognition node set is formed based on the node online status parameters and resource availability threshold parameters. The voice recognition node set includes compute nodes that participate in the current voice segment allocation and processing. Establish a node status record for each computing node in the speech recognition node set. The node status record includes at least the node queue length count, the node enqueue timestamp sequence, the node task processing completion timestamp sequence, and the node network round-trip delay measurement. For each computing node in the speech recognition node set, network round-trip time (RTT) measurement processing is performed. The RTT measurement processing includes sending a probe message to the corresponding node network address, recording the sending timestamp, receiving a response message and recording the receiving timestamp, and calculating the RTT measurement value based on the difference between the receiving timestamp and the sending timestamp. For each computing node in the speech recognition node set, queue length statistics processing is performed. Queue length statistics processing includes obtaining the number of speech segments to be processed for the corresponding node and writing the number of speech segments to be processed into the node queue length count value. For each computing node in the speech recognition node set, computational load statistics processing is performed. Computational load statistics processing includes reading the processor time slice occupancy ratio, reading the memory occupancy ratio, and reading the graphics processing unit utilization rate to form computational load data. Node allocation processing is performed on multiple voice segments. The node allocation process includes reading the voice segments sequentially according to the start timestamp of the voice segments, calculating the node score for each voice segment based on the node queue length count, network round-trip delay measurement and computing load data, and determining the computing node with the smallest score as the target node. Each voice segment is written into the processing queue corresponding to the target node, and the node queue length count of the target node is incremented by one to form a scheduling state vector containing the node queue length count, network round-trip time measurement, and computational load data.

[0009] Optionally, step three specifically includes: Receive the scheduling state vector and write it into the scheduling state buffer in chronological order; For each speech recognition node, read the corresponding node's scheduling state vector sequence from the scheduling state buffer; The queue length count values ​​of nodes in the scheduling state vector sequence undergo trend calculation processing, which includes calculating the change in queue length between adjacent sampling periods and forming a queue change trend value based on multiple continuous changes. Perform trend calculation processing on the network round-trip delay measurements in the scheduling state vector sequence. The trend calculation processing includes calculating the delay change between adjacent sampling periods and forming a delay change trend value based on multiple continuous changes. Based on the node queue length count value corresponding to a preset number of sampling periods in the scheduling state vector sequence, a weighted accumulation operation is performed on the node queue length count value and divided by the number of sampling periods to form the queue length mean parameter; based on the queue change trend value, a trend correction operation is performed on the queue length mean parameter to form the queue state parameter; Based on the network round-trip delay measurement values ​​corresponding to a preset number of sampling periods in the scheduling state vector sequence, a weighted summation operation is performed on the network round-trip delay measurement values ​​and divided by the number of sampling periods to form the mean delay parameter; based on the delay change trend value, a trend correction operation is performed on the mean delay parameter to form the inference delay parameter; Queuing status parameters are combined with inference delay parameters to form a queuing status index.

[0010] Optionally, step four specifically involves: Obtain the future speech features corresponding to the speech segments. The future speech features are defined as a sequence of speech feature frames within a preset future time range after the end timestamp of the speech segment. The improved CUSIDE model sets up a queuing state embedding path in the simulation of future context generation path. The queuing state embedding path includes a queuing stability potential mapping subnetwork and a context selection subnetwork connected in sequence. The queuing stable potential mapping subnetwork performs the following processing in hierarchical order: The first layer is a state-level system that performs amplitude scaling and interval mapping on the queuing state indicators. The second layer is the stable potential transformation layer, which performs nonlinear potential function transformation on the normalized queuing state index. The nonlinear potential function transformation includes: Read the normalized queuing status index value; calculate the difference between the normalized queuing status index value and the preset center value; perform absolute value operation on the difference to obtain the deviation amplitude; perform square operation on the deviation amplitude to obtain the energy term; multiply the deviation amplitude and the energy term by preset coefficients respectively and perform summation operation to obtain the potential value candidate quantity; perform saturation restriction processing on the potential value candidate quantity to restrict the potential value candidate quantity between the preset lower limit value and the preset upper limit value; input the restricted potential value candidate quantity into multiple preset basis function channels to obtain multiple channel output values, and concatenate them according to the preset channel order to form a stable potential representation vector; The third layer is the manifold projection layer, which performs dimension compression and orientation-preserving projection on the stable potential representation vector, and maps the output vector to a feature subspace isomorphic to the internal state space of the speech coding layer of the improved CUSIDE model, forming a queued state embedding vector. The context selection subnetwork receives the queuing state embedding vector and performs relevance scoring on the candidate future contexts output by the parallel future context generation path. Linear projection is then performed on the queuing state embedding vector and the features of each candidate future context. A dimension-wise multiplication operation is performed on the two types of projected features, and the multiplication results are summed to form a matching score. Bias superposition is performed on the matching scores to form the original score. Normalization is performed on each original score to obtain the path selection probability. The candidate future context with the highest path selection probability is determined as the target future context. The generation paths corresponding to non-target future contexts are masked, and only the target future context is retained as a simulated future context feature constrained by the queuing state.

[0011] Optionally, step five specifically includes: Multiple sets of training configurations are constructed based on the future context configuration parameters corresponding to different parallel future context generation paths. For each training configuration set, the speech segments and the corresponding simulated future context features constrained by queuing states are input into the streaming speech recognition model, and the model parameters are initialized. For each training configuration set, an iterative training process is performed on the streaming speech recognition model. The iterative training process includes reading speech segments in batches, calculating the loss value between the recognition output and the real annotation, and updating the model parameters according to the backpropagation algorithm. After each preset training round, record the average inference latency, queuing status index response value, and recognition accuracy of the model under the corresponding future context configuration parameters. A model performance record table is generated based on the average inference latency statistics, queuing status indicator response values, and recognition accuracy statistics, and the model parameter set corresponding to different future context configuration parameters is stored in the model repository; Establish an index mapping relationship between the set of model parameters in the model repository and the corresponding future context configuration parameters.

[0012] Optionally, step six specifically includes: During the online inference phase, the queuing status parameters and inference delay parameters corresponding to the current sampling period are collected in real time. Retrieves the future context configuration parameters and model performance record table corresponding to each model instance from the model repository; For each model instance, the difference between the current queuing state parameters and the inference delay parameters and the corresponding records in the model performance record table is calculated to form a state matching degree value. The state matching degree values ​​corresponding to each model instance are weighted and summed, and a comprehensive matching score is formed based on the preset weight coefficients. The model instance with the best overall matching score is identified as the target model instance; The target model instance is loaded into the online inference engine, and the target model instance is used to perform recognition processing on the input speech segments to output speech recognition tags.

[0013] Optionally, step seven specifically includes: Receive speech recognition tags and read the timestamp information, target computing node identifier, and speech session identifier of the corresponding speech segment; The end-to-end recognition latency is calculated based on the speech recognition markers and preset time threshold parameters, and the end-to-end recognition latency is written into the session scheduling status table; When the end-to-end recognition delay does not exceed the preset time threshold parameter, logical voice channels are allocated to the corresponding voice sessions according to the idle channel index order of the voice channel resource pool, and the current voice control identifier remains unchanged. When the end-to-end identification delay exceeds the preset time threshold parameter, the right-of-way control identifier is updated according to the preset right-of-way arbitration rules, and a right-of-way switching control message is sent to the terminal and the relay node to bind the subsequent voice stream to another logical voice channel. When the voice session identifier indicates the broadcast scheduling mode, the forwarding entries in the broadcast forwarding table are updated according to the voice recognition tags, the subsequent voice segments are mapped to the forwarding ports corresponding to the multicast node set, and the channel resources occupied by the original forwarding table entries are reclaimed.

[0014] The beneficial effects of this invention are: This invention constructs a queuing state modeling mechanism based on scheduling state vectors in a clustered speech recognition system and introduces queuing state indicators into the simulated future context generation path of an improved CUSIDE model. This achieves structural coupling between the speech recognition inference strategy and the cluster operation state. Addressing the problems of fixed model look-ahead strategies, disconnect between the scheduling system and the recognition model, and uncontrollable end-to-end latency under high load conditions in existing technologies, this invention proposes a context construction method based on queuing stability potential mapping and parallel future context path competition selection. This allows the model to dynamically adjust the future dependency scale and computation path under different queuing states. During the training phase, multiple model instances corresponding to future context configuration parameters are constructed and a model repository is formed. Online model selection is driven by queuing state parameters and inference latency parameters, achieving closed-loop linkage control of training, inference, and scheduling. During the online operation phase, the recognition results are further combined with time threshold constraints to link voice channel allocation, call right switching, and broadcast route adjustment operations. This stabilizes end-to-end latency, suppresses queuing accumulation, and improves the overall system throughput in multi-node concurrent scenarios, achieving low-latency clustered speech scheduling for real-time communication services. Attached Figure Description

[0015] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the overall process of a low-latency cluster voice scheduling method based on speech recognition proposed in this invention; Figure 2 This is a schematic diagram of the improved CUSIDE model and the cluster queuing state embedding path structure in a low-latency cluster voice scheduling method based on speech recognition proposed in this invention. Figure 3 This is a flowchart of the model repository construction and online model instance selection process in a low-latency cluster voice scheduling method based on speech recognition proposed in this invention. Detailed Implementation

[0016] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0017] refer to Figure 1-3 A low-latency cluster voice scheduling method based on speech recognition includes the following steps: Step 1: Collect the voice stream data output by the terminal device and divide it into multiple voice segments according to a preset time window; Step 2: Perform node allocation processing on multiple voice segments to form a scheduling state vector; Step 3: Calculate the queuing state parameters and inference delay parameters based on the scheduling state vector, and construct the queuing state index; Step 4: Process future speech features using the improved CUSIDE model. The improved CUSIDE model sets up a cluster queuing state embedding path in the simulated future context generation path, and generates simulated future context features constrained by queuing state based on queuing state indicators. Step 5: Input the speech segments and simulated future context features into the streaming speech recognition model, perform parameter training and update, and generate model instances corresponding to different future context configuration parameters; Step 6: During the online inference phase, based on the real-time collected queuing state parameters and inference delay parameters, select model instances corresponding to the queuing state parameters and inference delay parameters from the pre-trained model set, perform recognition processing on the input speech segments, and output speech recognition tags; Step 7: Perform voice channel allocation, voice right switching, or broadcast routing adjustment operations based on the voice recognition markers and preset time threshold parameters.

[0018] In this embodiment, step one specifically includes: The analog speech signal output from the microphone acquisition channel of the terminal device is processed by analog-to-digital conversion to form pulse code modulation speech stream data. The pulse code modulation speech stream data adopts the linear pulse code modulation format and carries sampling rate parameters and quantization bit depth parameters. Write the pulse code modulated speech stream data into a circular buffer and write a sampling timestamp for each sampling point in the circular buffer; Read the preset time window parameters, which include at least the time window duration parameter and the time window step parameter; Based on the sampling rate parameter, the time window duration parameter is converted into the number of time window sampling points, and the time window step parameter is converted into the number of step sampling points; Using the number of step sampling points as the sliding step size, a sequence of sampling points of consecutive time window sampling points is extracted from the circular buffer in chronological order to form multiple speech segments; Each speech segment is constructed into a data structure containing a continuous sequence of sampling points, a segment start timestamp, a segment end timestamp, and a segment sequence number; Amplitude normalization and DC component removal are performed on multiple voice segments, and the processed voice segments are written into the segment queue.

[0019] In this implementation, the preset time window parameters include a time window duration parameter and a time window step parameter. The time window duration parameter is set to any fixed value between 20 milliseconds and 60 milliseconds, and the time window step parameter is set to any fixed value between 10 milliseconds and 40 milliseconds, with the time window step parameter not exceeding the time window duration parameter. The time window duration parameter and time window step parameter are written to the configuration file by the cluster scheduling module during system initialization, and different parameter combinations are loaded according to different service priorities. The circular buffer adopts a ring storage structure, with a buffer capacity greater than 5 times the number of sampling points in a single time window. Amplitude normalization processing includes scaling operations based on the maximum absolute value of the sampled amplitude, and DC component removal processing includes calculating the mean of the sampling point sequence within the speech segment and performing mean cancellation operations. The segment queue is stored in sorted order according to the segment start timestamp and output to subsequent nodes in process order.

[0020] In this embodiment, step two specifically involves: After receiving multiple voice segments, the compute node identifier is read from the compute node registry and a voice recognition node set is formed based on the node online status parameters and resource availability threshold parameters. The voice recognition node set includes compute nodes that participate in the current voice segment allocation and processing. Establish a node status record for each computing node in the speech recognition node set. The node status record includes at least the node queue length count, the node enqueue timestamp sequence, the node task processing completion timestamp sequence, and the node network round-trip delay measurement. For each computing node in the speech recognition node set, network round-trip time (RTT) measurement processing is performed. The RTT measurement processing includes sending a probe message to the corresponding node network address, recording the sending timestamp, receiving a response message and recording the receiving timestamp, and calculating the RTT measurement value based on the difference between the receiving timestamp and the sending timestamp. For each computing node in the speech recognition node set, queue length statistics processing is performed. Queue length statistics processing includes obtaining the number of speech segments to be processed for the corresponding node and writing the number of speech segments to be processed into the node queue length count value. For each computing node in the speech recognition node set, computational load statistics processing is performed. Computational load statistics processing includes reading the processor time slice occupancy ratio, reading the memory occupancy ratio, and reading the graphics processing unit utilization rate to form computational load data. Node allocation processing is performed on multiple voice segments. The node allocation process includes reading the voice segments sequentially according to the start timestamp of the voice segments, calculating the node score for each voice segment based on the node queue length count, network round-trip delay measurement and computing load data, and determining the computing node with the smallest score as the target node. Each voice segment is written into the processing queue corresponding to the target node, and the node queue length count of the target node is incremented by one to form a scheduling state vector containing the node queue length count, network round-trip time measurement, and computational load data.

[0021] In this implementation, node online status parameters include the most recent heartbeat detection time and the current connection status identifier. Resource availability threshold parameters include a minimum of 2 idle processor cores, a minimum of 4 gigabytes of available memory, and a graphics processing unit utilization rate below 70%. The network round-trip latency measurement period is set to a fixed period between 50 milliseconds and 200 milliseconds. Each measurement involves sending at least 3 probe packets consecutively and averaging the time differences. The node score is obtained by weighting and summing the node queue length count, the network round-trip latency measurement, and the computational load data according to preset weight parameters. These preset weight parameters are written into the configuration file by the cluster scheduling module during system initialization. The node queue length count is updated synchronously during each voice segment enqueue and dequeue. The scheduling status vector identifies the current statistical period with a timestamp and is stored in the scheduling status buffer in chronological order for subsequent queue status parameter calculation processes.

[0022] In this embodiment, step three specifically includes: Receive the scheduling state vector and write it into the scheduling state buffer in chronological order; For each speech recognition node, read the corresponding node's scheduling state vector sequence from the scheduling state buffer; The queue length count values ​​of nodes in the scheduling state vector sequence undergo trend calculation processing, which includes calculating the change in queue length between adjacent sampling periods and forming a queue change trend value based on multiple continuous changes. Perform trend calculation processing on the network round-trip delay measurements in the scheduling state vector sequence. The trend calculation processing includes calculating the delay change between adjacent sampling periods and forming a delay change trend value based on multiple continuous changes. Based on the node queue length count value corresponding to a preset number of sampling periods in the scheduling state vector sequence, a weighted accumulation operation is performed on the node queue length count value and divided by the number of sampling periods to form the queue length mean parameter; based on the queue change trend value, a trend correction operation is performed on the queue length mean parameter to form the queue state parameter; Based on the network round-trip delay measurement values ​​corresponding to a preset number of sampling periods in the scheduling state vector sequence, a weighted summation operation is performed on the network round-trip delay measurement values ​​and divided by the number of sampling periods to form the mean delay parameter; based on the delay change trend value, a trend correction operation is performed on the mean delay parameter to form the inference delay parameter; Queuing status parameters are combined with inference delay parameters to form a queuing status index.

[0023] In this implementation, the sampling period of the scheduling state buffer is set to 100 milliseconds, and the sampling period for the most recent preset quantity is set to 5 consecutive periods. In the weighted accumulation operation, the weight coefficient corresponding to the most recent sampling period is set to 0.40, the weight coefficient of the second to last sampling period is set to 0.25, the weight coefficient of the third to last sampling period is set to 0.15, and the sum of the weight coefficients of the remaining sampling periods is set to 0.20. In the trend correction operation, the proportional correction coefficient corresponding to the upward change direction is set to 1.20, and the proportional correction coefficient corresponding to the downward change direction is set to 0.85. When the queuing state parameters and inference delay parameters are combined to form the queuing state index, linear normalization is adopted to map each parameter to the 0–1 interval.

[0024] In this embodiment, step four specifically includes: Obtain the future speech features corresponding to the speech segments. The future speech features are defined as a sequence of speech feature frames within a preset future time range after the end timestamp of the speech segment. The improved CUSIDE model sets up a queuing state embedding path in the simulation of future context generation path. The queuing state embedding path includes a queuing stability potential mapping subnetwork and a context selection subnetwork connected in sequence. The queuing stable potential mapping subnetwork performs the following processing in hierarchical order: The first layer is a state-level system that performs amplitude scaling and interval mapping on the queuing state indicators. The second layer is the stable potential transformation layer, which performs nonlinear potential function transformation on the normalized queuing state index. The nonlinear potential function transformation includes: Read the normalized queuing status index value; calculate the difference between the normalized queuing status index value and the preset center value; perform absolute value operation on the difference to obtain the deviation amplitude; perform square operation on the deviation amplitude to obtain the energy term; multiply the deviation amplitude and the energy term by preset coefficients respectively and perform summation operation to obtain the potential value candidate quantity; perform saturation restriction processing on the potential value candidate quantity to restrict the potential value candidate quantity between the preset lower limit value and the preset upper limit value; input the restricted potential value candidate quantity into multiple preset basis function channels to obtain multiple channel output values, and concatenate them according to the preset channel order to form a stable potential representation vector; The third layer is the manifold projection layer, which performs dimension compression and orientation-preserving projection on the stable potential representation vector, and maps the output vector to a feature subspace isomorphic to the internal state space of the speech coding layer of the improved CUSIDE model, forming a queued state embedding vector. The context selection subnetwork receives the queuing state embedding vector and performs relevance scoring on the candidate future contexts output by the parallel future context generation path. Linear projection is then performed on the queuing state embedding vector and the features of each candidate future context. A dimension-wise multiplication operation is performed on the two types of projected features, and the multiplication results are summed to form a matching score. Bias superposition is performed on the matching scores to form the original score. Normalization is performed on each original score to obtain the path selection probability. The candidate future context with the highest path selection probability is determined as the target future context. The generation paths corresponding to non-target future contexts are masked, and only the target future context is retained as a simulated future context feature constrained by the queuing state.

[0025] In this implementation, linear projection processing is achieved by the fully connected layer in the improved CUSIDE model. The parameters of the fully connected layer are updated during the training phase. The fully connected layer multiplies the input vector with the projection weight matrix stored in the model parameter area and then superimposes the bias vector to form the projection result. The preset basis function channels adopt a radial basis function structure, with each channel corresponding to a center value and a width parameter. The center values ​​are set to 0.20, 0.50, and 0.80, and the width parameter is set to 0.10. The amplitude scaling range of the state normalization layer is set to 0.05 to 0.95, and the preset center value is set to 0.50. The lower limit of the saturation constraint processing is set to 0, and the upper limit is set to 1. The number of parallel future context generation paths is set to 3, corresponding to future time span parameters of 40 milliseconds, 80 milliseconds, and 120 milliseconds, respectively.

[0026] The improved CUSIDE model retains the original CUSIDE streaming processing framework, time window segmentation strategy, and simulated future context training idea. It maintains the serialized computation structure between the encoding and decoding layers and continues to use future speech features to participate in the training stage modeling, enabling the model to complete the stepwise recognition and processing of continuous speech segments under limited look-ahead conditions. Based on the original structure, the improved CUSIDE model introduces a cluster queuing state embedding path in simulating the future context generation path. The path includes a queuing stability potential mapping subnetwork and a context selection subnetwork. The queuing state index drives the competitive selection of parallel future context generation paths, so that the construction of future features and the cluster running state form a structural coupling relationship. With the improved structure, the simulation of future context generation is directly constrained by the cluster queuing state, which reduces the scale of future dependencies under high load conditions and maintains the integrity of the context under low load conditions. This stabilizes the inference computation path, suppresses task queuing growth, and maintains low-latency output characteristics in multi-node concurrent scenarios.

[0027] In this embodiment, step five specifically includes: Multiple sets of training configurations are constructed based on the future context configuration parameters corresponding to different parallel future context generation paths. For each training configuration set, the speech segments and the corresponding simulated future context features constrained by queuing states are input into the streaming speech recognition model, and the model parameters are initialized. For each training configuration set, an iterative training process is performed on the streaming speech recognition model. The iterative training process includes reading speech segments in batches, calculating the loss value between the recognition output and the real annotation, and updating the model parameters according to the backpropagation algorithm. After each preset training round, record the average inference latency, queuing status index response value, and recognition accuracy of the model under the corresponding future context configuration parameters. A model performance record table is generated based on the average inference latency statistics, queuing status indicator response values, and recognition accuracy statistics, and the model parameter set corresponding to different future context configuration parameters is stored in the model repository; Establish an index mapping relationship between the set of model parameters in the model repository and the corresponding future context configuration parameters.

[0028] In this implementation, the streaming speech recognition model adopts an end-to-end sequence modeling structure, including a feature encoding layer, a temporal modeling layer, and a label decoding layer connected sequentially. The feature encoding layer performs linear transformation and normalization on continuous speech feature frames. The temporal modeling layer uses a recurrent neural network or a self-attention structure to update the state frame by frame. The label decoding layer outputs speech recognition labels using a connection-based temporal classification or attention decoding method. The backpropagation algorithm is used for parameter updates during the training phase. Its process includes: calculating the loss function value based on the recognition output and the ground truth labels; performing gradient differentiation on the loss function to obtain the gradients of the parameters at each layer; iteratively updating the parameters according to a preset learning rate; and repeating the above steps in multiple batch loops until the convergence condition is met.

[0029] In this embodiment, step six specifically includes: During the online inference phase, the queuing status parameters and inference delay parameters corresponding to the current sampling period are collected in real time. Retrieves the future context configuration parameters and model performance record table corresponding to each model instance from the model repository; For each model instance, the difference between the current queuing state parameters and the inference delay parameters and the corresponding records in the model performance record table is calculated to form a state matching degree value. The state matching degree values ​​corresponding to each model instance are weighted and summed, and a comprehensive matching score is formed based on the preset weight coefficients. The model instance with the best overall matching score is identified as the target model instance; The target model instance is loaded into the online inference engine, and the target model instance is used to perform recognition processing on the input speech segments to output speech recognition tags.

[0030] In this implementation, the model performance record table is generated synchronously with the model repository during the training phase, and stores future context configuration parameters and corresponding inference latency distribution intervals, queuing state indicator response intervals, and recognition accuracy intervals in a key-value format. During the online inference phase, the queuing state parameters and inference latency parameters collected in real time are mapped to the corresponding interval indices, and the candidate model subset is quickly located accordingly. The state matching degree calculation is completed in parallel using vectorized interpolation operations, and each weight coefficient is written to the configuration file through offline verification during the system deployment phase. When the target model instance is loaded, only the corresponding parameter tensor in the inference graph is replaced while keeping the network topology unchanged, so that the model switching is completed within a single sampling period, thereby forming a low-latency execution path with closed-loop linkage of training-selection-scheduling.

[0031] In this embodiment, step seven specifically includes: Receive speech recognition tags and read the timestamp information, target computing node identifier, and speech session identifier of the corresponding speech segment; The end-to-end recognition latency is calculated based on the speech recognition markers and preset time threshold parameters, and the end-to-end recognition latency is written into the session scheduling status table; When the end-to-end recognition delay does not exceed the preset time threshold parameter, logical voice channels are allocated to the corresponding voice sessions according to the idle channel index order of the voice channel resource pool, and the current voice control identifier remains unchanged. When the end-to-end identification delay exceeds the preset time threshold parameter, the right-of-way control identifier is updated according to the preset right-of-way arbitration rules, and a right-of-way switching control message is sent to the terminal and the relay node to bind the subsequent voice stream to another logical voice channel. When the voice session identifier indicates the broadcast scheduling mode, the forwarding entries in the broadcast forwarding table are updated according to the voice recognition tags, the subsequent voice segments are mapped to the forwarding ports corresponding to the multicast node set, and the channel resources occupied by the original forwarding table entries are reclaimed.

[0032] In this implementation, the session scheduling status table uses the voice session identifier as the primary key and stores the end-to-end recognition latency, the current logical voice channel number, the right-of-way control identifier, and the target computing node identifier. The voice channel resource pool uses a bitmap structure to record the idle channel index, with 64 channels. The preset time threshold parameter is set to 150 milliseconds to determine whether a right-of-way handover or routing adjustment is triggered. The broadcast forwarding table uses the multicast node set identifier as an index to record the forwarding port list and the occupied channel number. The right-of-way arbitration rule is determined by comparing the session priority level, the node queuing status indicator, and the historical occupancy time in sequence. When the session priorities are the same, the node with the smaller queuing status indicator is selected to obtain the right of speech; when the queuing status indicators are the same, the session with the shorter historical occupancy time is selected to obtain the right of speech.

[0033] Example 1: To verify the feasibility of this invention in practice, it was applied to a cluster speech recognition system of a large-scale emergency command and dispatch platform. This platform is used for multi-party voice conversation collaboration between public security, fire, and urban emergency departments, and needs to process more than 30,000 real-time voice streams simultaneously daily, involving various communication modes such as point-to-point calls, group conferences, and cross-regional broadcasts. The platform adopts a hybrid cluster architecture of edge computing nodes and cloud center nodes, containing twenty-four computing nodes. The processing capabilities and network link conditions of different nodes vary significantly. During peak periods, due to the concentrated occurrence of emergencies, a large number of voice streams enter the system simultaneously, easily causing a surge in queue lengths at some nodes, increasing end-to-end recognition latency, and leading to problems such as delayed call handover, delayed broadcast commands, and even voice channel blockage. Traditional scheduling methods allocate resources only based on node load, and the speech recognition model still maintains a fixed look-ahead window, which cannot reduce computational complexity when the cluster is congested, thus making it difficult to guarantee the real-time performance of the overall system.

[0034] In this embodiment, the analog voice signal acquired by the terminal-side microphone first undergoes analog-to-digital conversion and linear pulse code modulation processing at the front-end gateway, with the sampling rate set to 16 kHz and the quantization bit depth set to 16 bits. The voice stream is written into a circular buffer and then slid-segmented according to a time window length of 40 milliseconds and a step size of 20 milliseconds to form continuous voice segments. Each segment carries a start timestamp, an end timestamp, and a sequence number. After amplitude normalization and DC component removal, the segments enter a segment queue to await scheduling.

[0035] The scheduling module continuously reads the online node status from the compute node registry and filters it into a speech recognition node set based on the number of idle processor cores, available memory capacity, and graphics processing unit utilization. The system periodically sends probe messages to each node to measure network round-trip latency, and simultaneously counts the number of pending speech segments on each node, generating a node queue length count. For each speech segment, the scheduling module combines the queue length, network round-trip latency, and compute load node score, determining the node with the lowest score as the target node and updating the node queue length count upon enqueuing. This constructs a scheduling state vector containing three types of state data. The scheduling state vector is written to the scheduling state buffer at a sampling period of 100 milliseconds for subsequent calculation of queue state parameters and inference latency parameters.

[0036] The queuing state modeling module reads data from the buffer over multiple consecutive sampling periods, performs trend calculations on node queue length and network round-trip time, and calculates a weighted average. This average is then corrected through trend adjustments to form queue state parameters and inference delay parameters, which are combined to obtain a queue state index. This index is input into the simulated future context generation path of the improved CUSIDE model. The queue stability potential mapping sub-network performs potential function transformation and manifold projection processing. Finally, the context selection sub-network performs competitive selection among multiple parallel future context generation paths, retaining only the path output that best matches the current queue state as the simulated future context feature constrained by the queue state.

[0037] During the training phase, the system constructs multiple training configuration sets based on the configuration parameters corresponding to different parallel future contexts, and trains multiple sets of streaming speech recognition model instances respectively. Each model set records its average inference latency and recognition accuracy under different queuing state index intervals, and stores these in a unified model repository. During online runtime, the platform quickly matches the most suitable model instance from the model repository and loads it into the inference engine based on real-time collected queuing state parameters and inference latency parameters. This allows the system to automatically select models with lower computational complexity under high load and maintain high-precision model operation under low load.

[0038] After the speech recognition tags are generated, the scheduling and control module calculates the end-to-end recognition latency based on the 150-millisecond time threshold parameter. When the latency is within the threshold, an idle logical channel is directly allocated to the current session in the speech channel resource pool and the original right of speech is maintained. When the latency exceeds the threshold, the right of speech is switched according to the right of speech arbitration rules, and the subsequent speech stream is bound to another channel. In broadcast mode, the broadcast forwarding table is updated synchronously, and the speech segments are forwarded to the multicast node set, thus forming a closed-loop mechanism that links speech recognition, model selection and scheduling and control.

[0039] To verify the beneficial effects of the present invention, a traditional load-aware scheduling scheme and the method of the present invention were deployed on the same platform and run continuously for seven days to collect statistical data. The comparison was mainly based on the average end-to-end identification latency during peak periods, the peak node queue length, the system throughput, and the success rate of broadcast command issuance. The experimental results are shown in Table 1.

[0040] Table 1. Performance Comparison of Different Scheduling Schemes under High Load Scenarios

[0041] As shown in Table 1, the method of this invention achieves optimal performance across all four key indicators. Under high-load operating conditions, the method of this invention significantly outperforms traditional solutions in all key performance indicators. The average recognition latency during peak periods decreased from 312 milliseconds to 184 milliseconds, a reduction of approximately 41%, and the 95th percentile recognition latency decreased from 487 milliseconds to 246 milliseconds, indicating significant suppression of tail latency under extreme congestion conditions. The peak queue length per node decreased from 128 segments to 61 segments, indicating effective alleviation of queuing backlog. The number of voice segments processed per minute increased to 61,200, with throughput capacity increasing by over 44%. Simultaneously, the success rate of broadcast command issuance increased to 98.7%, and the average response time for call handover was shortened to 41 milliseconds, demonstrating the comprehensive advantages of this invention in terms of scheduling timeliness and communication reliability. This invention embeds queuing status indicators into the simulated future context generation process of the improved CUSIDE model and selects model instances based on queuing status during the online phase. This invention can effectively alleviate the impact of cluster congestion on the real-time performance of speech recognition and achieve the goal of low-latency cluster voice scheduling for emergency communication services.

[0042] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A low-latency trunking voice scheduling method based on speech recognition, characterized in that, Includes the following steps: Step 1: Collect the voice stream data output by the terminal device and divide it into multiple voice segments according to a preset time window; Step 2: Perform node allocation processing on the multiple voice segments to form a scheduling state vector; Step 3: Calculate the queuing state parameters and inference delay parameters based on the scheduling state vector, and construct the queuing state index; Step 4: Process future speech features using the improved CUSIDE model. The improved CUSIDE model sets up a cluster queuing state embedding path in the simulated future context generation path, and generates simulated future context features constrained by the queuing state based on the queuing state index. Step 5: Input the speech segments and the simulated future context features into the streaming speech recognition model, perform parameter training and update, and generate model instances corresponding to different future context configuration parameters; Step 6: During the online inference phase, based on the real-time collected queuing state parameters and inference delay parameters, select model instances corresponding to the queuing state parameters and inference delay parameters from the pre-trained model set, perform recognition processing on the input speech segments, and output speech recognition tags; Step 7: Perform voice channel allocation, voice right switching, or broadcast routing adjustment operations based on the voice recognition markers and preset time threshold parameters.

2. The low-latency trunking voice scheduling method based on speech recognition according to claim 1, characterized in that, Step one specifically involves: The analog speech signal output from the microphone acquisition channel of the terminal device is processed by analog-to-digital conversion to form pulse code modulation speech stream data. The pulse code modulation speech stream data adopts the linear pulse code modulation format and carries sampling rate parameters and quantization bit depth parameters. Write the pulse code modulated speech stream data into a circular buffer and write a sampling timestamp for each sampling point in the circular buffer; Read the preset time window parameters, which include at least the time window duration parameter and the time window step parameter; Based on the sampling rate parameter, the time window duration parameter is converted into the number of time window sampling points, and the time window step parameter is converted into the number of step sampling points; Using the number of step sampling points as the sliding step size, a sequence of sampling points of consecutive time window sampling points is extracted from the circular buffer in chronological order to form multiple speech segments; Each speech segment is constructed into a data structure containing a continuous sequence of sampling points, a segment start timestamp, a segment end timestamp, and a segment sequence number; Amplitude normalization and DC component removal are performed on multiple voice segments, and the processed voice segments are written into the segment queue.

3. The low-latency trunking voice scheduling method based on speech recognition according to claim 1, characterized in that, Step two specifically involves: After receiving multiple voice segments, the compute node identifier is read from the compute node registry and a voice recognition node set is formed based on the node online status parameters and resource availability threshold parameters. The voice recognition node set includes compute nodes that participate in the current voice segment allocation and processing. Establish a node status record for each computing node in the speech recognition node set. The node status record includes at least the node queue length count, the node enqueue timestamp sequence, the node task processing completion timestamp sequence, and the node network round-trip delay measurement. For each computing node in the speech recognition node set, network round-trip time (RTT) measurement processing is performed. The RTT measurement processing includes sending a probe message to the corresponding node network address, recording the sending timestamp, receiving a response message and recording the receiving timestamp, and calculating the RTT measurement value based on the difference between the receiving timestamp and the sending timestamp. For each computing node in the speech recognition node set, queue length statistics processing is performed. Queue length statistics processing includes obtaining the number of speech segments to be processed for the corresponding node and writing the number of speech segments to be processed into the node queue length count value. For each computing node in the speech recognition node set, computational load statistics processing is performed. Computational load statistics processing includes reading the processor time slice occupancy ratio, reading the memory occupancy ratio, and reading the graphics processing unit utilization rate to form computational load data. Node allocation processing is performed on multiple voice segments. The node allocation process includes reading the voice segments sequentially according to the start timestamp of the voice segments, calculating the node score for each voice segment based on the node queue length count, network round-trip delay measurement and computing load data, and determining the computing node with the smallest score as the target node. Each voice segment is written into the processing queue corresponding to the target node, and the node queue length count of the target node is incremented by one to form a scheduling state vector containing the node queue length count, network round-trip time measurement, and computational load data.

4. The low-latency trunking voice scheduling method based on speech recognition according to claim 1, characterized in that, Step three specifically involves: Receive the scheduling state vector and write it into the scheduling state buffer in chronological order; For each speech recognition node, read the corresponding node's scheduling state vector sequence from the scheduling state buffer; The queue length count values ​​of nodes in the scheduling state vector sequence undergo trend calculation processing, which includes calculating the change in queue length between adjacent sampling periods and forming a queue change trend value based on multiple continuous changes. Perform trend calculation processing on the network round-trip delay measurements in the scheduling state vector sequence. The trend calculation processing includes calculating the delay change between adjacent sampling periods and forming a delay change trend value based on multiple continuous changes. Based on the node queue length count value corresponding to a preset number of sampling periods in the scheduling state vector sequence, a weighted summation operation is performed on the node queue length count value and divided by the number of sampling periods to form the queue length mean parameter. Based on the queue change trend value, a trend correction operation is performed on the average queue length parameter to form the queue state parameter; Based on the network round-trip delay measurement values ​​corresponding to a preset number of sampling periods in the scheduling state vector sequence, a weighted summation operation is performed on the network round-trip delay measurement values ​​and divided by the number of sampling periods to form the mean delay parameter; based on the delay change trend value, a trend correction operation is performed on the mean delay parameter to form the inference delay parameter; Queuing status parameters are combined with inference delay parameters to form queuing status indices.

5. A low-latency trunking voice scheduling method based on speech recognition according to claim 1, characterized in that, Step four specifically involves: Obtain the future speech features corresponding to the speech segments. The future speech features are defined as a sequence of speech feature frames within a preset future time range after the end timestamp of the speech segment. The improved CUSIDE model sets up a queuing state embedding path in the simulation of future context generation path. The queuing state embedding path includes a queuing stability potential mapping subnetwork and a context selection subnetwork connected in sequence. The queuing stable potential mapping subnetwork performs the following processing in hierarchical order: The first layer is a state-level system that performs amplitude scaling and interval mapping on the queuing state indicators. The second layer is the stable potential transformation layer, which performs nonlinear potential function transformation on the normalized queuing state index. The nonlinear potential function transformation includes: Read the normalized queuing status index value; calculate the difference between the normalized queuing status index value and the preset center value; perform absolute value operation on the difference to obtain the deviation amplitude; perform square operation on the deviation amplitude to obtain the energy term; multiply the deviation amplitude and the energy term by preset coefficients respectively and perform summation operation to obtain the potential value candidate quantity; perform saturation restriction processing on the potential value candidate quantity to restrict the potential value candidate quantity between the preset lower limit value and the preset upper limit value; input the restricted potential value candidate quantity into multiple preset basis function channels to obtain multiple channel output values, and concatenate them according to the preset channel order to form a stable potential representation vector; The third layer is the manifold projection layer, which performs dimension compression and orientation-preserving projection on the stable potential representation vector, and maps the output vector to a feature subspace isomorphic to the internal state space of the speech coding layer of the improved CUSIDE model, forming a queued state embedding vector. The context selection subnetwork receives the queuing state embedding vector and performs relevance scoring on the candidate future contexts output by the parallel future context generation path. Linear projection is then performed on the queuing state embedding vector and the features of each candidate future context. A dimension-wise multiplication operation is performed on the two types of projected features, and the multiplication results are summed to form a matching score. Bias superposition is performed on the matching scores to form the original score. Normalization is performed on each original score to obtain the path selection probability. The candidate future context with the highest path selection probability is determined as the target future context. The generation paths corresponding to non-target future contexts are masked, and only the target future context is retained as a simulated future context feature constrained by the queuing state.

6. The low-latency trunking voice scheduling method based on speech recognition according to claim 1, characterized in that, Step five specifically involves: Multiple sets of training configurations are constructed based on the future context configuration parameters corresponding to different parallel future context generation paths. For each training configuration set, the speech segments and the corresponding simulated future context features constrained by queuing states are input into the streaming speech recognition model, and the model parameters are initialized. For each training configuration set, an iterative training process is performed on the streaming speech recognition model. The iterative training process includes reading speech segments in batches, calculating the loss value between the recognition output and the real annotation, and updating the model parameters according to the backpropagation algorithm. After each preset training round, record the average inference latency, queuing status index response value, and recognition accuracy of the model under the corresponding future context configuration parameters. A model performance record table is generated based on the average inference latency statistics, queuing status indicator response values, and recognition accuracy statistics, and the model parameter set corresponding to different future context configuration parameters is stored in the model repository; Establish an index mapping relationship between the set of model parameters in the model repository and the corresponding future context configuration parameters.

7. A low-latency trunking voice scheduling method based on speech recognition according to claim 1, characterized in that, Step six specifically involves: During the online inference phase, the queuing status parameters and inference delay parameters corresponding to the current sampling period are collected in real time. Retrieves the future context configuration parameters and model performance record table corresponding to each model instance from the model repository; For each model instance, the difference between the current queuing state parameters and the inference delay parameters and the corresponding records in the model performance record table is calculated to form a state matching degree value. The state matching degree values ​​corresponding to each model instance are weighted and summed, and a comprehensive matching score is formed based on the preset weight coefficients. The model instance with the best overall matching score is identified as the target model instance; The target model instance is loaded into the online inference engine, and the target model instance is used to perform recognition processing on the input speech segments to output speech recognition tags.

8. A low-latency trunking voice scheduling method based on speech recognition according to claim 1, characterized in that, Step seven specifically involves: Receive speech recognition tags and read the timestamp information, target computing node identifier, and speech session identifier of the corresponding speech segment; The end-to-end recognition latency is calculated based on the speech recognition markers and preset time threshold parameters, and the end-to-end recognition latency is written into the session scheduling status table; When the end-to-end recognition delay does not exceed the preset time threshold parameter, logical voice channels are allocated to the corresponding voice sessions according to the idle channel index order of the voice channel resource pool, and the current voice control identifier remains unchanged. When the end-to-end identification delay exceeds the preset time threshold parameter, the right-of-way control identifier is updated according to the preset right-of-way arbitration rules, and a right-of-way switching control message is sent to the terminal and the relay node to bind the subsequent voice stream to another logical voice channel. When the voice session identifier indicates the broadcast scheduling mode, the forwarding entries in the broadcast forwarding table are updated according to the voice recognition tags, the subsequent voice segments are mapped to the forwarding ports corresponding to the multicast node set, and the channel resources occupied by the original forwarding table entries are reclaimed.

Images