Method and system for concurrent request flow control of speech recognition service

Abstract

This invention discloses a method and system for concurrent request traffic control in speech recognition services, relating to the fields of speech recognition and traffic scheduling technology. The method includes: in the request admission phase, reading periodic concurrent audio streams and generating a computational complexity score; in the admission decision phase, performing admission pattern analysis to determine requests to be processed; in the request processing chain, performing extraction and combination based on acoustic fingerprints and semantic fingerprints on the requests to be processed to generate request processing results; and based on the request processing results, performing traffic control management on the requests to be processed. This invention solves the technical problems of existing technologies, such as coarse concurrent traffic control, insufficient cache reuse, and lagging scheduling decisions, which easily lead to system load imbalance and low concurrent processing efficiency in speech recognition services. It achieves refined traffic control across the entire speech recognition service chain, taking into account request admission, scheduling decisions, and cache reuse, thereby improving concurrent processing efficiency and ensuring stable system operation.

Description

Technical Field

[0001] This invention relates to the field of speech recognition and traffic scheduling technology, specifically to a method and system for controlling concurrent request traffic for speech recognition services. Background Technology

[0002] Existing speech recognition services generally employ single rate limiting or simple queuing methods for traffic control when facing massive concurrent audio stream requests. These methods are relatively crude and lack prior prediction of the computational complexity of the audio itself. At the same time, the coordination between the gateway and the recognition engine is insufficient, making it difficult to accurately identify real-time system bottlenecks and resulting in low efficiency in admission decisions. Furthermore, most services do not have a hierarchical cascading caching mechanism, leading to low reuse of request fingerprints in acoustic and semantic dimensions. A large number of repeated audio requests consume computing resources, which can easily result in uneven system load, high concurrent processing latency, and low resource utilization, making it difficult to meet the requirements of high-concurrency and high-stability speech recognition services.

[0003] Existing technologies suffer from problems such as crude concurrent traffic management for speech recognition services, insufficient cache reuse, and lagging scheduling decisions, which can easily lead to system load imbalance and low concurrent processing efficiency. Summary of the Invention

[0004] This application provides a method and system for controlling concurrent request traffic for speech recognition services, which addresses the technical problems of existing technologies, such as crude concurrent traffic control, insufficient cache reuse, and delayed scheduling decisions, which can easily lead to system load imbalance and low concurrent processing efficiency.

[0005] In view of the above problems, this application provides a method and system for controlling the concurrent request flow of speech recognition services.

[0006] A first aspect of this application provides a method for concurrent request flow control of a speech recognition service, the method comprising: During the request admission phase, periodic concurrent audio streams are read, and audio features are extracted and pre-analyzed using a lightweight engine to generate a computational complexity score. These audio features include duration, effective speech percentage, background noise energy, and endpoint detection activity. In the admission decision phase, a gateway-engine collaborative scheduling module performs bottleneck type judgment based on real-time water level vectors and admission pattern analysis based on the computational complexity score. Queue admission is implemented under a patterned deployment to determine the requests to be processed. In the request processing chain, a request processing module extracts and combines acoustic and semantic fingerprints for the requests to be processed. A cache query based on a cascading query chain is used to generate request processing results, including cache instructions and queue entry instructions. Based on the request processing results, traffic control management is applied to the requests to be processed.

[0007] A second aspect of this application provides a concurrent request traffic control system for a speech recognition service, the system comprising: The computational complexity score generation module is used during the request admission phase to read periodic concurrent audio streams, extract audio features through a lightweight engine, perform pre-analysis, and generate a computational complexity score. These audio features include duration, effective speech percentage, background noise energy, and endpoint detection activity. The pending request determination module is used during the admission decision phase to perform bottleneck type judgment based on real-time water level vectors and admission pattern analysis based on computational complexity scores using a gateway-engine collaborative scheduling module. It then performs queue admission under a patterned deployment to determine pending requests. The request processing result generation module is used in the request processing chain to extract and combine acoustic and semantic fingerprints from the pending requests, and performs cache queries based on a cascading query chain to generate request processing results. These results include cache instructions and queue entry instructions. The traffic control management module is used to manage traffic control for the pending requests based on the request processing results.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages: During the request admission phase, periodic concurrent audio streams are read, and audio features are extracted and pre-analyzed using a lightweight engine to generate a computational complexity score. In the admission decision phase, a gateway-engine collaborative scheduling module performs bottleneck type judgment based on real-time water level vectors and admission pattern analysis based on the computational complexity score, executing queue admission under a patterned deployment to determine requests to be processed. In the request processing chain, a request processing module extracts and combines acoustic and semantic fingerprints for the requests to be processed, employing a cache query based on a cascading query chain to generate request processing results. Based on the request processing results, traffic control management is applied to the requests to be processed. This achieves refined traffic control across the entire speech recognition service chain, balancing request admission, scheduling decisions, and cache reuse, improving concurrent processing efficiency, and ensuring stable system operation. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart illustrating the concurrent request flow control method for speech recognition services provided in this application embodiment; Figure 2A schematic diagram of the concurrent request traffic control system structure for the speech recognition service provided in this application embodiment.

[0011] Figure labeling: 10 computational complexity score generation module, 20 pending request determination module, 30 request processing result generation module, and 40 flow control management module. Detailed Implementation

[0012] This application provides a method and system for controlling concurrent request traffic for speech recognition services, which addresses the technical problems of existing technologies, such as crude concurrent traffic management, insufficient cache reuse, and delayed scheduling decisions, which can easily lead to system load imbalance and low concurrent processing efficiency.

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0014] Example 1, as Figure 1 As shown, this application provides a method for controlling the concurrent request flow of a speech recognition service, the method comprising: Step S100: During the admission request phase, read the periodic concurrent audio streams, extract audio features and perform pre-analysis through a lightweight engine, and generate a computational complexity score. The audio features include duration, effective speech ratio, background noise energy, and endpoint detection activity.

[0015] Specifically, during the admission request phase, multiple concurrent audio streams are periodically read, and the audio data is preprocessed using a lightweight engine to extract four features: duration, effective speech ratio, background noise energy, and endpoint detection activity. Each feature is quantified and weighted based on its occupancy of recognition computing resources. The normalized duration value, effective speech ratio, background noise energy level, and endpoint detection activity value are then combined to obtain a computational complexity score that characterizes the processing overhead of the audio stream. This score is used for traffic scheduling and hierarchical control in the subsequent admission phase.

[0016] Step S200: In the admission decision phase, the gateway-engine collaborative scheduling module is used to perform bottleneck type judgment based on real-time water level vector and admission mode analysis based on computational complexity score, execute queue admission under patterned deployment, and determine the requests to be processed.

[0017] Specifically, during the admission decision-making phase, water level monitoring probes deployed at the gateway and engine layers continuously collect queue length, average processing latency, CPU / GPU utilization, and memory usage to form a real-time water level vector. This vector, combined with a static dependency graph, determines whether the bottleneck is a gateway or engine bottleneck. Then, based on the generated computational complexity score, a tiered admission analysis is performed. If it is an engine bottleneck, high-complexity requests are directly returned with an overload code, medium-complexity requests are downgraded to asynchronous batch processing and merged into a low-priority queue, and low-complexity requests are admitted normally. If it is a gateway bottleneck, high-complexity requests are subjected to exponential backoff suppression and explicit rate-limiting instructions are issued. Finally, queue admission screening is completed according to the patterned deployment rules to determine the requests to be processed in the processing chain.

[0018] Step S300: In the request processing chain, the request processing module performs extraction and combination based on acoustic fingerprint and semantic fingerprint on the request to be processed, and performs cache query based on cascading query chain to generate request processing result, wherein the request processing result includes cache instruction and queue entry instruction.

[0019] Specifically, in the request processing chain, the request processing module first extracts the acoustic fingerprint and semantic fingerprint of the request to be processed in parallel and combines them to obtain a unique request identifier. Then, the identifier is sent to the cascading query chain composed of the local memory cache layer, the edge node cache layer, and the global distributed cache layer to perform cache hit queries in sequence. If a cache hit occurs in the local memory cache layer or the edge / global distributed cache layer, a local cache instruction is directly generated and the request is not included in the concurrency count. If all cache hits fail, a queue entry instruction is generated and the request is included in the concurrency count. Finally, the request processing result containing the cache instruction and the queue entry instruction is obtained.

[0020] Step S400: Based on the request processing result, perform flow control management on the request to be processed.

[0021] Specifically, based on the cache instructions and queue entry instructions contained in the request processing result, differentiated traffic control management is performed on the requests to be processed: if the corresponding cache instruction is a cache hit, the cache identification result is directly reused and the request is allowed, and the request is not included in the concurrency count to reduce service load; if the corresponding queue entry instruction is a cache miss, the request is allocated to the corresponding priority queue according to the computational complexity score and watermark vector, and at the same time, combined with the gateway... The engine coordinates the scheduling results to control the number of concurrent accesses, and completes precise traffic control and resource scheduling for concurrent requests to the speech recognition service.

[0022] In one possible implementation, step S200 further includes: Step S210: Deploy water level monitoring probes at the gateway layer and engine layer respectively to continuously collect real-time water level vectors of each node. The real-time water level vectors are determined by queue length, average processing latency, CPU and GPU utilization, and memory usage.

[0023] Step S220: Define a static dependency graph, wherein the static dependency graph annotates the interface dependency chain and critical path nodes of the gateway calling engine.

[0024] Step S230: By internally maintaining the static dependency graph, and using bottleneck identification based on the superposition of real-time water level vector and static dependency graph as the underlying logic, a gateway-engine collaborative scheduling module is constructed.

[0025] Specifically, water level monitoring probes are deployed at the gateway layer and engine layer of the speech recognition service, respectively. The gateway layer is used for request access and forwarding scheduling, while the engine layer is used for performing speech recognition calculations. The probes continuously collect real-time operating status data of each node, and quantify and combine five indicators, namely queue length, average processing latency, CPU utilization, GPU utilization, and memory usage, to form a real-time water level vector that characterizes the load and congestion level of the current node.

[0026] A static dependency graph is pre-built and defined. This static dependency graph is a topology graph used to represent the calling logic between the gateway and the engine. The graph clearly marks the interface dependency chain of the gateway calling the engine, that is, the interface calling order and transmission link through which the gateway initiates a speech recognition request to the engine. At the same time, the key path nodes that undertake the core computing and forwarding functions in the entire calling process are clearly marked, thereby establishing a stable calling association and topology dependency between the gateway layer and the engine layer.

[0027] The system continuously maintains and calls the defined static dependency graph in real time. The real-time water level vectors collected by the gateway layer and engine layer are bound and mapped one by one to the interface dependency chains marked on the static dependency graph and the critical path nodes. At runtime, the water level indicators corresponding to each critical path node are read in real time. Bottleneck nodes are located by comparing the indicators with preset thresholds. Then, with water level monitoring, topology dependency, and bottleneck determination as the core logic, functional units such as probe collection, threshold comparison, bottleneck identification, and scheduling decision are integrated and encapsulated to finally build a gateway-engine collaborative scheduling module that can link the gateway layer and engine layer and realize global collaborative scheduling.

[0028] In one possible implementation, step S230 further includes: Step S231: When the engine water level vector in the real-time water level vector is greater than the first threshold, it is determined to be an engine bottleneck, and the access-side differentiated throttling mode is triggered.

[0029] Step S232: Based on the admission-side differentiated throttling mode, perform hierarchical flow control decision and generate a forwarding rate limit, which is then sent to the gateway layer. The gateway layer limits the request rate forwarded to the engine layer.

[0030] Step S233: A tiered approach based on computational complexity scoring is adopted. If the first complexity level is met, an overload code is returned; if the second complexity level is met, the process is downgraded to asynchronous batch processing and enters a low-priority queue; if the third complexity level is met, the request is granted normal access.

[0031] Specifically, the engine water level vector in the real-time water level vector is monitored in real time, and the engine water level vector is compared with a preset first threshold. When the engine water level vector is greater than the first threshold, it is determined that the current engine resources are insufficient, the processing capacity has reached the limit, and there is an engine bottleneck. Then, the differentiated throttling mode on the access side is triggered and started.

[0032] The layered flow control decision is executed according to the triggered access-side differentiated throttling mode. The maximum allowable request forwarding rate limit is calculated based on the current engine load and real-time water level vector. This rate limit instruction is sent to the gateway layer. The gateway layer controls the rate of requests sent to the engine layer according to this instruction, constraining the number of requests forwarded per unit time, thereby reducing the processing pressure on the engine layer.

[0033] The system uses a three-tiered classification system based on computational complexity scores. First, the computational complexity score of each request is compared with a preset threshold. If the score reaches the first complexity level (high complexity), a service overload code is returned to the request source and access is denied. If the score falls into the second complexity level (medium complexity), the request is downgraded from synchronous processing mode to asynchronous batch processing mode and assigned to a low-priority queue for scheduling. If the score meets the third complexity level (low complexity), it is determined to have low resource consumption and can be processed quickly. Normal access is granted and the request enters the regular processing queue, thus achieving tiered and differentiated traffic control.

[0034] In one possible implementation, step S230 further includes: Step S234: When the gateway water level vector in the real-time water level vector is greater than the second threshold, it is determined to be a gateway bottleneck, and the forwarding side active rate limiting mode is triggered.

[0035] Step S235: According to the forwarding side active rate limiting mode, if the computational complexity score meets the first complexity level, an explicit rate limiting instruction is generated and sent to the client based on the exponential backoff suppression analysis for the forwarding rate, wherein the explicit rate limiting instruction carries a suggested sending interval.

[0036] Step S236: The gateway layer caches the rate-limiting token bucket based on the explicit rate-limiting instruction locally, and releases the forwarding token with the instruction speed as a constraint.

[0037] Specifically, the gateway water level vector in the real-time water level vector is obtained in real time, and the gateway water level vector is compared with a preset second threshold. When the gateway water level vector is greater than the second threshold, it is determined that the current gateway layer is experiencing forwarding congestion and the processing capacity has reached its limit, which is identified as a gateway bottleneck, and the forwarding side active rate limiting mode is immediately activated.

[0038] In the forwarding-side active rate limiting mode, for high-complexity requests that meet the first complexity level in terms of computational complexity score, the overload difference between the gateway's real-time water level vector and the second threshold is used as input. An exponential backoff algorithm is used to perform forwarding rate suppression analysis. This algorithm uses the initial interval as a benchmark and increases the request sending interval round by round with a fixed exponential coefficient. At the same time, it dynamically adjusts the backoff step size in combination with the gateway's real-time water level feedback mechanism. A stable suggested sending interval is obtained through round-by-round convergence calculation. Then, an explicit rate limiting instruction carrying the suggested sending interval is generated and returned to the client to enforce the client's request sending rate.

[0039] The gateway layer is the core network access node responsible for request forwarding and traffic control. In its local cache, i.e., the high-speed memory storage area of ​​the gateway itself, it constructs a rate-limiting token bucket based on the explicit rate-limiting instructions issued by the client. This rate-limiting token bucket is a rate control data structure used for traffic shaping. Internally, it uses the instruction speed calculated according to the suggested sending interval carried in the explicit rate-limiting instruction as a constraint condition to continuously generate and release forwarding tokens at a fixed rate. Requests initiated by the client must obtain the corresponding forwarding token before they can be forwarded by the gateway layer. In this way, the request forwarding rate is strictly controlled, and precise traffic limitation is achieved on the gateway side.

[0040] In one possible implementation, step S300 further includes: Step S310: Deploy the first extraction node by extracting acoustic fingerprints and semantic fingerprints in parallel.

[0041] Step S320: Deploy cascading query links based on the cascading cache query determination of the three-layer cache group architecture, wherein the three-layer cache group includes a local memory cache layer, an edge node cache layer and a global distributed cache layer.

[0042] Step S330: By associating the first extraction node with the cascading query link, a request processing module is formed.

[0043] Specifically, a first extraction node is deployed, which is configured with parallel processing logic. It can simultaneously perform acoustic fingerprint extraction and semantic fingerprint extraction on the received voice request data. The acoustic fingerprint is used to characterize the waveform features of the audio itself, and the semantic fingerprint is used to characterize the semantic features of the text after speech translation. By executing the two types of fingerprint extraction tasks in parallel, the extraction efficiency of fingerprint features is improved, and the synchronous output of the two types of fingerprints is achieved.

[0044] The cascading query chain is deployed based on a three-layer cache group architecture. The three-layer cache group consists of a local memory cache layer, an edge node cache layer, and a global distributed cache layer, arranged from top to bottom. The local memory cache layer is the high-speed memory of the gateway or processing node itself, used for fast querying of high-frequency hot requests. The edge node cache layer consists of cache nodes deployed in the nearest region to share the global query pressure. The global distributed cache layer is a centralized cache cluster shared across regions to store the full recognition results. The cascading query chain initiates cache hit queries layer by layer in the order from local to edge to global, completing the rapid determination of the request recognition results and realizing the cache query logic of prioritizing proximity and returning to the source step by step.

[0045] The first extraction node, used for parallel extraction of acoustic and semantic fingerprints, is logically connected and data-associated with the cascaded query link used for layer-by-layer cache query determination. The fingerprint features output by the first extraction node are used as the query input of the cascaded query link, which completes the cache hit determination based on the fingerprint. The two are functionally connected and data-transferring with each other, integrating to form a request processing module with fingerprint extraction, cache query, and result determination functions, realizing integrated recognition and processing of voice requests to be processed.

[0046] In one possible implementation, step S320 further includes: Step S321: The local memory cache layer is deployed inside each gateway node, and uses a high-speed hash table to store recent request fingerprints and identification results, and performs interception of duplicate requests.

[0047] Step S322: The edge node caching layer is deployed on each edge cache node, adopts distributed caching and covers multiple gateway nodes in the same area, and is used to share the results of repeated requests in the same area.

[0048] Step S323: The global distributed cache layer is deployed in the central cloud and uses key-value storage to cover the entire service cluster for cross-regional global deduplication and cold start cache filling.

[0049] Specifically, the local memory caching layer is integrated and deployed in the internal storage space of each gateway node. A high-speed hash table with fast query speed and accurate positioning is used as the storage structure to cache recently received request fingerprints and corresponding voice recognition results. By quickly comparing whether the request fingerprint already exists, the same request that arrives repeatedly can be intercepted in advance to avoid repeated recognition operations.

[0050] The edge node caching layer is deployed on each independently configured edge caching node. A distributed caching architecture is adopted to realize distributed data storage. This caching layer can cover multiple gateway nodes in the same service area, realize the interconnection and sharing of cached data in the area, and reuse the recognition results of repeated voice requests in the same area, reducing the duplication of calculations across nodes.

[0051] The global distributed caching layer is deployed on the central cloud server, and a key-value storage method is used to achieve unified data management. The caching scope covers the entire service cluster, which can complete global duplicate request deduplication verification between different regions. At the same time, during the cold start phase of the edge node cache, cache data is batch-filled to ensure the stable operation of the entire system caching system.

[0052] In one possible implementation, step S300 further includes: Step S340: Based on the first extraction node, perform audio content summary extraction based on spectral hashing on the request to be processed, and combine it with text embedding based on historical confidence recognition results to obtain the request identifier.

[0053] Step S350: Input the request identifier into the cascading query chain, execute a query based on the local memory cache layer. If the query is successful, the pending request is not included in the concurrency count, and a local cache instruction is generated.

[0054] Step S360: If no match is found, perform sequential queries based on the edge node caching layer and the global distributed caching layer. If no match is found, add the pending request to the concurrency count and generate a queue entry instruction.

[0055] Specifically, through the first extraction node, the speech audio signal is first processed by framing and windowing, and the spectral hashing algorithm is used to perform binary hash encoding on the spectral features to extract a compact audio content summary; then, based on the confidence weight of historical speech recognition results, the recognized text is semantically encoded to generate a text embedding vector; the binary audio summary and the confidence weighted text embedding vector are concatenated and fused to generate a unique indexable request identifier for subsequent fast caching matching.

[0056] The concatenated request identifier is used as the retrieval key to input into the cascading query chain. A precise match is performed in the high-speed hash table of the local memory cache layer through the hash index. If the query hits, meaning that the current request already has a cached result, the system does not increment the concurrent count for the pending request. Instead, it directly generates a local cache instruction to return the cache identification result, thus quickly completing the request response.

[0057] If a match is not found in the local memory cache layer, a request identifier matching query is initiated sequentially to the edge node cache layer and the global distributed cache layer in hierarchical order. If a match is found in any cache layer during the query process, the cache result is immediately synchronized to the current layer for temporary cache storage, and the request is marked as pending and stored for later processing. If no matching result is found in either the edge node cache layer or the global distributed cache layer, the pending request is added to the system concurrency count, and a queue entry instruction is generated to place it into the current pending queue for identification and processing.

[0058] In one possible implementation, step S100 further includes: Step S110: Collect multi-dimensional behavioral features for the request source of the concurrent audio stream, wherein the multi-dimensional behavioral features include request frequency, length distribution, content repetition, endpoint detection activity and request time interval regularity.

[0059] Step S120: Based on the multidimensional behavioral characteristics, use an anomaly detector to calculate an anomaly score.

[0060] Step S130: Based on the anomaly score, mark the request source as a malicious source and impose processing constraints on the admission decision stage and request processing link.

[0061] Specifically, for request sources that generate concurrent audio streams, i.e., multiple voice requests accessing simultaneously, multi-dimensional behavioral features are collected to identify abnormal access behavior. Among them, request frequency is the number of times the request source initiates voice requests per unit time; length distribution is the distribution of the duration intervals of each audio data of the request source; content repetition is the degree of similarity or repetition of audio content sent multiple times by the request source; endpoint detection activity is the detection activity status of effective voice segments in the audio; and request time interval regularity is whether there is a fixed or abnormal pattern in the time interval between two adjacent requests. The above multi-dimensional features comprehensively characterize the access behavior of the request source.

[0062] The collected multi-dimensional behavioral features, including request frequency, length distribution, content redundancy, endpoint detection activity, and request time interval regularity, are first normalized and then input into a preset anomaly detector. This detector uses a weighted scoring model, assigning a corresponding anomaly weight to each feature, calculating the deviation of each feature from the normal baseline, multiplying each feature's deviation value by its corresponding weight, summing the results, and finally normalizing to obtain an anomaly score in the range of 0 to 1. The higher the score, the more obvious the abnormal behavior of the request source.

[0063] Based on the anomaly score output by the anomaly detector and the preset threshold, the request source is marked as a normal source or a malicious source. During the admission decision stage, a differentiated quota mapping is performed on the request source marked as abnormal. Even if it submits a low-complexity audio request, it is mapped to a high virtual quota consumption. Natural traffic suppression is achieved through the quota occupation mechanism. At the same time, in the request processing chain, for abnormal request sources with low trust, even if the current system meets the resource carrying capacity, rate limiting is performed or it is allocated to the slow start queue for delayed processing. This is to constrain the occupation of system concurrent resources by abnormal requests.

[0064] In one possible implementation, step S120 further includes: Step S121: Using the isolated forest approach, the feature space of the multidimensional behavioral features is randomly and recursively divided to construct multiple isolated trees, and the first anomaly score is calculated based on the path length within the tree.

[0065] Step S122: Using density clustering, calculate the local reachability density of the multidimensional behavioral features in the feature space to determine the second anomaly score.

[0066] Step S123: Weighted fusion of the first abnormal score and the second abnormal score to obtain the abnormal score.

[0067] Specifically, an anomaly detection method using isolated forests is adopted. Multidimensional behavioral features consisting of request frequency, length distribution, content redundancy, endpoint detection activity, and request time interval regularity are used as input. Multiple independent isolation trees are constructed by randomly and recursively dividing the feature space. In each isolation tree, the shorter the path from the request source feature to the leaf node, the easier it is for the algorithm to isolate it in advance and the higher the probability that it belongs to an abnormal behavior. Based on the path length, normalization calculation is performed, and finally, the first anomaly score representing the behavior isolation is output.

[0068] An anomaly detection method using density clustering is adopted, which maps the collected multidimensional behavioral features to a unified feature space. By calculating the local reachability density of each request source sample point, the density difference between the sample point and other normal samples in its neighborhood is compared. Sample points with local densities significantly lower than the overall density of the neighborhood are identified as anomalies. A quantitative score is given based on the degree of density deviation, and finally, a second anomaly score representing the degree of behavioral dispersion is determined.

[0069] The first anomaly score obtained through isolated forest and the second anomaly score obtained through density clustering are weighted according to a pre-set weighting coefficient. The two weighted scores are then added together and normalized. The anomaly judgment results from both behavioral isolation and spatial density are combined to obtain a comprehensive anomaly score, which is used to accurately identify the level of abnormal behavior of the request source.

[0070] Example 2, based on the same inventive concept as the concurrent request flow control method for the speech recognition service in the foregoing examples, such as... Figure 2 As shown, this application provides a concurrent request traffic control system for speech recognition services. The system and method embodiments in this application are based on the same inventive concept. The system includes: The computational complexity score generation module 10 is used to read periodic concurrent audio streams during the request admission phase, extract audio features through a lightweight engine and perform pre-analysis to generate a computational complexity score. The audio features include duration, effective speech ratio, background noise energy and endpoint detection activity.

[0071] The pending request determination module 20 is used in the admission decision stage to perform bottleneck type judgment based on real-time water level vector and admission mode analysis based on computational complexity score by using the gateway-engine collaborative scheduling module to perform queue admission under the patterned deployment and determine pending requests.

[0072] The request processing result generation module 30 is used in the request processing chain to perform extraction and combination based on acoustic fingerprint and semantic fingerprint on the request to be processed, and to perform cache query based on cascading query chain to generate request processing result, wherein the request processing result includes cache instruction and queue entry instruction.

[0073] The flow control management module 40 is used to perform flow control management on the pending request based on the request processing result.

[0074] Furthermore, the system is also used to implement the following functions: Water level monitoring probes are deployed at both the gateway and engine layers to continuously collect real-time water level vectors from each node. These real-time water level vectors are determined by queuing length, average processing latency, CPU and GPU utilization, and memory usage. A static dependency graph is defined, which annotates the interface dependency chain and critical path nodes of the gateway calling the engine. By internally maintaining the static dependency graph, a gateway-engine collaborative scheduling module is constructed based on bottleneck identification under the superposition of real-time water level vectors and the static dependency graph.

[0075] Furthermore, the system is also used to implement the following functions: When the engine water level vector in the real-time water level vector exceeds the first threshold, it is determined to be an engine bottleneck, triggering the admission-side differentiated throttling mode. According to the admission-side differentiated throttling mode, a hierarchical flow control decision is executed, and a forwarding rate limit is generated and sent to the gateway layer. The gateway layer limits the request rate forwarded to the engine layer. Among them, a hierarchical classification based on computational complexity scoring is adopted. If the first complexity level is met, an overload code is returned. If the second complexity level is met, it is downgraded to asynchronous batch processing and enters a low-priority queue. If the third complexity level is met, normal access for the request is executed.

[0076] Furthermore, the system is also used to implement the following functions: When the gateway water level vector in the real-time water level vector is greater than the second threshold, it is judged as a gateway bottleneck, triggering the forwarding side active rate limiting mode; according to the forwarding side active rate limiting mode, if the computational complexity score meets the first complexity level, an explicit rate limiting instruction is generated and sent to the client based on the exponential backoff suppression analysis for the forwarding rate, wherein the explicit rate limiting instruction carries a suggested sending interval; the gateway layer caches the rate limiting token bucket based on the explicit rate limiting instruction locally, and releases the forwarding token with the instruction speed as a constraint.

[0077] Furthermore, the system is also used to implement the following functions: The first extraction node is deployed by extracting acoustic fingerprints and semantic fingerprints in parallel; the cascading query chain is deployed by using a cascading cache query decision based on a three-layer cache group architecture, wherein the three-layer cache group includes a local memory cache layer, an edge node cache layer and a global distributed cache layer; and a request processing module is formed by associating the first extraction node with the cascading query chain.

[0078] Furthermore, the system is also used to implement the following functions: The local memory cache layer is deployed inside each gateway node, using a high-speed hash table to store recent request fingerprints and identification results, and intercepting duplicate requests. The edge node cache layer is deployed on each edge cache node, using distributed caching and covering multiple gateway nodes in the same region, for sharing duplicate request results in the same region. The global distributed cache layer is deployed in the central cloud, using key-value storage to cover the entire service cluster, for cross-regional global deduplication and cold start cache filling.

[0079] Furthermore, the system is also used to implement the following functions: Based on the first extraction node, the request to be processed is subjected to audio content summary extraction based on spectral hashing, which is combined with text embedding based on historical confidence recognition results to obtain a request identifier. The request identifier is input into the cascading query chain to perform a query based on the local memory cache layer. If the query is successful, the request to be processed is not included in the concurrency count, and a local cache instruction is generated. If the query is unsuccessful, sequential queries based on the edge node cache layer and the global distributed cache layer are performed. If the query is unsuccessful, the request to be processed is included in the concurrency count, and a queue entry instruction is generated.

[0080] Furthermore, the system is also used to implement the following functions: For the request sources of the concurrent audio streams, multi-dimensional behavioral features are collected, including request frequency, length distribution, content repetition, endpoint detection activity, and regularity of request time intervals. Based on the multi-dimensional behavioral features, an anomaly detector is used to calculate an anomaly score. Based on the anomaly score, the request sources are marked as malicious sources, and processing constraints are imposed on the admission decision stage and request processing chain.

[0081] Furthermore, the system is also used to implement the following functions: Using an isolated forest approach, the feature space of the multidimensional behavioral features is randomly and recursively divided to construct multiple isolated trees, and a first anomaly score is calculated based on the path length within the tree. Using density clustering, the local reachability density of the multidimensional behavioral features in the feature space is calculated to determine a second anomaly score. The first anomaly score and the second anomaly score are then weighted and fused to obtain the final anomaly score.

[0082] It should be noted that the order of the embodiments described above is for descriptive purposes only and does not represent the superiority or inferiority of the embodiments. Specific embodiments of this specification have been described above. Furthermore, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0083] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0084] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application intends to include such modifications and variations.

Claims

1. A method for controlling the flow of concurrent requests in a speech recognition service, characterized in that, The method includes: During the admission request phase, periodic concurrent audio streams are read, and audio features are extracted and pre-analyzed using a lightweight engine to generate a computational complexity score. The audio features include duration, effective speech percentage, background noise energy, and endpoint detection activity. During the admission decision-making phase, the gateway-engine collaborative scheduling module is used to perform bottleneck type judgment based on real-time water level vector and admission pattern analysis based on computational complexity score, execute queue admission under patterned deployment, and determine the requests to be processed. In the request processing chain, a request processing module is used to extract and combine acoustic fingerprints and semantic fingerprints for the request to be processed, and a cache query based on a cascading query chain is used to generate a request processing result, wherein the request processing result includes a cache instruction and a queue entry instruction. Based on the request processing result, traffic control management is performed on the pending requests.

2. The concurrent request flow control method for speech recognition services as described in claim 1, characterized in that, The gateway-engine collaborative scheduling module includes: Water level monitoring probes are deployed at the gateway layer and engine layer respectively to continuously collect the real-time water level vector of each node. The real-time water level vector is determined by the queue length, average processing latency, CPU and GPU utilization, and memory usage. Define a static dependency graph, wherein the static dependency graph annotates the interface dependency chain and critical path nodes of the gateway call engine; By internally maintaining the static dependency graph, and using bottleneck identification based on the superposition of real-time water level vector and static dependency graph as the underlying logic, a gateway-engine collaborative scheduling module is constructed.

3. The concurrent request flow control method for speech recognition services as described in claim 2, characterized in that, When the engine water level vector in the real-time water level vector is greater than the first threshold, it is determined to be an engine bottleneck, triggering the access-side differentiated throttling mode. Based on the differentiated throttling mode on the admission side, a hierarchical flow control decision is executed, and a forwarding rate limit is generated and sent to the gateway layer. The gateway layer then limits the rate of requests forwarded to the engine layer. The system employs a tiered approach based on computational complexity scores. If the first complexity level is met, an overload code is returned. If the second complexity level is met, the system is downgraded to asynchronous batch processing and enters a low-priority queue. If the third complexity level is met, the request is granted normal access.

4. The concurrent request flow control method for speech recognition services as described in claim 2, characterized in that, When the gateway water level vector in the real-time water level vector is greater than the second threshold, it is determined to be a gateway bottleneck, and the forwarding side active rate limiting mode is triggered. According to the forwarding-side active rate limiting mode, if the computational complexity score meets the first complexity level, an explicit rate limiting instruction is generated and sent to the client based on the exponential backoff suppression analysis for the forwarding rate, wherein the explicit rate limiting instruction carries a suggested sending interval. The gateway layer caches a rate-limiting token bucket locally based on the explicit rate-limiting instruction, and releases forwarding tokens with the instruction speed as a constraint.

5. The concurrent request flow control method for speech recognition service as described in claim 1, characterized in that, The request processing module includes: The first extraction node is deployed by extracting acoustic fingerprints and semantic fingerprints in parallel. A cascading query chain is deployed based on a three-layer cache group architecture, wherein the three-layer cache group includes a local memory cache layer, an edge node cache layer, and a global distributed cache layer. By associating the first extraction node with the cascading query link, a request processing module is formed.

6. The concurrent request flow control method for speech recognition service as described in claim 5, characterized in that, The local memory cache layer is deployed inside each gateway node, and uses a high-speed hash table to store recent request fingerprints and identification results, and performs interception of duplicate requests; The edge node caching layer is deployed on each edge caching node, adopts distributed caching and covers multiple gateway nodes in the same area, and is used to share the results of duplicate requests in the same area. The global distributed cache layer is deployed in the central cloud and uses key-value storage to cover the entire service cluster for cross-regional global deduplication and cold start cache filling.

7. The concurrent request flow control method for speech recognition service as described in claim 6, characterized in that, Perform extraction based on acoustic fingerprint and semantic fingerprint, employ a cache query based on a cascading query chain, and generate request processing results, including: Based on the first extraction node, audio content summary extraction based on spectral hashing is performed on the request to be processed, and combined with text embedding based on historical confidence recognition results to obtain the request identifier; The request identifier is input into the cascading query chain, and a query based on the local memory cache layer is executed. If the query is successful, the pending request is not counted in the concurrency count, and a local cache instruction is generated. If no match is found, perform sequential queries based on the edge node caching layer and the global distributed caching layer. If no match is found, add the pending request to the concurrency count and generate a queue entry instruction.

8. The concurrent request flow control method for speech recognition service as described in claim 1, characterized in that, The method further includes: For the request source of the concurrent audio stream, multi-dimensional behavioral features are collected, including request frequency, length distribution, content repetition, endpoint detection activity, and regularity of request time intervals. Based on the aforementioned multidimensional behavioral characteristics, an anomaly detector is used to calculate an anomaly score; Based on the anomaly score, the request source is marked as a malicious source, and processing constraints are imposed on the admission decision stage and the request processing link.

9. The concurrent request flow control method for speech recognition service as described in claim 8, characterized in that, An anomaly detector is used to calculate an anomaly score, including: Using the isolated forest approach, the feature space of the multidimensional behavioral features is randomly and recursively divided to construct multiple isolated trees, and the first anomaly score is calculated based on the path length within the tree. Density clustering is used to calculate the local reachability density of the multidimensional behavioral features in the feature space to determine the second anomaly score. The first abnormal score and the second abnormal score are weighted and fused to obtain the abnormal score.

10. A concurrent request flow control system for voice recognition services, characterized in that, The system is used to implement the concurrent request flow control method for the speech recognition service according to any one of claims 1-9, the system comprising: The computational complexity score generation module is used to read periodic concurrent audio streams during the request admission phase, extract audio features through a lightweight engine and perform pre-analysis to generate a computational complexity score. The audio features include duration, effective speech ratio, background noise energy and endpoint detection activity. The pending request determination module is used in the admission decision stage to perform bottleneck type judgment based on real-time water level vector and admission mode analysis based on computational complexity score by using the gateway-engine collaborative scheduling module to perform queue admission under the patterned deployment and determine pending requests. The request processing result generation module is used in the request processing chain to perform extraction and combination based on acoustic fingerprint and semantic fingerprint on the request to be processed, and to perform cache query based on cascading query chain to generate request processing result, wherein the request processing result includes cache instruction and queue entry instruction; The flow control management module is used to perform flow control management on the pending request based on the request processing result.

Images