Intelligent display real-time voice communication method based on edge computing

Abstract

The application relates to an intelligent display real-time voice communication method based on edge computing, in particular to the field of intelligent displays, which fuses the multidimensional states of local and neighbor nodes in real time and generates standardized features, drives a space-time attention model to accurately predict future loads and task types, based on the prediction, a reinforcement learning scheduler combines a quality of service strategy to prospectively allocate computing resources, schedules high real-time tasks to local idle AI units, collaboratively schedules decomposable tasks to the lightest load neighbor nodes, and efficiently offloads deep semantic tasks to the cloud, executes the process according to the decision elastic scaling container instance, continuously fine-tunes the prediction and scheduling model by comparing actual and expected data, and the closed loop realizes low-delay and high-reliability processing of concurrent voice tasks and dynamic optimal adaptation of system resources in a resource-limited edge environment, significantly improving the interaction real-time performance, resource utilization and overall intelligent collaboration level of the system.

Description

Technical Field

[0001] This invention relates to the field of smart displays, and more specifically, to a real-time voice communication method for smart displays based on edge computing. Background Technology

[0002] With the deep integration of IoT and AI technologies, smart home and smart office environments with smart displays as the core of interaction are gradually becoming more widespread. In these scenarios, multiple AI smart displays with ambient light sensing, human body sensing, and voice recognition capabilities are deployed in a distributed manner to form a network for collaborative perception and interaction. Users expect to interact instantly with the nearest display through natural voice, such as controlling lights and curtains in the living room, looking up recipes in the kitchen, initiating a video conference in the study, or issuing instructions to check schedules and documents in an open office area. These interaction requests are usually random and concurrent, especially during family gatherings or team collaborations, where multiple users may simultaneously or nearly simultaneously initiate voice calls to the devices in front of them. To ensure the real-time nature and privacy of the interaction, a large number of voice processing tasks, including audio acquisition, real-time noise reduction, wake word detection, and preliminary voice recognition, are deployed on local edge computing nodes. This node may be an enhanced home gateway or a local lightweight server. This architecture aims to reduce reliance on cloud services, reduce network transmission latency, and protect the privacy of user voice data.

[0003] However, this architecture, which relies on edge computing nodes for real-time voice processing, faces a severe technical challenge: the imbalance between the limited computing resources of edge nodes and the dynamic, high-concurrency voice processing load leads to processing latency and response jitter. Due to cost and power consumption limitations, the computing power of edge computing nodes, including the core resources of general-purpose CPUs, memory, and dedicated AI acceleration units, is far less than that of cloud data centers. Existing technical solutions typically reserve fixed computing resources for different voice processing units or adopt a simple polling static scheduling strategy. This rigid resource allocation method cannot effectively perceive and predict real-time changes in load pressure. When multiple users initiate interactions simultaneously, and edge nodes need to process multiple voice streams in parallel, the statically allocated resource pool is easily exhausted quickly, forcing subsequent tasks to enter a queuing state, directly causing voice command response delays. More complexly, different voice tasks have significantly different computational resource requirements. For example, the computational requirements for simple fixed instruction recognition and open-domain continuous speech understanding are vastly different. A uniform static resource allocation cannot achieve fine-grained optimization, resulting in low resource utilization or insufficient performance on critical tasks. Although there are solutions to offload tasks to the cloud when high load is detected, this approach introduces additional network transmission latency and may degrade the experience due to instantaneous network congestion, which violates the original intention of using edge computing to achieve low-latency localized processing. Therefore, existing technologies lack a mechanism that can deeply integrate perception, prediction, and scheduling capabilities to achieve intelligent and elastic allocation of computing resources within and between edge nodes, thereby ensuring the real-time performance and overall system stability of voice interaction tasks, especially high-priority tasks, even under sudden concurrency scenarios. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by providing a real-time voice communication method for smart displays based on edge computing, thereby solving the problems mentioned in the background art.

[0005] The technical solution of this invention to solve the above-mentioned technical problems is as follows: specifically, it includes the following steps:

[0006] Step S1: The edge computing node periodically collects and merges the local system resource indicators, local pending voice task queue status, local voice task computing characteristics, and obtains a snapshot of the load status of at least one neighboring edge computing node from the edge network through the edge network heartbeat protocol to generate a standardized multi-dimensional feature vector; wherein, the local system resource indicators include the utilization rate of the central processing unit, the memory utilization rate, and the utilization rate of the dedicated AI acceleration unit.

[0007] Step S2: Input the multidimensional feature vector generated in step S1 into a spatiotemporal attention prediction model. The spatiotemporal attention prediction model synchronously analyzes the multidimensional feature vector and outputs the predicted sequence of system resource requirements and the probability distribution of task types within a future set time window.

[0008] Step S3: Based on the predicted sequence and task type distribution probability output in Step S2, combined with the local system resource indicators collected in Step S1 and the service quality policy pre-stored by the edge computing nodes, a forward-looking scheduling decision is generated through a reinforcement learning-optimized dynamic scheduler, and the expected resource consumption and expected processing time corresponding to the forward-looking scheduling decision are generated simultaneously. The forward-looking scheduling decision includes: allocating high real-time voice tasks to the idle dedicated AI acceleration units of the edge computing nodes, allocating decomposable non-real-time subtasks to the lightest-loaded neighbor edge computing nodes in the neighbor edge computing node load status snapshot obtained in Step S1, and uploading only the feature data of voice tasks requiring deep semantic understanding to the cloud.

[0009] Step S4: Based on the forward-looking scheduling decision generated in Step S3, the speech processing instance in the edge computing node is elastically scaled through the container management engine, and the actual resource consumption and processing time data after the task is completed are collected. The actual resource consumption and processing time data are compared with the prediction sequence output in Step S2 and the expected resource consumption and expected processing time generated in Step S3. The resulting difference data is used to fine-tune the spatiotemporal attention prediction model in Step S2 and the dynamic scheduler optimized by reinforcement learning in Step S3 online.

[0010] In a preferred embodiment, step S1, which involves the edge computing node periodically collecting local system resource indicators, local pending voice task queue status, and local voice task computing characteristics, and obtaining a snapshot of the load status of at least one neighboring edge computing node from the edge network via the edge network's heartbeat protocol, specifically includes:

[0011] The edge computing node performs data acquisition operations when each fixed monitoring cycle arrives, or when the local system resource utilization exceeds a dynamically adjusted threshold, or when triggered by a new voice task arrival event.

[0012] Local system resource metrics include: the first utilization value, the second utilization value, and the third utilization value obtained by calling the performance interface provided by the operating system kernel and querying the hardware performance counter. The first utilization value, the second utilization value, and the third utilization value are arranged in order to form a local resource status triplet.

[0013] The status of the local pending voice task queue includes: the total number of voice tasks currently waiting to be processed, obtained from the task scheduler; the average waiting time of each task calculated based on the timestamp of each task in the queue; and the statistically obtained first proportion of high-priority voice tasks to the total number of tasks in the queue, second proportion of medium-priority voice tasks to the total number of tasks in the queue, and third proportion of low-priority voice tasks to the total number of tasks in the queue. The total number of voice tasks, the average waiting time of each task, and the task priority ratio triplet composed of the first proportion, the second proportion, and the third proportion together constitute the status data representing the pressure on the task queue.

[0014] The local speech task computation features include: real-time analysis of the input audio stream corresponding to the newly arrived speech task through a lightweight pre-analysis unit to estimate the estimated duration of the audio segment; evaluation of the estimated signal-to-noise ratio scalar value of the audio segment; and determination of the task type identifier to which the speech task belongs. The estimated audio duration, the estimated signal-to-noise ratio scalar value, and the task type identifier are sequentially combined into a task feature triplet.

[0015] The load status snapshot of neighboring edge computing nodes includes: receiving status messages periodically broadcast by multiple neighboring edge computing nodes through a UDP multicast heartbeat protocol established with neighboring edge computing nodes; each status message contains at least one neighboring edge computing node's CPU utilization, memory utilization, and dedicated AI acceleration unit utilization; multiple sets of triplets consisting of CPU utilization, memory utilization, and dedicated AI acceleration unit utilization received from all neighboring edge computing nodes are grouped by neighboring edge computing nodes and arranged in a fixed order of CPU utilization, memory utilization, and dedicated AI acceleration unit utilization to form a set of neighboring edge computing node load statuses.

[0016] In a preferred embodiment, the process of generating a standardized multidimensional feature vector specifically includes:

[0017] First, the first utilization rate value, the second utilization rate value, and the third utilization rate value contained in the local resource status triplet are subjected to dynamic weight standardization processing based on exponential decay to obtain the local resource standardized vector; the average waiting time of the task is subjected to dynamic weight standardization processing based on exponential decay to obtain the queue waiting time standardized value; the estimated audio duration and the estimated signal-to-noise ratio scalar value contained in the task feature triplet are subjected to dynamic weight standardization processing based on exponential decay to obtain the task duration standardized value and the task signal-to-noise ratio standardized value.

[0018] Secondly, the first, second, and third proportions in the task priority ratio triplet, as well as the task type identifier, are vectorized using one-hot encoding to obtain the task priority encoding vector and the task type encoding vector, respectively.

[0019] Next, the CPU utilization, memory utilization, and dedicated AI acceleration unit utilization of each neighbor edge computing node in the neighbor edge computing node load state set are linearly normalized, and the processed values ​​of all neighbor edge computing nodes are flattened and connected in node order to form a neighbor state flattening vector.

[0020] Finally, the local resource normalization vector, queue waiting time normalization value, task duration normalization value, task signal-to-noise ratio normalization value, task priority encoding vector, task type encoding vector, and neighbor state flattening vector are concatenated into a normalized multidimensional feature vector in a predefined fixed order.

[0021] In a preferred embodiment, the specific process of synchronously analyzing multidimensional feature vectors by the spatiotemporal attention prediction model in step S2 is as follows:

[0022] First, based on the standardized multidimensional feature vectors generated in step S1 at multiple consecutive historical moments, temporal context codes and spatial context codes are extracted from them. The extraction of temporal context codes is completed by a gated recurrent unit network processing the standardized multidimensional feature vectors at multiple consecutive historical moments. The gated recurrent unit network outputs a fixed-dimensional vector as the temporal context code through its internal update gate and reset gate mechanism.

[0023] The extraction of spatial context encoding is accomplished by processing the flattened neighbor state vector parsed from the normalized multidimensional feature vector using a graph attention network, specifically as follows:

[0024] A1. Reshape the flattened neighbor state vector into a neighbor node feature matrix. The rows of this neighbor node feature matrix correspond to different neighbor edge computing nodes, and the columns correspond to three types of node resource indicators: CPU utilization, memory utilization, and utilization of dedicated AI acceleration units.

[0025] A2. Use a learnable parameter weight matrix to perform a linear transformation on the node features represented by each row in the neighbor node feature matrix to obtain the transformed feature vector.

[0026] A3. Calculate an attention coefficient for each row in the neighbor node feature matrix that represents the edge computing node itself and each of its neighbor edge computing nodes.

[0027] A4. Based on all the calculated attention coefficients, perform a weighted summation on the transformed feature vectors of each neighboring edge computing node and the edge computing node itself, and use an activation function to process the weighted summation result. Finally, output a fixed-dimensional vector as a spatial context code.

[0028] In a preferred embodiment, the process by which the spatiotemporal attention prediction model outputs the prediction sequence and the probability distribution of task type is specifically as follows:

[0029] First, the obtained temporal context encoding and spatial context encoding are adaptively fused to generate a joint context representation;

[0030] Then, this joint context representation is input into a decoder network consisting of multiple fully connected layers; the decoder network outputs two results in parallel:

[0031] The first output is a predicted sequence of system resource requirements within a set number of future time steps. The prediction result for each time step is represented as a three-dimensional real number vector. The first value in the three-dimensional real number vector represents the predicted value of the CPU utilization in the future time step, the second value represents the predicted value of the memory utilization in the future time step, and the third value represents the predicted value of the utilization rate of the dedicated AI acceleration unit in the future time step.

[0032] The second output is a three-dimensional real vector. This three-dimensional real vector is processed by the softmax normalization function to obtain the prior probability distribution of the task type. The prior probability distribution of the task type is represented as a three-dimensional real vector. Its first element represents the probability that the newly arrived speech task within the future set time window belongs to the simple command word recognition type, the second element represents the probability that it belongs to the continuous speech dictation type, and the third element represents the probability that it belongs to the free dialogue comprehension type. The sum of these three probability values ​​is one.

[0033] In a preferred embodiment, the adaptive fusion is performed by an adaptive gated fusion unit that generates a joint context representation through operations:

[0034] B1. Concatenate the temporal context code and the spatial context code to form a merged code vector;

[0035] B2. Use a learnable weight matrix to perform a linear transformation on the merged encoded vector, and then add a learnable bias vector to obtain an intermediate vector.

[0036] B3. Apply the sigmoid activation function to the intermediate vector to generate a gated vector;

[0037] B4. Multiply the gating vector element-wise with the temporal context code to obtain the first product vector; at the same time, calculate the difference between the first and the gating vector to obtain the complementary gating vector, and multiply the complementary gating vector element-wise with the spatial context code to obtain the second product vector.

[0038] B5. Add the first product vector to the second product vector and output the result as a joint context representation.

[0039] In a preferred embodiment, step S3, which involves generating a forward-looking scheduling decision using a reinforcement learning-optimized dynamic scheduler, specifically comprises:

[0040] First, an enhanced state representation is constructed, which is composed of local resource state triples, all predicted values ​​in the prediction sequence, the prior probability distribution of the task type, and the flattened vector of the neighbor state in sequence.

[0041] Secondly, the dynamic scheduler maintains a set of optional actions including local processing, neighbor cooperative processing, and cloud processing, and uses a two-branch evaluation network to evaluate each optional action. The first branch of the two-branch evaluation network is the value evaluation branch, which receives the enhanced state representation and a candidate action as input and outputs a value score representing the long-term expected benefit of performing the action. The second branch of the two-branch evaluation network is the constraint and cost evaluation branch, which calculates real-time evaluation indicators for different action types.

[0042] For actions involving offloading to a neighboring edge computing node, the immediate evaluation metric is collaborative load balancing.

[0043] For actions uploaded to the cloud, the real-time evaluation metric is cloud communication utility;

[0044] Finally, the dynamic scheduler calculates a comprehensive decision score for each optional action based on a preset service quality policy. The comprehensive decision score is obtained by multiplying the first weighting coefficient by the value score, adding the second weighting coefficient by the corresponding real-time evaluation indicator, and then subtracting the third weighting coefficient by the risk estimate of the action violating the service quality policy. The dynamic scheduler selects actions to generate forward-looking scheduling decisions based on the following logic: for voice tasks identified as high real-time tasks, it prioritizes determining whether the comprehensive decision score of the local processing action is higher than a first preset threshold and whether the utilization rate of the dedicated AI acceleration unit of the edge computing node is lower than a fourth preset threshold representing an idle state. If these conditions are met, local processing is selected. If the conditions are not met, further determine whether the voice task is a decomposable non-real-time task. If so, select actions with a collaborative load balancing index higher than the second preset threshold from all actions offloaded to neighboring edge computing nodes, and select the action with the highest comprehensive decision score. The neighboring edge computing node corresponding to this action is considered the node with the lightest load. If there are still no actions that meet the conditions, and the prior probability distribution of the task type indicates that the current voice task has the highest probability of belonging to the free dialogue understanding type, that is, it is judged to be a voice task that requires deep semantic understanding, then evaluate the comprehensive decision score of the action uploaded to the cloud. If its cloud communication utility index is higher than the third preset threshold, then select the action.

[0045] In a preferred embodiment, the process of synchronously generating the expected resource consumption and expected processing time corresponding to the forward-looking scheduling decision specifically includes:

[0046] For each specific speech task allocation scheme in the generated forward scheduling decision, the dynamic scheduler calculates the expected resource consumption and expected processing time in parallel based on the task type, allocation target and related prediction and status information of the speech task.

[0047] The expected processing time is obtained by adding the baseline calculation time, network transmission time, and queuing time.

[0048] The network transmission time is determined based on the allocation target: if the voice task is allocated to local processing, the network transmission time is zero; if it is offloaded to a neighboring edge computing node, the network transmission time is the estimated network link delay to that neighboring edge computing node; if it is uploaded to the cloud, the network transmission time is the round-trip network transmission delay to the cloud.

[0049] The queuing time is calculated by querying the current voice task queue length of the target dedicated AI acceleration unit or cloud server, and combining it with the predicted value of the corresponding computing unit utilization in the prediction sequence for the target node output in step S2, through a function that outputs the queuing delay based on the queue length and the utilization prediction value.

[0050] The calculation of expected resource consumption adopts different models depending on the allocation target: if the voice task is allocated to a dedicated AI acceleration unit of a local or neighboring edge computing node, the expected resource consumption is represented by the amount of computation of the dedicated AI acceleration unit expected to be required to complete the voice task; if the voice task is uploaded to the cloud, the expected resource consumption is represented by the network transmission resources expected to be occupied by uploading the data corresponding to the voice task used to generate the local voice task computation features in step S1.

[0051] In a preferred embodiment, the process of comparing the actual resource consumption and processing time data with the predicted sequence output in step S2 and the expected resource consumption and expected processing time generated in step S3 in step S4 specifically involves:

[0052] First, the container management engine receives and parses the forward-looking scheduling decision generated in step S3. Based on the forward-looking scheduling decision generated in step S3, it creates or destroys container instances running the specified speech processing module within milliseconds by calling the container runtime interface, thus completing the elastic orchestration and execution of computing resources. Throughout the entire execution process of a speech task, from its scheduling and allocation to the availability of its processing results, monitoring probes deployed inside the container instance and on the host machine continuously collect the actual resource consumption data and the actual end-to-end processing time data after the task is completed. The actual resource consumption data includes at least the actual CPU time occupied by the task, the peak memory usage, and the cumulative computing time of the dedicated AI acceleration unit.

[0053] Secondly, each piece of actual resource consumption data and actual end-to-end processing time data collected is associated and bound with the unique identifier of the voice task that generated the data, the task type of the voice task, and the forward scheduling decision information in step S3 that triggers the execution of the voice task, to form an execution trajectory record containing task identity, decision basis and execution result.

[0054] Next, the actual resource consumption data and actual end-to-end processing time data in the execution trajectory record are compared with the corresponding baseline data from steps S2 and S3 in a multi-dimensional comparison. The comparison operation generates two types of error data: the first type is the resource prediction error vector; the second type is the scheduling expectation error.

[0055] Finally, for each set of error data generated by comparison, a data credibility weight is calculated. The calculation of the data credibility weight is based on the intensity of the unforeseen system disturbances encountered by the speech task corresponding to the set of error data during execution, and whether the task was completed normally. The error data associated with the data credibility weight, the historical decision state recorded in the corresponding execution trajectory record, and the scheduling action executed are integrated to form a data record set for subsequent online fine-tuning.

[0056] In a preferred embodiment, the process of using the generated difference data for online fine-tuning of the spatiotemporal attention prediction model in step S2 and the dynamic scheduler optimized by reinforcement learning in step S3 specifically involves:

[0057] First, the online fine-tuning process is initiated by periodic triggering or triggering when the cumulative error exceeds a threshold; the data record set used for online fine-tuning is organized into batches to form data batches for model fine-tuning;

[0058] Secondly, the spatiotemporal attention prediction model in step S2 is fine-tuned: taking the actual resource consumption data contained in each data record in the data batch as the true value, and the corresponding resource utilization prediction value output in step S2 as the model prediction value, a loss function is constructed. This loss function is equal to the sum of the squares of the numerical errors between the data confidence weights of all data records in the data batch and the corresponding predicted values ​​of the actual resource consumption data; the parameters of the spatiotemporal attention prediction model are updated using the mini-batch gradient descent algorithm combined with the elastic weight consolidation technique.

[0059] Finally, the dynamic scheduler optimized by reinforcement learning in step S3 is fine-tuned: the temporal difference error is calculated by taking the historical decision state, the scheduling action executed, the actual immediate gain and subsequent state information contained in each data record in the data batch as input; the calculated temporal difference error is used to update the network parameters of the value evaluation branch in the dynamic scheduler through the gradient descent method, thereby calibrating its estimation of the long-term expected gain of different scheduling actions and making it closer to the feedback of the real environment.

[0060] The beneficial effects of this invention are as follows: By fusing the multidimensional states of local and neighboring nodes in real time and generating standardized features, the spatiotemporal attention model is driven to accurately predict future load and task types. Based on this prediction, the reinforcement learning scheduler, combined with the quality of service strategy, proactively allocates computing resources, schedules high real-time tasks to local idle AI units, coordinates decomposable tasks to the lightest-loaded neighboring nodes, and efficiently offloads deep semantic tasks to the cloud. During execution, the container instances are elastically scaled according to the decision, and the prediction and scheduling model is continuously fine-tuned online by comparing actual and expected data. This closed loop achieves low-latency, high-reliability processing of concurrent voice tasks and dynamic optimal adaptation of system resources in resource-constrained edge environments, significantly improving the real-time interaction, resource utilization, and overall intelligent collaboration level of the system. Attached Figure Description

[0061] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0064] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0065] Example 1

[0066] This embodiment provides, for example Figure 1 The present invention discloses a real-time voice communication method for smart displays based on edge computing, which specifically includes the following steps:

[0067] Step S1: The edge computing node periodically collects and merges the local system resource indicators, local pending voice task queue status, local voice task computing characteristics, and obtains a snapshot of the load status of at least one neighboring edge computing node from the edge network through the edge network heartbeat protocol to generate a standardized multi-dimensional feature vector; wherein, the local system resource indicators include the utilization rate of the central processing unit, the memory utilization rate, and the utilization rate of the dedicated AI acceleration unit.

[0068] Step S2: Input the multidimensional feature vector generated in step S1 into a spatiotemporal attention prediction model. The spatiotemporal attention prediction model synchronously analyzes the multidimensional feature vector and outputs the predicted sequence of system resource requirements and the probability distribution of task types within a future set time window.

[0069] Step S3: Based on the predicted sequence and task type distribution probability output in Step S2, combined with the local system resource indicators collected in Step S1 and the service quality policy pre-stored by the edge computing nodes, a forward-looking scheduling decision is generated through a reinforcement learning-optimized dynamic scheduler, and the expected resource consumption and expected processing time corresponding to the forward-looking scheduling decision are generated simultaneously. The forward-looking scheduling decision includes: allocating high real-time voice tasks to the idle dedicated AI acceleration units of the edge computing nodes, allocating decomposable non-real-time subtasks to the lightest-loaded neighbor edge computing nodes in the neighbor edge computing node load status snapshot obtained in Step S1, and uploading only the feature data of voice tasks requiring deep semantic understanding to the cloud.

[0070] Step S4: Based on the forward-looking scheduling decision generated in Step S3, the speech processing instance in the edge computing node is elastically scaled through the container management engine, and the actual resource consumption and processing time data after the task is completed are collected. The actual resource consumption and processing time data are compared with the prediction sequence output in Step S2 and the expected resource consumption and expected processing time generated in Step S3. The resulting difference data is used to fine-tune the spatiotemporal attention prediction model in Step S2 and the dynamic scheduler optimized by reinforcement learning in Step S3 online.

[0071] In this embodiment, it is specifically necessary to explain the process in step S1, in which the edge computing node periodically collects the local system resource indicators, the status of the local voice task queue, the local voice task computing characteristics, and obtains a snapshot of the load status of at least one neighboring edge computing node from the edge network through the heartbeat protocol of the edge network.

[0072] Data collection is performed by edge computing nodes at fixed monitoring intervals, when local system resource utilization exceeds a dynamically adjusted threshold, or when triggered by a new voice task. The fixed monitoring interval can be dynamically adjusted according to the scenario. For example, it can be set to 100 milliseconds during regular working hours in an office scenario (such as 9:00-18:00) and extended to 500 milliseconds during nighttime or low-load periods to balance monitoring overhead and real-time performance. The dynamically adjusted threshold is not a fixed value but an adaptive value based on historical load patterns. For example, it can be set to immediately trigger data collection when the CPU utilization exceeds 70% for three consecutive sampling points or when the utilization of the dedicated AI acceleration unit instantaneously exceeds 85% to capture sudden loads. The combination of event triggering and periodic triggering ensures the timeliness of status collection and the controllability of system overhead.

[0073] Local system resource metrics include: a first utilization rate, a second utilization rate, and a third utilization rate obtained by calling the performance interface provided by the operating system kernel and querying hardware performance counters. The first utilization rate represents the average utilization rate of all logical cores in the edge computing node, ranging from zero to one. The second utilization rate represents the percentage of used memory capacity relative to the total memory capacity, also ranging from zero to one. The third utilization rate represents the computing power utilization rate of the dedicated AI acceleration unit, also ranging from zero to one. The first, second, and third utilization rates are sequentially combined to form a local resource status triplet. Querying hardware performance counters is a direct means of obtaining the utilization rate of the dedicated AI acceleration unit. For example, for a neural network processing unit, the third utilization rate can be accurately calculated by reading the ratio of its streaming multiprocessor active cycle counter to its total cycle counter. The local resource status triplet (r_cpu, r_mem, r_acc) is one of the core underlying inputs for subsequent load assessment and prediction.

[0074] The local pending voice task queue status includes: the total number of voice tasks currently waiting to be processed, obtained from the task scheduler (the total number of voice tasks is a natural number); the average waiting time of each task in the queue, calculated based on the timestamps of each task, in milliseconds; and the statistically determined proportions of high-priority voice tasks, medium-priority voice tasks, and low-priority voice tasks, where the sum of the first, second, and third proportions is one. The total number of voice tasks, the average waiting time, and the task priority ratio triplets composed of the first, second, and third proportions together constitute a single priority ratio. The status data that constitutes the pressure of the task queue can be predefined based on business logic. For example, real-time voice call control commands and device emergency stop commands can be defined as high priority; information query and music playback control can be defined as medium priority; and log upload and non-real-time voice data batch processing can be defined as low priority. The average waiting time of the task directly reflects the real-time processing bottleneck of the edge computing node, while the task priority ratio triplet (p_high, p_medium, p_low) describes the quality composition of the current load, which is crucial for scheduling decisions. For example, when the proportion of high-priority tasks is high, even if the total queue length is not long, more resources may need to be scheduled immediately.

[0075] The local speech task computation features include: real-time analysis of the input audio stream corresponding to the newly arrived speech task using a lightweight pre-analysis unit; estimation of the estimated audio duration in seconds; evaluation of the estimated signal-to-noise ratio (SNR) scalar value; and determination of the task type identifier, which distinguishes between three different speech task categories: simple command word recognition, continuous speech dictation, and free-flowing dialogue comprehension. The estimated audio duration, estimated SNR scalar value, and task type identifier are sequentially combined into a task feature triplet. The lightweight pre-analysis unit can perform this feature without performing full speech recognition. For example, by calculating the rising and falling edges of audio energy, the duration of audio can be roughly estimated; by calculating the ratio of signal power to noise power in silent segments, an estimated signal-to-noise ratio scalar value can be obtained; by analyzing the zero-crossing rate and Mel-frequency cepstral coefficient features of the initial few hundred milliseconds of audio, combined with a simple classifier (such as support vector machine), the task type can be quickly identified. The task feature triplet (c_dur, c_snr, c_type) provides a key basis for predicting the computational resource requirements of different tasks. For example, free dialogue understanding tasks usually consume an order of magnitude more computational resources than simple command word recognition.

[0076] The load status snapshot of neighboring edge computing nodes includes: receiving status messages periodically broadcast by multiple neighboring edge computing nodes through a UDP multicast heartbeat protocol established with neighboring edge computing nodes; each status message contains at least one neighboring edge computing node's CPU utilization, memory utilization, and dedicated AI acceleration unit utilization, with these utilization values ​​ranging from zero to one; multiple sets of triplets consisting of CPU utilization, memory utilization, and dedicated AI acceleration unit utilization received from all neighboring edge computing nodes, grouped by neighboring edge computing node and sorted by CPU utilization, memory utilization, and... The fixed order of utilization rates of dedicated AI acceleration units constitutes the load status set of neighboring edge computing nodes. The period of the UDP multicast heartbeat protocol can be set to 200 milliseconds. The heartbeat message is extremely concise, containing only the necessary utilization values ​​and timestamps. The load status set of neighboring edge computing nodes can be represented in memory as a K-row, 3-column matrix, where K is the number of active neighboring edge computing nodes, and each row corresponds to a node's (CPU utilization, memory utilization, and dedicated AI acceleration unit utilization). This set is the core information foundation for achieving end-to-end collaborative scheduling, enabling local nodes to perceive the resource distribution of the entire edge network.

[0077] The process of generating a standardized multidimensional feature vector is as follows:

[0078] First, the first utilization rate value, the second utilization rate value, and the third utilization rate value contained in the local resource status triplet are subjected to dynamic weight standardization processing based on exponential decay to obtain the local resource standardized vector; the average waiting time of the task is subjected to dynamic weight standardization processing based on exponential decay to obtain the queue waiting time standardized value; the estimated audio duration and the estimated signal-to-noise ratio scalar value contained in the task feature triplet are subjected to dynamic weight standardization processing based on exponential decay to obtain the task duration standardized value and the task signal-to-noise ratio standardized value.

[0079] The dynamic weight standardization process based on exponential decay is as follows: The first, second, and third utilization values ​​in the local resource status triplet, or the average task wait time, or the estimated audio duration and estimated signal-to-noise ratio scalar value in the task feature triplet, are recorded as original feature values ​​for processing. Within a 60-second historical time window, the minimum value among all the values ​​that the original feature value belongs to is found and recorded as the historical minimum value. Then, a second intermediate value is calculated, which is equal to the historical maximum value minus the historical minimum value. Finally, a very small positive constant is added to the second intermediate value to prevent issues in subsequent division operations. If the divisor is zero, the third intermediate value is calculated. This third intermediate value equals the first intermediate value divided by the second intermediate value, which includes a zero-prevention constant. Next, the time decay weighting factor is calculated. This time decay weighting factor equals a preset base weighting coefficient, plus the difference between the base weighting coefficient and the base weighting coefficient, multiplied by an exponential function with a natural constant as the base and a negative decay coefficient multiplied by the time difference as the exponent. The decay coefficient is a preset positive number, and the time difference is the time interval between the current moment and the previous voice task arrival event or the local system resource utilization exceeding the dynamic adjustment threshold. Finally, the standardized result of the original feature value equals the third intermediate value multiplied by the time decay weighting factor.

[0080] The selection of the historical time window (e.g., 60 seconds) is to balance the memory of recent load patterns with the forgetting of old data, preventing extremely small positive constants such as 1e-7 from being divided by zero. The preset base weight coefficient α is recommended to be between 0.6 and 0.8, for example, 0.7. This ensures that even if the system remains stable for a long time, historical statistics still have a certain weight. The decay coefficient β is recommended to be between 0.05 and 0.2, for example, 0.1. It controls the speed at which the weight decays from the peak (close to 1) to the base weight α. The design intent of the formula W(t)=α+(1-α)*exp(-β*Δt) is that when the system has just experienced a high load event (Δt), the system will still have a certain weight. When t is small, W(t) is close to 1, and the current instantaneous value is given the maximum weight in the standardization, which makes the feature vector sensitive to the instantaneous changes in the system. As time calms down (Δt increases), W(t) decays exponentially to close to α. At this time, the standardization result depends more on the historical range of the past 60 seconds (historical maximum and minimum values), which enhances the robustness of the feature in the steady period and avoids drastic changes in the feature due to fluctuations in a single instantaneous value. This dynamic weighting mechanism is a key improvement over simple linear normalization. It enables the generated multidimensional feature vector to capture both sudden states and smooth daily fluctuations, providing a better input for the stable prediction of the subsequent model.

[0081] Secondly, the first, second, and third proportions in the task priority ratio triplet, as well as the task type identifier, are vectorized using one-hot encoding to obtain task priority encoding vectors and task type encoding vectors, respectively. For example, the task priority ratio triplet (0.2, 0.5, 0.3) can be expanded by one-hot encoding, and the high, medium, and low priorities can be represented by three-dimensional binary vectors [1, 0, 0], [0, 1, 0], and [0, 0, 1], respectively. Then, the weighted sums are obtained according to the proportions to obtain the final task priority encoding vector, such as (0.2, 0.5, 0.3). The task type identifier (such as 0, 1, 2) is directly encoded into a three-dimensional vector, such as the simple command word recognition as [1, 0, 0]. This encoding method transforms the classification information into a numerical vector that the model can process, while retaining its discrete characteristics.

[0082] Next, the CPU utilization, memory utilization, and dedicated AI acceleration unit utilization of each neighbor edge computing node in the neighbor edge computing node load state set are linearly normalized to map the values ​​of CPU utilization, memory utilization, and dedicated AI acceleration unit utilization to the range of zero to one. The processed values ​​of all neighbor edge computing nodes are flattened and connected in node order to form a neighbor state flattening vector. The linear normalization can be (x-0) / (1-0)=x, since the utilization itself is already in the range of [0,1]. The flattening and connecting operation is to concatenate the K-row, 3-column neighbor state matrix in row priority order into a one-dimensional vector of length 3K. The neighbor state flattening vector integrates the state information of multiple nodes in the distributed network into a unified data structure.

[0083] Finally, the local resource normalization vector, queue waiting time normalization value, task duration normalization value, task signal-to-noise ratio normalization value, task priority encoding vector, task type encoding vector, and neighbor state flattening vector are concatenated into a normalized multidimensional feature vector in a predefined fixed order. The predefined fixed order is as follows: [local resource normalization vector (3D), queue waiting time normalization value (1D), task duration normalization value (1D), task signal-to-noise ratio normalization value (1D), task priority encoding vector (3D), task type encoding vector (3D), neighbor state flattening vector (3KD)]. This final generated normalized multidimensional feature vector is a high-dimensional, dense, and scale-uniform mathematical representation that integrates local real-time load, task characteristics, queue pressure, and a global network resource view. It is the only and complete input for the subsequent spatiotemporal attention prediction model to perform accurate analysis and is the data foundation of the entire intelligent scheduling method. The order of each part in the vector must be kept consistent during the training and inference phases.

[0084] In this embodiment, the specific process of synchronously analyzing multi-dimensional feature vectors by the spatiotemporal attention prediction model in step S2 is as follows:

[0085] First, based on the standardized multidimensional feature vectors generated in step S1 at multiple consecutive historical moments, temporal context encoding and spatial context encoding are extracted. The extraction of temporal context encoding is completed by a gated recurrent unit network (GRN) processing the standardized multidimensional feature vectors at multiple consecutive historical moments. The GRN network captures the evolution and dependency of the standardized multidimensional feature vectors in the time dimension through its internal update and reset gate mechanisms, and outputs a fixed-dimensional vector as the temporal context encoding. The GRN network is a variant of the recurrent neural network. Compared with the Long Short-Term Memory network, it has a simpler structure, fewer parameters, faster training, and is more suitable for deployment in edge computing environments with limited computing resources. The number of consecutive historical moments can be set according to the prediction requirements, such as taking the sequence of the most recent 10 moments (corresponding to the data of the past 1 second). The fixed-dimensional vector can be, for example, 128-dimensional or 256-dimensional. Its dimension is a hyperparameter that determines the model's ability to represent temporal context information.

[0086] The extraction of spatial context encoding is accomplished by processing the flattened neighbor state vector parsed from the normalized multidimensional feature vector using a graph attention network, specifically as follows:

[0087] A1. Reshape the flattened vector of the neighbor state into a neighbor node feature matrix. The rows of the neighbor node feature matrix correspond to different neighbor edge computing nodes, and the columns correspond to three types of node resource indicators: CPU utilization, memory utilization, and utilization of dedicated AI acceleration units. The reshaping operation is to rearrange the flattened vector of length 3K into a matrix of K rows and 3 columns based on the known number of neighbor edge computing nodes K. Each row of the matrix represents the state point of a physical or logical node in the resource space.

[0088] A2. A linear transformation is performed on the node features represented by each row in the neighbor node feature matrix using a learnable parameter weight matrix to obtain the transformed feature vector. The dimension of the learnable parameter weight matrix is ​​determined according to the input feature dimension (3 here) and the expected transformed feature dimension (e.g., 8 dimensions). The purpose of the linear transformation is to project the original utilization index into a higher-dimensional and more expressive feature space, which facilitates the subsequent calculation of the relationship between nodes.

[0089] A3. For each row of the neighbor node feature matrix representing the edge computing node itself and each of its neighboring edge computing nodes, calculate an attention coefficient. The specific process for calculating the attention coefficient is as follows: Concatenate the transformed feature vector representing the edge computing node itself with the transformed feature vector representing one of its neighboring edge computing nodes to form a merged vector. Then, perform a dot product operation with this merged vector using a learnable attention vector. Input the dot product result into a non-linear activation function. Finally, normalize the multiple dot product results calculated for the edge computing node itself and all its neighboring edge computing nodes so that the sum of all attention coefficients is one. This attention coefficient is used to quantify the influence weight of the corresponding neighboring edge computing node on the edge computing node. The learnable attention vector is the target of model training. First, it has learned to "pay attention" to which types of node states are more important for local load prediction. For example, it may learn to pay more attention to neighboring edge computing nodes with high CPU utilization but idle AI acceleration units, because the latter may be more suitable for sharing local AI computing tasks. The non-linear activation function usually adopts LeakyReLU, whose negative slope coefficient can be set to 0.2. The normalization process adopts the softmax function to ensure that the sum of all attention coefficients is 1, forming a probability distribution. This mechanism of dynamically calculating attention coefficients replaces the traditional association model based on fixed distance (such as network hop count) or static weights. It can adaptively capture the complex, time-varying spatial dependencies in the edge network. For example, when the network is congested, the attention coefficient of nodes that are physically close but have poor link quality will automatically decrease.

[0090] A4. Based on all the calculated attention coefficients, the transformed feature vectors of each neighboring edge computing node and the edge computing node itself are weighted and summed. The weighted summation result is then processed using an activation function, and a fixed-dimensional vector is finally output as the spatial context code. The weighted summation operation aggregates global (neighbor + self) information to generate the "spatial receptive field" of the local node. The activation function used can be ReLU, etc., to introduce nonlinearity. The final output spatial context code has the same dimension as the temporal context code (e.g., 128 dimensions) for subsequent fusion. This encoded vector is essentially a high-level, compressed representation of "what the current state of the entire edge network means to the local node".

[0091] The process by which the spatiotemporal attention prediction model outputs the predicted sequence and the probability distribution of task type is as follows:

[0092] First, the obtained temporal context encoding and spatial context encoding are adaptively fused to generate a joint context representation;

[0093] Then, this joint context representation is input into a decoder network consisting of multiple fully connected layers; the decoder network outputs two results in parallel:

[0094] The first output is a predicted sequence of system resource requirements within a set number of future time steps. The prediction result for each time step is represented as a three-dimensional real number vector. The first value in this vector represents the predicted CPU utilization for that time step, the second value represents the predicted memory utilization for that time step, and the third value represents the predicted utilization of the dedicated AI acceleration unit for that time step. All predicted values ​​are between zero and one. The set number of future time steps could be, for example, 5 steps, corresponding to 500 milliseconds in the future (assuming each step is 100 milliseconds). The decoder network can be a multilayer perceptron with 2-3 hidden layers. The prediction sequence can be generated using an "autoregressive" method, that is, using the prediction of the previous step as part of the input to predict the next step; or it can be generated using a "direct multi-output" method, where the decoder outputs the predictions for all time steps at once. The latter has lower latency and is more suitable for real-time systems. Each predicted value is constrained to the range [0,1] by the Sigmoid function to meet the definition of utilization.

[0095] The second output is a three-dimensional real vector. This three-dimensional real vector is processed by the softmax normalization function to obtain the prior probability distribution of task types. This prior probability distribution of task types is represented as a three-dimensional real vector. Its first element represents the probability that a newly arrived voice task within a set future time window belongs to the simple command word recognition type, the second element represents the probability that it belongs to the continuous speech dictation type, and the third element represents the probability that it belongs to the free dialogue comprehension type. The sum of these three probability values ​​is one. The prior probability distribution of task types provides a prediction of the "quality" of future workload. For example, if it is predicted that the probability of free dialogue comprehension tasks is high within the next 500 milliseconds, the scheduler can prepare more AI acceleration resources in advance. The softmax function ensures that the sum of the three probability values ​​is 1, which conforms to the axiom of probability distribution. This distribution, together with the resource demand prediction sequence, provides more comprehensive forward-looking information for scheduling decisions.

[0096] Adaptive fusion is accomplished through an adaptive gated fusion unit, which generates a joint context representation through operations:

[0097] B1. Concatenate the temporal context code and the spatial context code to form a merged encoded vector. B2. Apply a learnable weight matrix to the merged encoded vector using a linear transformation, then add a learnable bias vector to obtain an intermediate vector. B3. Apply a sigmoid activation function to the intermediate vector to generate a gating vector, where each element takes a value between zero and one. B4. Multiply the gating vector element-wise with the temporal context code to obtain the first product vector. Simultaneously, calculate the difference between one and the gating vector to obtain a complementary gating vector, and multiply the complementary gating vector element-wise with the spatial context code to obtain the second product vector. B5. Add the first and second product vectors, and output the result as a joint context representation. The learnable weight matrix and bias vector are optimized during training. The sigmoid function maps each element of the intermediate vector to (0,1). In the given interval, the physical meaning of the generated gate vector g is: for each dimension of the joint context representation, how much information should be retained from the temporal context encoding and how much information should be retained from the spatial context encoding. The complementary gate vector (1-g) is the proportion of spatial information retained. The element-wise multiplication operation allows the model to adopt different fusion strategies on different feature dimensions. For example, on the dimension representing the historical trend of CPU, g may be close to 1, relying more on the temporal context; on the dimension representing the possibility of network collaboration, g may be close to 0, relying more on the spatial context. This fine-grained, data-driven fusion method can more effectively integrate spatiotemporal information than simple vector concatenation or fixed weight addition, thereby generating a more discriminative joint context representation. This is the key to improving the accuracy of subsequent predictions. The final output joint context representation integrates the historical behavior patterns of the system and the current network environment state, and is the cornerstone of the final prediction.

[0098] In this embodiment, it is particularly important to explain the process in step S3, where a forward-looking scheduling decision is generated through a reinforcement learning-optimized dynamic scheduler:

[0099] First, an enhanced state representation for scheduling decisions is constructed. This enhanced state representation is composed of the following parts concatenated in sequence: the local resource state triplet extracted from the local system resource indicators collected in step S1; all predicted values ​​in the predicted sequence of system resource demand in multiple consecutive future time steps output in step S2; the multiple consecutive future time steps from the next time step up to a preset future S-th time step; the prior probability distribution of task types output in step S2; and the neighbor state flattening vector obtained by flattening the resource utilization values ​​of all neighbor edge computing nodes in node order from the set of neighbor edge computing node load states obtained in step S1. This enhanced state representation integrates real-time resource status, future load prediction, task type probability, and a global network view, forming a complete input for scheduling decisions. The enhanced state representation is a high-dimensional real vector. The future S-th time step defines the length of the prediction field of view. For example, S=5, which means considering the prediction information of the next 5 time steps (such as the next 500 milliseconds). The introduction of the neighbor state flattening vector enables scheduling decisions to be based on the resource distribution of the entire edge network, not just the local state. This is the foundation for achieving efficient end-to-end collaboration.

[0100] Secondly, the dynamic scheduler maintains a set of optional actions, including local processing, neighbor cooperative processing, and cloud processing, and uses a two-branch evaluation network to evaluate each optional action. The first branch of this two-branch evaluation network is the value evaluation branch, which receives an enhanced state representation and a candidate action as input and outputs a value score representing the long-term expected benefit of executing the action. The second branch of this two-branch evaluation network is the constraint and cost evaluation branch, which calculates an immediate evaluation index for different action types. The two-branch evaluation network is the core of the scheduler. The value evaluation branch is usually a deep Q-network, whose parameters are trained offline and fine-tuned online through historical scheduling experience and reinforcement learning algorithms (such as deep deterministic policy gradients) to learn strategies that maximize long-term system benefits (such as minimizing average task latency and maximizing resource utilization). The constraint and cost evaluation branch provides a reactive evaluation of the immediate state and network conditions to ensure the feasibility and safety of the decision. This evaluation architecture that combines "long-term value" and "immediate constraints" is the key difference between this scheduler and simple rules or single value networks.

[0101] For the action of offloading to a neighboring edge computing node, the immediate evaluation metric is the collaborative load balancing degree, which is calculated as follows: subtract the product of an absolute value and an exponential decay factor from a number. The absolute value is the absolute value of the quotient obtained by dividing the difference between the current utilization rate of the local dedicated AI acceleration unit and the utilization rate of the dedicated AI acceleration unit of the neighboring edge computing node by the sum of the utilization rates of the local and neighboring dedicated AI acceleration units, plus a very small constant coefficient. The exponential decay factor is the value of an exponential function with a base of a natural constant, an exponent of a negative decay coefficient, and the estimated network link latency to the neighboring edge computing node. The current utilization rate of the local dedicated AI acceleration unit is derived from the third utilization value in the local resource status triplet collected in step S1. The neighboring edge computing node... The utilization rate of the dedicated AI acceleration unit of the node is derived from the data of the corresponding node in the load status set of neighboring edge computing nodes obtained in step S1. The extremely small constant coefficient can be taken as 1e-7 to prevent the denominator from being zero. The attenuation coefficient is a positive number, for example, 0.1, to control the penalty strength of network latency on the collaborative effect. The estimated network link latency can be obtained by periodic ping measurement or historical transmission latency statistics. The design intention of this collaborative load balancing index is: when the utilization rates of the local and neighboring AI acceleration units are close (load balancing) and the network latency is low, the index value is close to 1, indicating high collaborative benefits; when the load difference is large or the network latency is high, the index value decreases. The exponential attenuation factor ensures sensitive penalty for high network latency and avoids inefficient task offloading on congested links.

[0102] For actions uploaded to the cloud, the real-time evaluation metric is cloud communication utility, which is calculated as follows: The estimated computational complexity of the current voice task is divided by the predicted load value of the next local dedicated AI acceleration unit, plus a very small constant coefficient. This sum is then multiplied by the difference between the number one and the recent average packet loss rate of the network to the cloud. The estimated computational complexity of the current voice task is derived from the prior probability distribution of the task type output in step S2. The predicted load value of the next local dedicated AI acceleration unit is derived from the next time step of the predicted sequence of system resource requirements over multiple consecutive time steps output in step S2. The third value of the inter-step prediction vector is a key formula for evaluating edge-cloud collaborative actions. The extremely small constant coefficient is also used to prevent zero. The estimated computational complexity can be easily mapped. For example, the complexity of three types, "simple command word recognition", "continuous speech dictation" and "free dialogue understanding", can be quantified as 1, 3 and 10 respectively. The design intent of this cloud communication utility index is: when the task computational complexity is high, the local AI prediction load is heavy and the cloud network quality is good (low packet loss rate), the utility value is high, encouraging the use of the cloud; otherwise, the utility value is low. This achieves an intelligent trade-off between edge computing and cloud computing.

[0103] Finally, the dynamic scheduler calculates a comprehensive decision score for each optional action based on a preset quality of service (QoS) policy. The comprehensive decision score is obtained by multiplying the first weighting coefficient by the value score, adding the second weighting coefficient by the corresponding real-time evaluation indicator, and then subtracting the third weighting coefficient by the risk estimate of the action violating the QoS policy. The QoS policy predefines at least the following: priority mapping relationships corresponding to different voice task types, maximum end-to-end processing latency constraints allowed for various voice tasks, and requirements for reserving computing resources for high-priority tasks. The dynamic scheduler selects actions based on the following logic to generate a forward-looking scheduling decision: for actions identified as... For high real-time voice tasks, the system prioritizes determining whether the comprehensive decision score of the local processing action is higher than a first preset threshold and whether the utilization rate of the dedicated AI acceleration unit of the edge computing node is lower than a fourth preset threshold representing idle status. If these conditions are met, the local processing action is selected. If not, the system further determines whether the voice task is a decomposable non-real-time task. If so, actions with a collaborative load balancing index higher than a second preset threshold are selected from all actions offloaded to neighboring edge computing nodes, and the action with the highest comprehensive decision score is chosen. The neighboring edge computing node corresponding to this action is considered the node with the lightest load. If no action still meets the conditions, and the task type... The prior probability distribution indicates that the current speech task has the highest probability of belonging to the free dialogue understanding type, i.e., it is judged as a speech task requiring deep semantic understanding. Then, the comprehensive decision score of the action uploaded to the cloud is evaluated. If its cloud communication utility index is higher than the third preset threshold, the action is selected. The first, second, third, and fourth preset thresholds, as well as the first, second, and third weighting coefficients, are all adjustable parameters. For example, the first preset threshold (local processing score threshold) can be set to 0.8, the second preset threshold (collaborative load balancing threshold) can be set to 0.6, the third preset threshold (cloud communication utility threshold) can be set to 0.7, and the fourth preset threshold (AI acceleration unit idle threshold) can be set to 0.2 (i.e., ...). (A usage rate below 20% is considered idle). The weighting coefficients can be fine-tuned after system deployment based on actual results. For example, w_q=0.5, w_c=0.3, w_v=0.2 can be set to balance long-term value, immediate benefits, and violation risks. "Decomposable non-real-time tasks" usually refer to background analysis tasks whose computation process can be divided into multiple independent sub-tasks and are not sensitive to overall processing latency. In this decision-making logic, the "node with the lightest load" is selected by two conditions: "cooperative load balancing degree is higher than the threshold" and "highest comprehensive decision score". It not only considers the load but also takes into account network quality and long-term benefits, making it a more intelligent determination of the lightest load.

[0104] The process of synchronously generating the expected resource consumption and expected processing time corresponding to the forward-looking scheduling decisions is as follows:

[0105] For each specific speech task allocation scheme in the generated forward scheduling decision, the dynamic scheduler calculates the expected resource consumption and expected processing time in parallel based on the task type, allocation target and related prediction and status information of the speech task.

[0106] The expected processing time is obtained by adding the baseline computation time, network transmission time, and queuing time. The baseline computation time is determined by looking up the value in a predefined task type and baseline time mapping table based on the task type of the speech task. The task type and baseline time mapping table is pre-established based on statistical analysis of the execution time of historical tasks on a standard-configured computing unit. For example, the baseline time for simple command word recognition can be 50 milliseconds, continuous speech dictation can be 200 milliseconds, and free dialogue comprehension can be 1000 milliseconds.

[0107] The network transmission duration is determined based on the allocation target: if the voice task is assigned to local processing, the network transmission duration is zero; if it is offloaded to a neighboring edge computing node, the network transmission duration is the estimated network link delay to that neighboring edge computing node; if it is uploaded to the cloud, the network transmission duration is the round-trip network transmission delay to the cloud. The estimated network link delay to the neighboring node and the round-trip network transmission delay to the cloud need to be continuously measured and updated. The round-trip delay takes into account the total time for task data upload and result (or acknowledgment signal) downlink.

[0108] The queuing time is calculated by querying the current voice task queue length of the target dedicated AI acceleration unit or cloud server, and combining it with the predicted value of the corresponding computing unit utilization in the prediction sequence for the target node output in step S2. The queuing delay is calculated by a function that outputs the queuing delay based on the queue length and the predicted utilization value. The function that outputs the queuing delay based on the queue length and the predicted utilization value can be an empirical formula or a small machine learning model. A simple empirical model can be: Queuing time = Average task processing time × Current queue length × (1 + Future utilization prediction value). This model takes into account the existing backlog of tasks and future load pressure, making the estimation of the expected processing time more accurate and providing a reliable benchmark for the feedback comparison in step S4.

[0109] The calculation of expected resource consumption adopts different models depending on the allocation target: if the voice task is assigned to a dedicated AI acceleration unit on a local or neighboring edge computing node, the expected resource consumption is represented by the amount of computation of the dedicated AI acceleration unit expected to be required to complete the voice task; if the voice task is uploaded to the cloud, the expected resource consumption is represented by the network transmission resources expected to be occupied by the data used to generate the local voice task computation features in step S1. For dedicated AI acceleration units, the expected computation can be quantified as "core seconds" or "TOPS (trillion operations per second) seconds". Based on the task type and baseline duration, the expected network transmission resources occupied for cloud transmission can be quantified as "bandwidth seconds" or directly as the expected amount of data to be transmitted. These quantified expected consumption values ​​are compared with the data collected after actual execution (step S4) to accurately assess the accuracy of the scheduling decision in terms of resource utilization efficiency and provide key feedback for model fine-tuning.

[0110] In this embodiment, it is specifically necessary to explain the process in step S4 of comparing the actual resource consumption and processing time data with the predicted sequence output in step S2 and the expected resource consumption and expected processing time generated in step S3.

[0111] First, the container management engine receives and parses the forward-looking scheduling decision generated in step S3. Based on this decision, it creates or destroys container instances running the specified speech processing module within milliseconds by calling the container runtime interface. This completes the elastic orchestration and execution of computing resources. The container management engine can be a deeply customized Operator based on the Kubernetes framework. It listens to the decision message queue from the scheduler. The decision message specifies the speech processing module to be scaled (e.g., "wake word detection v2"), the target number of instances (e.g., "+2"), and resource limits (e.g., "CPU: 0.5, memory: 256Mi, NPU: 1"). Calling the container runtime interface can be based on containerd or CRI-O to achieve rapid (usually within 100 milliseconds) start and stop of container instances. This fine-grained elastic scaling capability allows computing resources to dynamically adjust closely to the predicted load curve, which is key to achieving efficient resource utilization and ensuring real-time performance. In the execution phase, from the moment a voice task is scheduled until its processing result is available, monitoring probes deployed inside the container instance and on the host machine continuously collect actual resource consumption data and actual end-to-end processing time data after the task is completed. Actual resource consumption data includes at least the actual CPU time occupied by the task, the peak memory usage, and the cumulative computing time of the dedicated AI acceleration unit. Actual end-to-end processing time data is the total time elapsed from when the task is assigned by the scheduler until its processing result is available. Monitoring probes include cAdvisor inside the container to collect resource usage within the container, and performance event collectors (such as perf) on the host machine to accurately collect the computing time of the dedicated AI acceleration unit. The end-to-end processing time is collected by adding a unique timestamp during task scheduling and recording the completion timestamp when the processing result is returned. Accurate collection of actual execution data is the data foundation for forming high-quality feedback and driving continuous model optimization.

[0112] Secondly, each piece of actual resource consumption data and actual end-to-end processing time data collected is associated and bound with the unique identifier of the voice task that generated the data, the task type of the voice task, and the forward scheduling decision information in step S3 that triggers the execution of the voice task, forming an execution trajectory record containing task identity, decision basis and execution result. The association and binding operation is completed in memory or a lightweight database to form a structured log. Each execution trajectory record is the smallest data unit for subsequent difference analysis, model training and problem backtracking.

[0113] Next, the actual resource consumption data and actual end-to-end processing time data in the execution trajectory record are compared with the corresponding baseline data from steps S2 and S3 in a multi-dimensional comparison. The comparison operation generates two types of error data: the first type is a resource prediction error vector, each component of which is obtained by subtracting the corresponding value of the predicted value in the prediction sequence of system resource demand within the same future time window output by step S2 from the corresponding value in the actual resource consumption data at the start of task processing, and then taking the absolute value; the second type is the scheduling expectation error, which includes a duration expectation error value and a resource expectation error vector, wherein the duration expectation error value is obtained by subtracting the actual end-to-end processing time data from the actual resource consumption data. The absolute value of the expected processing time data generated in step S3 is taken and then divided by the expected processing time data. Each element of the resource expected error vector is obtained by subtracting the value of the corresponding dimension in the expected resource consumption data generated in step S3 from the value of the corresponding dimension in the actual resource consumption data, taking the absolute value, and then dividing by the value of the corresponding dimension in the expected resource consumption data. This multi-dimensional difference comparison not only compares the final results, but also aligns the actual consumption with the predicted value at the corresponding time, which can accurately locate the source of error: whether it is the overall load prediction is inaccurate (resource prediction error), or the prediction model for the execution of a single task is inaccurate (scheduling expected error). The calculation of relative error makes the errors between tasks of different dimensions and scales comparable.

[0114] Finally, a data reliability weight is calculated for each set of error data. The calculation of the data reliability weight is based on the intensity of the unforeseen system disturbances encountered by the corresponding speech task during execution, and whether the task was completed normally. Specifically, first, a disturbance intensity index representing the disturbance intensity is obtained. Then, using the negative value of the product of a preset positive attenuation coefficient and the disturbance intensity index as the exponent, an exponential function value with a base of the natural constant is calculated. Finally, this exponential function value is multiplied by a completion flag, and the resulting product is the data reliability weight. The completion flag is set to one when the task is completed normally and to zero when the task is abnormally interrupted. The disturbance intensity index I_perturb can be quantified as the maximum value of the network packet loss rate, the burst increase in CPU utilization, etc., observed during task execution. The preset positive decay coefficient λ can be set to, for example, 0.5, to control the decay rate of data credibility caused by disturbances. The intention of the formula (ω=exp(-λ*I_perturb)*CompletionFlag) is to assign high weight (close to 1) to task data that is successfully completed in a stable environment; and to assign low weight or zero weight to task data that is terminated abnormally in a high-disturbance environment. This effectively filters out "noise" data caused by external sudden interference (such as network storms), making the fine-tuning process focus more on the deviation of the model's own prediction and decision, rather than environmental noise, thereby significantly improving the stability and efficiency of online learning. The error data associated with the data credibility weight, the historical decision state recorded in the corresponding execution trajectory record, and the executed scheduling action are integrated to form a data record set for subsequent online fine-tuning.

[0115] The process of using the generated discrepancy data for online fine-tuning of the spatiotemporal attention prediction model in step S2 and the dynamic scheduler optimized by reinforcement learning in step S3 is as follows:

[0116] First, the online fine-tuning process is initiated by either periodic triggering or cumulative error exceeding a threshold triggering. The data record set used for online fine-tuning is organized into batches to form data batches for model fine-tuning. Periodic triggering can be set to execute once every 5 minutes. Cumulative error exceeding a threshold triggering means that when the weighted average error of the data collected in the recent period exceeds a preset threshold (for example, the average relative error exceeds 20%), fine-tuning is triggered immediately. This dual-trigger mechanism balances the timeliness of learning with system overhead.

[0117] Secondly, the spatiotemporal attention prediction model in step S2 is fine-tuned: Using the actual resource consumption data contained in each data record in the data batch as the true value, and the corresponding resource utilization prediction value output in step S2 as the model prediction value, a loss function is constructed. This loss function is equal to the sum of the squares of the numerical errors between the data confidence weights of all data records in the data batch and the corresponding predicted values ​​of the actual resource consumption data. The parameters of the spatiotemporal attention prediction model are updated using a mini-batch gradient descent algorithm combined with elastic weight consolidation technology. The elastic weight consolidation technology, by calculating the importance of model parameters in historical data, imposes constraints on changes in important parameters when learning new data batches to prevent model corruption. When adapting to new data patterns, the model suffers catastrophic forgetting of previously learned patterns. The loss function here is essentially a weighted mean squared error, and the Elastic Weight Consolidation (EWC) technique is key. It evaluates the importance of each parameter by calculating the diagonal value (F_i) of the Fisher information matrix of the model parameters on historical training data. A regularization term is added to the fine-tuning loss function: Σ_iF_i*(θ_i-θ*_i)^2, where θ_i is the new parameter and θ*_i is the old parameter. This penalty constrains the important parameter θ_i from deviating too far from its old value θ*_i. This allows the model to continuously learn new load patterns without forgetting old knowledge, which is crucial for edge intelligent systems that need to continuously adapt to environmental changes during long-term operation.

[0118] Finally, the dynamic scheduler optimized by reinforcement learning in step S3 is fine-tuned: The temporal difference error is calculated using the historical decision state, executed scheduling action, actual immediate gain, and subsequent state information contained in each data record in the data batch as input. The calculation process for the temporal difference error is as follows: the actual immediate gain is added to a discount factor ranging from zero to one, and the maximum future gain prediction value estimated based on subsequent state information; then, the initial gain prediction value estimated based on the historical decision state and the executed scheduling action is subtracted. The immediate gain is obtained by negatively mapping the actual end-to-end processing time data to represent the actual cost of the decision. The maximum future gain prediction value is the maximum value among all possible scheduling action value predictions under subsequent state information. The initial gain prediction value is the value prediction value of the executed scheduling action under the historical decision state. The calculated temporal difference error is used to update the network parameters of the value evaluation branch in the dynamic scheduler through gradient descent, thereby calibrating its estimation of the long-term expected returns of different scheduling actions and making it closer to the feedback of the real environment. This describes the process of optimizing the reinforcement learning agent online with actual interaction data. The calculation formula of temporal difference error (δ) reflects the core idea of ​​reinforcement learning: using actual feedback (immediate return R) and the valuation of subsequent states (maxQ(S')) to correct the valuation of the current state-action pair (Q(S,A)). The discount factor γ can be, for example, 0.9, which determines the present value of future returns. By minimizing the temporal difference error, the scheduler can learn from each successful or poor actual scheduling result, continuously correct its value function, and make its estimation of the long-term value of actions more and more accurate, thereby making better scheduling decisions. This is a kind of intelligence that learns from practice and evolves continuously.

[0119] After fine-tuning, the updated model parameters are synchronously loaded into the online service instances of steps S2 and S3. Subsequently, the performance of steps S2 and S3 with the updated model loaded is evaluated on an isolated validation set independent of the production environment. If the resource prediction error or the simulation processing latency of the scheduling decision deteriorates compared to the model before the update, a rollback operation is triggered, the update is abandoned, and the model parameters before the update are restored to ensure the stability and reliability of the system's core functions. This is a security mechanism to ensure system robustness. The isolated validation set is a set of pre-collected historical states and tasks that do not participate in online fine-tuning. On the validation set, the resource prediction error can be calculated as the root mean square error (RMSE) between the predicted value and the value labeled on the validation set. The simulation processing latency of the scheduling decision is calculated by simulating the decision on the validation set using the updated scheduler model and calculating the average task latency based on the known task execution time (or baseline model). The threshold for triggering rollback can be set when the performance degradation exceeds 5%. This mechanism effectively prevents the decline in online service performance caused by "learning bias" or overfitting that may occur during the online learning process. It is the last line of defense to ensure that the intelligent system can run safely and reliably in complex production environments for a long time.

[0120] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0121] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0122] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0123] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0125] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0126] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A real-time voice communication method for smart displays based on edge computing, characterized in that, Specifically, the steps include the following: Step S1: The edge computing node periodically collects and merges the local system resource indicators, local pending voice task queue status, local voice task computing characteristics, and obtains a snapshot of the load status of at least one neighboring edge computing node from the edge network through the edge network heartbeat protocol to generate a standardized multi-dimensional feature vector; wherein, the local system resource indicators include the utilization rate of the central processing unit, the memory utilization rate, and the utilization rate of the dedicated AI acceleration unit. Step S2: Input the multidimensional feature vector generated in step S1 into a spatiotemporal attention prediction model. The spatiotemporal attention prediction model synchronously analyzes the multidimensional feature vector and outputs the predicted sequence of system resource requirements and the probability distribution of task types within a future set time window. Step S3: Based on the predicted sequence and task type distribution probability output in Step S2, combined with the local system resource indicators collected in Step S1 and the service quality policy pre-stored by the edge computing nodes, a forward-looking scheduling decision is generated through a reinforcement learning-optimized dynamic scheduler, and the expected resource consumption and expected processing time corresponding to the forward-looking scheduling decision are generated simultaneously. The forward-looking scheduling decision includes: allocating high real-time voice tasks to the idle dedicated AI acceleration units of the edge computing nodes, allocating decomposable non-real-time subtasks to the lightest-loaded neighbor edge computing nodes in the neighbor edge computing node load status snapshot obtained in Step S1, and uploading only the feature data of voice tasks requiring deep semantic understanding to the cloud. Step S4: Based on the forward-looking scheduling decision generated in Step S3, the speech processing instance in the edge computing node is elastically scaled through the container management engine, and the actual resource consumption and processing time data after the task is completed are collected. The actual resource consumption and processing time data are compared with the prediction sequence output in Step S2 and the expected resource consumption and expected processing time generated in Step S3. The resulting difference data is used to fine-tune the spatiotemporal attention prediction model in Step S2 and the dynamic scheduler optimized by reinforcement learning in Step S3 online.

2. The real-time voice communication method for smart displays based on edge computing according to claim 1, characterized in that: In step S1, the process of periodically collecting local system resource indicators, local pending voice task queue status, and local voice task computing characteristics of the edge computing node, and obtaining a snapshot of the load status of at least one neighboring edge computing node from the edge network through the edge network's heartbeat protocol, specifically involves: The edge computing node performs data acquisition operations when each fixed monitoring cycle arrives, or when the local system resource utilization exceeds a dynamically adjusted threshold, or when triggered by a new voice task arrival event. Local system resource metrics include: the first utilization value, the second utilization value, and the third utilization value obtained by calling the performance interface provided by the operating system kernel and querying the hardware performance counter. The first utilization value, the second utilization value, and the third utilization value are arranged in order to form a local resource status triplet. The status of the local pending voice task queue includes: the total number of voice tasks currently waiting to be processed, obtained from the task scheduler; the average waiting time of each task calculated based on the timestamp of each task in the queue; and the statistically obtained first proportion of high-priority voice tasks to the total number of tasks in the queue, second proportion of medium-priority voice tasks to the total number of tasks in the queue, and third proportion of low-priority voice tasks to the total number of tasks in the queue. The total number of voice tasks, the average waiting time of each task, and the task priority ratio triplet composed of the first proportion, the second proportion, and the third proportion together constitute the status data representing the pressure on the task queue. The local speech task computation features include: real-time analysis of the input audio stream corresponding to the newly arrived speech task through a lightweight pre-analysis unit to estimate the estimated duration of the audio segment; evaluation of the estimated signal-to-noise ratio scalar value of the audio segment; and determination of the task type identifier to which the speech task belongs. The estimated audio duration, the estimated signal-to-noise ratio scalar value, and the task type identifier are sequentially combined into a task feature triplet. The load status snapshot of neighboring edge computing nodes includes: receiving status messages periodically broadcast by multiple neighboring edge computing nodes through a UDP multicast heartbeat protocol established with neighboring edge computing nodes; each status message contains at least one neighboring edge computing node's CPU utilization, memory utilization, and dedicated AI acceleration unit utilization; multiple sets of triplets consisting of CPU utilization, memory utilization, and dedicated AI acceleration unit utilization received from all neighboring edge computing nodes are grouped by neighboring edge computing nodes and arranged in a fixed order of CPU utilization, memory utilization, and dedicated AI acceleration unit utilization to form a set of neighboring edge computing node load statuses.

3. The real-time voice communication method for smart displays based on edge computing according to claim 2, characterized in that: The process of generating a standardized multidimensional feature vector is as follows: First, the first utilization rate value, the second utilization rate value, and the third utilization rate value contained in the local resource status triplet are subjected to dynamic weight standardization processing based on exponential decay to obtain the local resource standardized vector; the average waiting time of the task is subjected to dynamic weight standardization processing based on exponential decay to obtain the queue waiting time standardized value; the estimated audio duration and the estimated signal-to-noise ratio scalar value contained in the task feature triplet are subjected to dynamic weight standardization processing based on exponential decay to obtain the task duration standardized value and the task signal-to-noise ratio standardized value. Secondly, the first, second, and third proportions in the task priority ratio triplet, as well as the task type identifier, are vectorized using one-hot encoding to obtain the task priority encoding vector and the task type encoding vector, respectively. Next, the CPU utilization, memory utilization, and dedicated AI acceleration unit utilization of each neighbor edge computing node in the neighbor edge computing node load state set are linearly normalized, and the processed values ​​of all neighbor edge computing nodes are flattened and connected in node order to form a neighbor state flattening vector. Finally, the local resource normalization vector, queue waiting time normalization value, task duration normalization value, task signal-to-noise ratio normalization value, task priority encoding vector, task type encoding vector, and neighbor state flattening vector are concatenated into a normalized multidimensional feature vector in a predefined fixed order.

4. The real-time voice communication method for smart displays based on edge computing according to claim 3, characterized in that: In step S2, the specific process of synchronously analyzing multidimensional feature vectors by the spatiotemporal attention prediction model is as follows: First, based on the standardized multidimensional feature vectors generated in step S1 at multiple consecutive historical moments, temporal context codes and spatial context codes are extracted from them. The extraction of temporal context codes is completed by a gated recurrent unit network processing the standardized multidimensional feature vectors at multiple consecutive historical moments. The gated recurrent unit network outputs a fixed-dimensional vector as the temporal context code through its internal update gate and reset gate mechanism. The extraction of spatial context encoding is accomplished by processing the flattened neighbor state vector parsed from the normalized multidimensional feature vector using a graph attention network, specifically as follows: A1. Reshape the flattened neighbor state vector into a neighbor node feature matrix. The rows of this neighbor node feature matrix correspond to different neighbor edge computing nodes, and the columns correspond to three types of node resource indicators: CPU utilization, memory utilization, and utilization of dedicated AI acceleration units. A2. Use a learnable parameter weight matrix to perform a linear transformation on the node features represented by each row in the neighbor node feature matrix to obtain the transformed feature vector. A3. Calculate an attention coefficient for each row in the neighbor node feature matrix that represents the edge computing node itself and each of its neighbor edge computing nodes. A4. Based on all the calculated attention coefficients, perform a weighted summation on the transformed feature vectors of each neighboring edge computing node and the edge computing node itself, and use an activation function to process the weighted summation result. Finally, output a fixed-dimensional vector as a spatial context code.

5. The real-time voice communication method for smart displays based on edge computing according to claim 4, characterized in that: The process by which the spatiotemporal attention prediction model outputs the predicted sequence and the probability distribution of task type is as follows: First, the obtained temporal context encoding and spatial context encoding are adaptively fused to generate a joint context representation; Then, this joint context representation is input into a decoder network consisting of multiple fully connected layers; The decoder network outputs two results in parallel: The first output is a predicted sequence of system resource requirements within a set number of future time steps. The prediction result for each time step is represented as a three-dimensional real number vector. The first value in the three-dimensional real number vector represents the predicted value of the CPU utilization in the future time step, the second value represents the predicted value of the memory utilization in the future time step, and the third value represents the predicted value of the utilization rate of the dedicated AI acceleration unit in the future time step. The second output is a three-dimensional real vector. This three-dimensional real vector is processed by the softmax normalization function to obtain the prior probability distribution of the task type. The prior probability distribution of the task type is represented as a three-dimensional real vector. Its first element represents the probability that the newly arrived speech task within the future set time window belongs to the simple command word recognition type, the second element represents the probability that it belongs to the continuous speech dictation type, and the third element represents the probability that it belongs to the free dialogue comprehension type. The sum of these three probability values ​​is one.

6. The real-time voice communication method for smart displays based on edge computing according to claim 5, characterized in that: The adaptive fusion is accomplished through an adaptive gated fusion unit, which generates a joint context representation through operations: B1. Concatenate the temporal context code and the spatial context code to form a merged code vector; B2. Use a learnable weight matrix to perform a linear transformation on the merged encoded vector, and then add a learnable bias vector to obtain an intermediate vector. B3. Apply the sigmoid activation function to the intermediate vector to generate a gated vector; B4. Multiply the gating vector element-wise with the temporal context code to obtain the first product vector; at the same time, calculate the difference between the first and the gating vector to obtain the complementary gating vector, and multiply the complementary gating vector element-wise with the spatial context code to obtain the second product vector. B5. Add the first product vector to the second product vector and output the result as a joint context representation.

7. The real-time voice communication method for smart displays based on edge computing according to claim 6, characterized in that: In step S3, the process of generating forward-looking scheduling decisions through a reinforcement learning-optimized dynamic scheduler is specifically as follows: First, an enhanced state representation is constructed, which is composed of the local resource state triplet, all predicted values ​​in the prediction sequence, the prior probability distribution of the task type, and the flattened vector of the neighbor state in sequence. Secondly, the dynamic scheduler maintains a set of optional actions including local processing, neighbor cooperative processing, and cloud processing, and uses a two-branch evaluation network to evaluate each optional action. The first branch of the two-branch evaluation network is the value evaluation branch, which receives the enhanced state representation and a candidate action as input and outputs a value score representing the long-term expected benefit of performing the action. The second branch of the two-branch evaluation network is the constraint and cost evaluation branch, which calculates real-time evaluation indicators for different action types. For actions involving offloading to a neighboring edge computing node, the immediate evaluation metric is collaborative load balancing. For actions uploaded to the cloud, the real-time evaluation metric is cloud communication utility; Finally, the dynamic scheduler calculates a comprehensive decision score for each optional action based on the preset service quality policy. The comprehensive decision score is obtained by multiplying the first weight coefficient by the value score, adding the second weight coefficient by the corresponding real-time evaluation index, and then subtracting the third weight coefficient by the risk estimate of the action violating the service quality policy. The dynamic scheduler selects actions to generate forward-looking scheduling decisions based on the following logic: For voice tasks identified as high real-time tasks, it first determines whether the comprehensive decision score of the local processing action is higher than a first preset threshold and whether the utilization rate of the dedicated AI acceleration unit of the edge computing node is lower than a fourth preset threshold representing an idle state. If these conditions are met, the local processing action is selected. If not, it further determines whether the voice task is a decomposable non-real-time task. If so, it filters out actions with a collaborative load balancing index higher than a second preset threshold from all actions offloaded to neighboring edge computing nodes, and selects the action with the highest comprehensive decision score. The neighboring edge computing node corresponding to this action is considered the node with the lightest load. If there are still no actions that meet the conditions, and the prior probability distribution of the task type indicates that the current voice task has the highest probability of belonging to the free dialogue understanding type, i.e., it is determined to be a voice task requiring deep semantic understanding, it evaluates the comprehensive decision score of the action uploaded to the cloud. If its cloud communication utility index is higher than a third preset threshold, the action is selected.

8. The real-time voice communication method for smart displays based on edge computing according to claim 7, characterized in that: The process of synchronously generating and forward-looking scheduling decisions corresponding to expected resource consumption and expected processing time is as follows: For each specific speech task allocation scheme in the generated forward scheduling decision, the dynamic scheduler calculates the expected resource consumption and expected processing time in parallel based on the task type, allocation target and related prediction and status information of the speech task. The expected processing time is obtained by adding the baseline calculation time, network transmission time, and queuing time. The network transmission time is determined based on the allocation target: if the voice task is allocated to local processing, the network transmission time is zero; if it is offloaded to a neighboring edge computing node, the network transmission time is the estimated network link delay to that neighboring edge computing node; if it is uploaded to the cloud, the network transmission time is the round-trip network transmission delay to the cloud. The queuing time is calculated by querying the current voice task queue length of the target dedicated AI acceleration unit or cloud server, and combining it with the predicted value of the corresponding computing unit utilization in the prediction sequence for the target node output in step S2, through a function that outputs the queuing delay based on the queue length and the utilization prediction value. The calculation of expected resource consumption adopts different models depending on the allocation target: if the voice task is allocated to a dedicated AI acceleration unit of a local or neighboring edge computing node, the expected resource consumption is represented by the amount of computation of the dedicated AI acceleration unit expected to be required to complete the voice task; if the voice task is uploaded to the cloud, the expected resource consumption is represented by the network transmission resources expected to be occupied by uploading the data corresponding to the voice task used to generate the local voice task computation features in step S1.

9. A real-time voice communication method for smart displays based on edge computing according to claim 8, characterized in that: In step S4, the process of comparing the actual resource consumption and processing time data with the predicted sequence output in step S2 and the expected resource consumption and expected processing time generated in step S3 is as follows: First, the container management engine receives and parses the forward-looking scheduling decision generated in step S3. Based on the forward-looking scheduling decision generated in step S3, it creates or destroys container instances running the specified speech processing module within milliseconds by calling the container runtime interface, thus completing the elastic orchestration and execution of computing resources. Throughout the entire execution process of a speech task, from the time it is scheduled and assigned until its processing result is available, monitoring probes deployed inside the container instance and on the host machine continuously collect the actual resource consumption data and the actual end-to-end processing time data after the task is completed. Actual resource consumption data should include at least the CPU time actually used by the task, the peak memory usage, and the cumulative computing time of the dedicated AI acceleration unit. Secondly, each piece of actual resource consumption data and actual end-to-end processing time data collected is associated and bound with the unique identifier of the voice task that generated the data, the task type of the voice task, and the forward scheduling decision information in step S3 that triggers the execution of the voice task, to form an execution trajectory record containing task identity, decision basis and execution result. Next, the actual resource consumption data and actual end-to-end processing time data in the execution trajectory record are compared with the corresponding baseline data from steps S2 and S3 in a multi-dimensional comparison. The comparison operation generates two types of error data: the first type is the resource prediction error vector; the second type is the scheduling expectation error. Finally, for each set of error data generated by comparison, a data credibility weight is calculated. The calculation of the data credibility weight is based on the intensity of the unforeseen system disturbances encountered by the speech task corresponding to the set of error data during execution, and whether the task was completed normally. The error data associated with the data credibility weight, the historical decision state recorded in the corresponding execution trajectory record, and the scheduling action executed are integrated to form a data record set for subsequent online fine-tuning.

10. A real-time voice communication method for an intelligent display based on edge computing according to claim 9, characterized in that: The process of using the generated difference data for online fine-tuning of the spatiotemporal attention prediction model in step S2 and the dynamic scheduler optimized by reinforcement learning in step S3 is as follows: First, the online fine-tuning process is initiated by periodic triggering or triggering when the cumulative error exceeds a threshold; the data record set used for online fine-tuning is organized into batches to form data batches for model fine-tuning; Secondly, the spatiotemporal attention prediction model in step S2 is fine-tuned: taking the actual resource consumption data contained in each data record in the data batch as the true value, and the corresponding resource utilization prediction value output in step S2 as the model prediction value, a loss function is constructed. This loss function is equal to the sum of the squares of the numerical errors between the data confidence weights of all data records in the data batch and the corresponding predicted values ​​of the actual resource consumption data; the parameters of the spatiotemporal attention prediction model are updated using the mini-batch gradient descent algorithm combined with the elastic weight consolidation technique. Finally, the dynamic scheduler optimized by reinforcement learning in step S3 is fine-tuned: the temporal difference error is calculated by taking the historical decision state, the scheduling action executed, the actual immediate gain and subsequent state information contained in each data record in the data batch as input; the calculated temporal difference error is used to update the network parameters of the value evaluation branch in the dynamic scheduler through the gradient descent method, thereby calibrating its estimation of the long-term expected gain of different scheduling actions and making it closer to the feedback of the real environment.

Images