Data latency optimization control method based on ai inference engine
By constructing a semantic value assessment model and a latency budget allocation model, and allocating computing power based on the semantic contribution of data, the problem of high-value data being squeezed out by redundant data in AI inference engines is solved, and stable and accurate inference of high-value data and dynamic closed-loop optimization of the entire link are realized.
Patent Information
- Application Number
- CN202610248592.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-02
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2046-03-02
AI Technical Summary
Existing AI inference engine data latency optimization methods fail to effectively consider the differences in business value of the data to be inferred, resulting in high-value data being squeezed out by redundant data, latency exceeding the standard, and inability to adapt to the dynamic fluctuations of real-time data streams, making it difficult to achieve dynamic closed-loop optimization across the entire link.
By constructing a semantic value assessment model and a latency budget allocation model, computing power is allocated based on the semantic contribution of data, hierarchical inference execution channels are established, and an online incremental update model is combined to ensure that high-value data is processed first.
It achieves stable and accurate inference of high-value data, avoids the problem of excessive latency, and adapts to the fluctuations of real-time data streams, realizing dynamic closed-loop optimization across the entire link.
Smart Images

Figure CN122154920B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a data latency optimization and control method based on an AI inference engine. Background Technology
[0002] With the large-scale deployment of deep learning in real-time scenarios such as industrial quality inspection, financial risk control, and autonomous driving, the AI inference engine, as the core carrier for the execution of algorithm models, directly determines the response speed and service availability of business systems through its data processing latency. In online inference scenarios, the inference engine needs to continuously process high-concurrency real-time data streams. Latency control is affected by multiple factors such as model structure, hardware performance, data scheduling, and resource allocation, and is a core technical link to ensure that business service level agreements are met.
[0003] Traditional AI inference engine data latency optimization often focuses on model-level optimizations such as model compression, operator fusion, and quantization pruning, or execution-level optimizations such as static computing power scheduling and fixed queue priority configuration. These methods have certain drawbacks: First, the optimization logic only revolves around the inference execution process itself and does not consider the differences in business value of the data to be inferred. This can easily lead to high-value critical data being squeezed out by low-value redundant data, resulting in latency exceeding the standard. Second, the use of static fixed strategies cannot adapt to the dynamic fluctuations of real-time data streams, making it difficult to balance latency control accuracy and business effectiveness, and failing to achieve dynamic closed-loop optimization across the entire link. Summary of the Invention
[0004] In view of this, the present invention proposes a data latency optimization and control method based on an AI inference engine, which allocates computing power based on data priority to reduce the latency problem of high-value data.
[0005] The technical solution of this invention is implemented as follows: The data latency optimization and control method based on AI inference engine includes the following steps: Step S1: Obtain the historical business attribute tags, historical business processing data, and data stream to be inferred for the target inference business, and perform preprocessing. Step S2: Construct a semantic value assessment model based on historical business attribute tags. The semantic value assessment model processes the data stream to be reasoned and generates the semantic contribution of each piece of data to be reasoned. Step S3: Allocate inference latency budget and computing power channel access permissions to each piece of data to be inferred using a latency budget allocation model trained from historical business processing data, with semantic contribution as the constraint; Step S4: Based on semantic contribution, the allocated inference latency budget and computing power channel access permissions, classify all data to be inferred and map them to the corresponding inference execution channels for inference. Step S5: Collect business feedback data corresponding to the inference results. Based on the business feedback data and the measured inference latency data, perform online incremental updates to the semantic value assessment model and the latency budget allocation model.
[0006] Preferably, step S1 includes the following steps: Step S11: From the business system of the target inference business, retrieve the business criticality label, inference latency compliance requirement label and business result label for a preset period as historical business attribute labels; Step S12: Extract computing load data, latency data, and business metrics for a preset period from the business system of the target inference business as historical business processing data; Step S13: Real-time access to the data stream to be inferred, and perform format unification, frame alignment and noise filtering on the data stream to be inferred; Step S14: Perform deduplication, format normalization, and invalid tag removal on historical business attribute tags; perform time-series alignment, missing value completion, and outlier filtering on historical business processing data.
[0007] Preferably, step S2 includes the following specific steps: Step S21: Construct a semantic feature extraction backbone network based on the contrastive learning framework, and complete the pre-training of the semantic feature extraction backbone network using historical business attribute labels as supervision signals; Step S22: Superimpose a regression prediction head on the output of the semantic feature extraction backbone network to construct a semantic value assessment model. Use historical business attribute labels as supervision signals to complete the training and verification of the semantic value assessment model. Step S23: The real-time collected data stream to be reasoned is sent to the semantic value evaluation model, and the semantic feature extraction backbone network extracts semantic features to obtain a high-dimensional semantic feature vector for each piece of data to be reasoned. Step S24: Perform information gain quantization calculation on the high-dimensional semantic feature vector based on the regression prediction head to obtain the semantic contribution of each piece of data to be inferred.
[0008] Preferably, the specific steps for the information gain quantization calculation are as follows: The high-dimensional semantic feature vector corresponding to the data to be reasoned is input into the regression prediction head. The high-dimensional semantic feature vector is weighted and mapped by the pre-trained and solidified business feature weight matrix that is fitted based on historical business attribute labels, and key semantic features that are strongly related to the core decision of the target reasoning business are extracted. The key semantic features are matched and compared with the business value benchmark library constructed by historical business attribute tags. The marginal information gain value of the data to be reasoned for the target reasoning business decision is calculated. The marginal information gain value is then normalized by min-max to generate the semantic contribution.
[0009] Preferably, step S3 includes the following specific steps: Step S31: Construct a reinforcement learning simulation environment based on historical business processing data, and construct an initial delay budget allocation model based on a deep reinforcement learning framework; Step S32: Train and converge the initial time delay budget allocation model in a reinforcement learning simulation environment to obtain the time delay budget allocation model; Step S33: Collect real-time computing power operation data of the AI inference engine, and construct the state space by combining the semantic contribution of each piece of data to be inferred; Step S34: Input the state space into the time delay budget allocation model, and the time delay budget allocation model outputs the inference time delay budget and computing power channel access permissions corresponding to each piece of data to be inferred.
[0010] Preferably, the reward function expression of the time-delay budget allocation model is:
[0011] Where R is the total reward value, α, β, and γ are preset weight coefficients, and R1, R2, and R3 are the latency compliance rate of high-priority data to be inferred, the overall computing power utilization rate of the AI inference engine, and the number of priority reversals of data to be inferred in the same batch, respectively.
[0012] Preferably, step S4 includes the following specific steps: Step S41: Calculate the comprehensive priority score based on semantic contribution, allocated inference latency budget, and computing power channel access permissions; Step S42: Compare the overall priority score with the preset threshold, and divide the data to be reasoned into high-priority data to be reasoned, medium-priority data to be reasoned, and low-priority data to be reasoned. Step S43: Map dedicated computing power resource execution channels, shared computing power resource execution channels, and idle computing power resource execution channels to high-priority data to be inferred, medium-priority data to be inferred, and low-priority data to be inferred, respectively.
[0013] Preferably, the specific steps of step S41 are as follows: reverse normalize the inference latency budget, quantify the computing power channel access permission, and sum the semantic contribution, the reverse normalized inference latency budget, and the quantified computing power channel access permission after assigning different weights to them to obtain a comprehensive priority score.
[0014] Preferably, step S5 includes the following specific steps: Step S51: Collect the inference results of the execution channel, the measured inference latency data of each piece of data to be inferred, and the business feedback data corresponding to the inference results returned by the target business system; Step S52: Perform format alignment, outlier filtering, and time-series matching on the inference results, measured inference latency data, and business feedback data to obtain an incremental training dataset. Step S53: Based on the incremental training dataset, and using business feedback data as a supervision signal, incrementally fine-tune the regression prediction head parameters of the semantic value assessment model. Step S54: Based on the incremental training dataset, and using the measured inference latency data and business feedback data as joint supervision signals, incrementally update the policy network parameters and preset weights of the reward function of the latency budget allocation model.
[0015] Compared with the prior art, the beneficial effects of the present invention are: The data latency optimization and control method based on the AI inference engine of this invention differs from traditional optimization methods that only focus on technical indicators and are detached from business objectives. By acquiring historical business attribute tags and historical business processing data, a semantic value assessment model and a latency budget allocation model can be constructed respectively. The semantic value assessment model can score the semantic contribution of the real-time collected data to be inferred. The semantic contribution can be used by the latency budget allocation model as a constraint to solve the inference latency budget and computing power channel access permissions of the data to be inferred. Then, based on the semantic contribution, inference latency budget, and computing power channel access permissions, the data to be inferred can be classified. Based on different priorities, corresponding execution channels are allocated to the data to be processed. The higher the priority, the more computing power resources are obtained, avoiding the problem of critical data being squeezed out by redundant data, resulting in latency exceeding the standard. Through priority calculation, it can be ensured that high-value data to be inferred can be inferred stably and accurately, ensuring the accuracy of the inference results. In addition, the semantic value assessment model and the latency budget allocation model can be updated online incrementally according to the inference results, realizing dynamic closed-loop optimization of the entire link, which can adapt to the fluctuations of real-time data streams. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only preferred embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is a flowchart of the data latency optimization control method based on an AI inference engine according to the present invention; Figure 2 This is a flowchart of step S1 of the data latency optimization control method based on an AI inference engine of the present invention; Figure 3This is a flowchart of step S2 of the data latency optimization control method based on an AI inference engine of the present invention; Figure 4 This is a flowchart of step S3 of the data latency optimization control method based on an AI inference engine of the present invention; Figure 5 This is a flowchart of step S4 of the data latency optimization control method based on an AI inference engine of the present invention; Figure 6 This is a flowchart of step S5 of the data latency optimization control method based on the AI inference engine of the present invention. Detailed Implementation
[0018] To better understand the technical content of this invention, a specific embodiment is provided below, and the invention will be further described in conjunction with the accompanying drawings.
[0019] See Figures 1 to 6 The data latency optimization and control method based on an AI inference engine provided by this invention includes the following steps: Step S1: Obtain the historical business attribute tags, historical business processing data, and the data stream to be inferred for the target inference business, and perform preprocessing. Specific steps include: Step S11: From the business system of the target inference business, retrieve the business criticality label, inference latency compliance requirement label and business result label for a preset period as historical business attribute labels; Step S12: Extract computing load data, latency data, and business metrics for a preset period from the business system of the target inference business as historical business processing data; Step S13: Real-time access to the data stream to be inferred, and perform format unification, frame alignment and noise filtering on the data stream to be inferred; Step S14: Perform deduplication, format normalization, and invalid tag removal on historical business attribute tags; perform time-series alignment, missing value completion, and outlier filtering on historical business processing data.
[0020] The data latency optimization and control method based on the AI inference engine of this invention can be applied to online AI inference business scenarios such as industrial visual quality inspection, financial real-time risk control, autonomous driving environmental perception, and real-time content recommendation. It assesses contribution and allocates execution channels through a constructed semantic value evaluation model and a latency budget allocation model, respectively. To ensure the accuracy of the processing results of the semantic value evaluation model and the latency budget allocation model, relevant training data needs to be collected to train the models. The semantic value evaluation model is trained using historical business attribute tags as the core supervision signal, while the latency budget allocation model is trained based on historical business processing data and combined with the semantic value priority conservation rule. After extracting historical business attribute tags and historical business processing data within a preset historical period from the business system of the target inference business, corresponding preprocessing processes are performed. Invalid tags with no corresponding business results or missing tags, as well as invalid running data with abnormal computing power peaks or latency exceeding limits, are filtered out. This provides full-dimensional supervision signals for the subsequent training of the two models, fundamentally avoiding the problem of optimization strategies being out of sync with business needs.
[0021] Taking the visual quality inspection scenario of new energy battery cells as an example, the business criticality label includes the level divided according to the impact of defects on battery safety, the inference latency compliance requirement label is the inference latency set according to the different levels, which serves as the core constraint boundary for subsequent latency budget allocation, and the business result label is the true label confirmed by manual re-inspection.
[0022] The computing load data includes historical real-time computing power utilization rate of the inference cluster, memory usage rate, computing power quota usage data of each inference channel, peak computing power data when the production line is running at full speed, and time-series records of computing power congestion periods. The latency measurement data includes the latency of the entire inference process, the latency of a single operator execution, the distribution of inference latency for images of different defect types, records of latency exceeding the standard, and details of latency exceeding the standard for high-value data caused by priority inversion. The business indicator data includes historical quality inspection defect detection rate, misjudgment rate, missed detection rate, number of production line shutdowns, customer complaint records corresponding to missed defects, and production line cycle time compliance rate.
[0023] The data stream to be inferred is a continuous online image data stream collected in real time from the production line. A single piece of data to be inferred is a single frame / multiple frame quality inspection image corresponding to a single cell in the data stream.
[0024] Step S2: Construct a semantic value assessment model based on historical business attribute tags. The semantic value assessment model processes the data stream to be inferred and generates the semantic contribution value corresponding to each piece of data to be inferred. The specific steps include: Step S21: Construct a semantic feature extraction backbone network based on the contrastive learning framework, and complete the pre-training of the semantic feature extraction backbone network using historical business attribute labels as supervision signals; Step S22: Superimpose a regression prediction head on the output of the semantic feature extraction backbone network to construct a semantic value assessment model. Use historical business attribute labels as supervision signals to complete the training and verification of the semantic value assessment model. Step S23: The real-time collected data stream to be reasoned is sent to the semantic value evaluation model, and the semantic feature extraction backbone network extracts semantic features to obtain a high-dimensional semantic feature vector for each piece of data to be reasoned. Step S24: Perform information gain quantization calculation on the high-dimensional semantic feature vector based on the regression prediction head to obtain the semantic contribution of each piece of data to be inferred.
[0025] Preferably, the specific steps for the information gain quantization calculation are as follows: The high-dimensional semantic feature vector corresponding to the data to be reasoned is input into the regression prediction head. The high-dimensional semantic feature vector is weighted and mapped by the pre-trained and solidified business feature weight matrix that is fitted based on historical business attribute labels, and key semantic features that are strongly related to the core decision of the target reasoning business are extracted. The key semantic features are matched and compared with the business value benchmark library constructed by historical business attribute tags. The marginal information gain value of the data to be reasoned for the target reasoning business decision is calculated. The marginal information gain value is then normalized by min-max to generate the semantic contribution.
[0026] Existing AI inference latency optimization solutions are mostly model-centric, focusing only on optimizing computational power during the inference execution process, without considering the differences in business value of the data to be inferred. They apply the same inference process and latency requirements to all data, resulting in high-value critical data being squeezed out by low-value redundant data, leading to overall latency issues. To address this, this invention trains a semantic value assessment model after acquiring historical business attribute tags, and evaluates the semantic contribution of each piece of data to be inferred based on this model, determining the business value of different data. The semantic value assessment model consists of a semantic feature extraction backbone network and a regression prediction head. Using historical business attribute tags, the semantic feature backbone network and the semantic feature extraction backbone network superimposed with the regression prediction head are trained sequentially to form the semantic value assessment model. After training, real-time collected data to be inferred is input into the semantic value assessment model. First, features are extracted through the semantic feature extraction backbone network, and then information gain is quantified through the regression prediction head to obtain the semantic contribution of each piece of data to be inferred.
[0027] When performing information gain quantization calculation, the semantic contribution is calculated by extracting key semantic features and matching and comparing to calculate the marginal information gain value. Finally, a semantic contribution value with a range of [0,1] can be generated, which provides a basis for subsequent latency budget allocation, access permission division of computing power channels, and hierarchical scheduling.
[0028] Step S3: Using a latency budget allocation model trained from historical business processing data, allocate inference latency budget and computing power channel access permissions to each piece of data to be inferred based on semantic contribution. Specific steps include: Step S31: Construct a reinforcement learning simulation environment based on historical business processing data, and construct an initial delay budget allocation model based on a deep reinforcement learning framework; Step S32: Train and converge the initial time delay budget allocation model in a reinforcement learning simulation environment to obtain the time delay budget allocation model; Step S33: Collect real-time computing power operation data of the AI inference engine, and construct the state space by combining the semantic contribution of each piece of data to be inferred; Step S34: Input the state space into the time delay budget allocation model, and the time delay budget allocation model outputs the inference time delay budget and computing power channel access permissions corresponding to each piece of data to be inferred.
[0029] Preferably, the reward function expression of the time-delay budget allocation model is:
[0030] Where R is the total reward value, α, β, and γ are preset weight coefficients, R1, R2, and R3 are the latency compliance rate of high-priority data to be inferred, the overall computing power utilization rate of the AI inference engine, and the number of priority reversals of data to be inferred in the same batch, respectively. R3 is a penalty item, which deducts the corresponding reward when a priority reversal occurs.
[0031] The latency budget allocation model is built on a deep reinforcement learning framework. It uses historical business processing data to construct an initial state space and a reinforcement learning simulation environment. Based on pre-defined semantic value priority conservation rules, action boundary constraints are constructed to obtain the initial latency budget allocation model. These rules include a negative correlation between semantic contribution and latency tolerance threshold, consistency between the inference timeline of high-priority data and semantic contribution ranking, and prohibition of encroaching on high-priority data resources during computing power congestion. The model is then pre-trained and converged in the simulation environment to obtain the final latency budget allocation model. In the reward function of the latency budget allocation model, pre-defined weight coefficients α > β > γ are set to ensure that the latency compliance rate of high-priority data is the core optimization objective. Real-time computing power operation data is combined with the semantic contribution of the data to be inferred to construct a state space, which is then input into the latency budget allocation model. The model outputs the final inference latency budget and computing power channel access permissions, enabling the allocation of dedicated resources and latency budgets to each piece of data to be inferred based on data business value and real-time computing power status.
[0032] Step S4: Based on semantic contribution, the allocated inference latency budget, and computing power channel access permissions, all data to be inferred is classified and mapped to the corresponding inference execution channel for inference. Specific steps include: Step S41: Calculate the comprehensive priority score based on semantic contribution, the allocated inference latency budget, and the computing power channel access permission. The specific steps are as follows: reverse normalize the inference latency budget and quantify the computing power channel access permission. Then, assign different weights to the semantic contribution, the reverse normalized inference latency budget, and the quantified computing power channel access permission, and sum them to obtain the comprehensive priority score. Step S42: Compare the overall priority score with the preset threshold, and divide the data to be reasoned into high-priority data to be reasoned, medium-priority data to be reasoned, and low-priority data to be reasoned. Step S43: Map dedicated computing power resource execution channels, shared computing power resource execution channels, and idle computing power resource execution channels to high-priority data to be inferred, medium-priority data to be inferred, and low-priority data to be inferred, respectively.
[0033] The inference latency budget output by the latency budget allocation model is received. Based on the maximum allowable inference latency per data item preset for the target inference service, the inference latency budget is reverse normalized. Simultaneously, the access permissions for computing power channels are quantified, unifying the data across the three dimensions of semantic contribution, inference latency budget, and computing power channel access permissions to the same value. Then, different weights are assigned to each dimension, and the results are summed to obtain a comprehensive priority score. This comprehensive priority score is compared with preset thresholds A and B, where threshold A is greater than threshold B. When the comprehensive priority score is greater than threshold A, the data to be inferred is classified as high-priority data; when the comprehensive priority score is greater than threshold B, the data to be inferred is classified as low-priority data. At threshold A, data is classified as medium-priority data to be inferred. When the comprehensive priority score is less than threshold B, it is classified as low-priority data to be inferred. Different execution channels are allocated to data of different priorities. High-priority data is allocated a dedicated computing power resource execution channel, which locks the corresponding computing power resources to prevent them from being occupied by other data and prioritizes the inference calculation. Medium-priority data is allocated a shared computing power resource execution channel, which performs inference through shared computing power. Low-priority data is allocated an idle computing power resource execution channel, which performs inference when computing power resources are idle. This ensures that high-priority data has dedicated computing power and priority inference rights, and guarantees that the latency of core business data meets the standard by 100%.
[0034] Step S5: Collect business feedback data corresponding to the inference results. Based on the business feedback data and the measured inference latency data, perform online incremental updates to the semantic value assessment model and the latency budget allocation model. Specific steps include: Step S51: Collect the inference results of the execution channel, the measured inference latency data of each piece of data to be inferred, and the business feedback data corresponding to the inference results returned by the target business system; Step S52: Perform format alignment, outlier filtering, and time-series matching on the inference results, measured inference latency data, and business feedback data to obtain an incremental training dataset. Step S53: Based on the incremental training dataset, and using business feedback data as a supervision signal, incrementally fine-tune the regression prediction head parameters of the semantic value assessment model. Step S54: Based on the incremental training dataset, and using the measured inference latency data and business feedback data as joint supervision signals, incrementally update the policy network parameters and preset weights of the reward function of the latency budget allocation model.
[0035] Existing latency optimization strategies are mostly offline training and static deployment. After deployment, they are not dynamically updated based on business feedback and actual latency measurement data. They cannot adapt to dynamic situations such as changes in business scenarios, data flow distribution drift, and fluctuations in computing power environment. As the running time increases, the optimization effect will continue to decline, and long-term stable latency control cannot be achieved. In contrast, after the inference is completed, this invention collects the inference results, actual inference latency measurement data, and business feedback data corresponding to the inference results of each execution channel. After preprocessing, an incremental training dataset is obtained. The incremental training dataset can be fed back to the semantic value evaluation model and the latency budget allocation model for incremental fine-tuning and updating. This updates the calculation logic of semantic contribution, adapts to changes in business value judgment standards, optimizes budget allocation accuracy, and adapts to changes in computing power environment and data flow. It establishes a closed-loop latency optimization system, ensuring the long-term adaptability and stability of the optimization strategy.
[0036] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A data latency optimization and control method based on an AI inference engine, characterized in that, Includes the following steps: Step S1: Obtain the historical business attribute tags, historical business processing data, and data stream to be inferred for the target inference business, and perform preprocessing. Step S2: Construct a semantic value assessment model based on historical business attribute tags. The semantic value assessment model processes the data stream to be reasoned and generates the semantic contribution of each piece of data to be reasoned. Step S3: Allocate inference latency budget and computing power channel access permissions to each piece of data to be inferred using a latency budget allocation model trained from historical business processing data, with semantic contribution as the constraint; Step S4: Based on semantic contribution, the allocated inference latency budget and computing power channel access permissions, classify all data to be inferred and map them to the corresponding inference execution channels for inference. Step S5: Collect business feedback data corresponding to the inference results. Based on the business feedback data and the measured inference latency data, perform online incremental updates to the semantic value assessment model and the latency budget allocation model. The specific steps of step S2 include: Step S21: Construct a semantic feature extraction backbone network based on the contrastive learning framework, and complete the pre-training of the semantic feature extraction backbone network using historical business attribute labels as supervision signals; Step S22: Superimpose a regression prediction head on the output of the semantic feature extraction backbone network to construct a semantic value assessment model. Use historical business attribute labels as supervision signals to complete the training and verification of the semantic value assessment model. Step S23: The real-time collected data stream to be reasoned is sent to the semantic value evaluation model, and the semantic feature extraction backbone network extracts semantic features to obtain a high-dimensional semantic feature vector for each piece of data to be reasoned. Step S24: Perform information gain quantization calculation on the high-dimensional semantic feature vector based on the regression prediction head to obtain the semantic contribution of each piece of data to be inferred; The specific steps of step S3 include: Step S31: Construct a reinforcement learning simulation environment based on historical business processing data, and construct an initial delay budget allocation model based on a deep reinforcement learning framework; Step S32: Train and converge the initial time delay budget allocation model in a reinforcement learning simulation environment to obtain the time delay budget allocation model; Step S33: Collect real-time computing power operation data of the AI inference engine, and construct the state space by combining the semantic contribution of each piece of data to be inferred; Step S34: Input the state space into the time delay budget allocation model, and the time delay budget allocation model outputs the inference time delay budget and computing power channel access permissions corresponding to each piece of data to be inferred.
2. The data latency optimization and control method based on an AI inference engine according to claim 1, characterized in that, The specific steps of step S1 include: Step S11: From the business system of the target inference business, retrieve the business criticality label, inference latency compliance requirement label and business result label for a preset period as historical business attribute labels; Step S12: Extract computing load data, latency data, and business metrics for a preset period from the business system of the target inference business as historical business processing data; Step S13: Real-time access to the data stream to be inferred, and perform format unification, frame alignment and noise filtering on the data stream to be inferred; Step S14: Perform deduplication, format normalization, and invalid tag removal on historical business attribute tags; perform time-series alignment, missing value completion, and outlier filtering on historical business processing data.
3. The data latency optimization and control method based on an AI inference engine according to claim 1, characterized in that, The specific steps for the information gain quantization calculation are as follows: The high-dimensional semantic feature vector corresponding to the data to be reasoned is input into the regression prediction head. The high-dimensional semantic feature vector is weighted and mapped by the pre-trained and solidified business feature weight matrix that is fitted based on historical business attribute labels, and key semantic features that are strongly related to the core decision of the target reasoning business are extracted. The key semantic features are matched and compared with the business value benchmark library constructed by historical business attribute tags. The marginal information gain value of the data to be reasoned for the target reasoning business decision is calculated. The marginal information gain value is then normalized by min-max to generate the semantic contribution.
4. The data latency optimization and control method based on an AI inference engine according to claim 1, characterized in that, The reward function expression for the time-delay budget allocation model is: Where R is the total reward value. For preset weighting coefficients, These are the latency compliance rate of high-priority data to be inferred, the overall computing power utilization rate of the AI inference engine, and the number of times the priority of data to be inferred in the same batch is reversed.
5. The data latency optimization and control method based on an AI inference engine according to claim 1, characterized in that, The specific steps of step S4 include: Step S41: Calculate the comprehensive priority score based on semantic contribution, allocated inference latency budget, and computing power channel access permissions; Step S42: Compare the overall priority score with the preset threshold, and divide the data to be reasoned into high-priority data to be reasoned, medium-priority data to be reasoned, and low-priority data to be reasoned. Step S43: Map dedicated computing power resource execution channels, shared computing power resource execution channels, and idle computing power resource execution channels to high-priority data to be inferred, medium-priority data to be inferred, and low-priority data to be inferred, respectively.
6. The data latency optimization and control method based on an AI inference engine according to claim 5, characterized in that, The specific steps of step S41 are as follows: reverse normalize the inference latency budget, quantify the computing power channel access permissions, and sum the semantic contribution, the reverse normalized inference latency budget, and the quantified computing power channel access permissions to obtain a comprehensive priority score.
7. The data latency optimization and control method based on an AI inference engine according to claim 1, characterized in that, The specific steps of step S5 include: Step S51: Collect the inference results of the execution channel, the measured inference latency data of each piece of data to be inferred, and the business feedback data corresponding to the inference results returned by the target business system; Step S52: Perform format alignment, outlier filtering, and time-series matching on the inference results, measured inference latency data, and business feedback data to obtain an incremental training dataset. Step S53: Based on the incremental training dataset, and using business feedback data as a supervision signal, incrementally fine-tune the regression prediction head parameters of the semantic value assessment model. Step S54: Based on the incremental training dataset, and using the measured inference latency data and business feedback data as joint supervision signals, incrementally update the policy network parameters and preset weights of the reward function of the latency budget allocation model.
Citation Information
Patent Citations
Explanatable training method of unconditional image generation diffusion probability model
CN119600144A
Deep exploration AI reasoning method and system
CN120387515A
