An industrial large model layered dynamic inference acceleration method, device and equipment

Abstract

The application discloses an industrial large model layered dynamic reasoning acceleration method, device and equipment. The method forms a verifiable industrial context with an industrial equipment state version, an industrial knowledge version, process constraints and safety risks, and selects a reasoning level corresponding to a calculation budget in a cache reuse layer, a lightweight reasoning layer, a standard reasoning layer and a high-reliability reasoning layer; cache results are reused only when the context version is consistent and passes the industrial constraint verification. In the reasoning process, the output uncertainty, search evidence consistency, constraint violation state and time limit margin are input into a hierarchical state machine with a hysteresis interval according to a decoding window, hierarchical switching and shared reasoning state migration are performed; after switching, the industrial constraints are checked again, and if the check fails or the control instruction output condition is not met, the control instruction is blocked and safety rollback or manual confirmation is performed. The application can reduce the reasoning calculation amount, time delay and storage occupation under the condition of meeting the industrial safety and time limit constraints.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence reasoning, industrial internet and computer resource scheduling, and in particular to a method, apparatus and equipment for accelerating hierarchical dynamic reasoning of large industrial models. Background Technology

[0002] Large-scale industrial models can be used for tasks such as equipment fault diagnosis, process parameter analysis, handling production anomalies, and safety risk identification. Different tasks vary significantly in terms of complexity, time constraints, safety risks, and resource consumption.

[0003] Existing inference services typically employ fixed model depth, quantization precision, and decoding strategies: simple tasks generate unnecessary computations, while complex or high-risk tasks may suffer from reduced reliability due to insufficient computation.

[0004] Acceleration methods such as existing quantization, pruning, continuous batch processing, key-value caching, early exit, and speculative decoding do not use industrial state, risk, process constraints, control time limits, and resource state together for the dynamic determination of the inference level; level switching may also bring state reconstruction overhead, and industrial rule verification is usually separated from acceleration control. Summary of the Invention

[0005] Technical problem to be solved: The purpose of this invention is to provide a method, apparatus and equipment for accelerating hierarchical dynamic inference of large industrial models, so as to solve the problems in the prior art that cause waste of computing resources due to fixed model execution strategy, static hierarchical selection cannot adapt to changes in the inference process, model switching requires repeated calculation, request scheduling does not take into account industrial risks and deadlines, and inference acceleration and industrial rule verification are separated.

[0006] The purpose of this invention is to provide a method, apparatus, and equipment for accelerating hierarchical dynamic inference of large industrial models, in order to solve the problems in the prior art such as the waste of computing resources caused by fixed model execution strategies, the inability of static hierarchical selection to adapt to changes in the inference process, the need for repeated calculations when switching models, the inability of general cache reuse to reflect changes in industrial state versions, the failure of request scheduling to simultaneously consider industrial risks and deadlines, and the separation of inference acceleration and industrial rule verification.

[0007] To achieve the above objectives, this invention provides a method for accelerating hierarchical dynamic inference in industrial large-scale models, executed by an industrial large-scale model inference service system, comprising the following steps: S101, Obtain Industrial Inference Request Obtain the industrial inference request and the associated industrial context information and computing resource status. The industrial inference request includes at least the input data, request type, target output type, and maximum allowed response time. The industrial context information includes at least one or more of the following: equipment identifier, equipment operating status, process stage, alarm level, data sensitivity level, security risk level, and industrial knowledge version. The computing resource status includes at least one or more of the following: graphics processor utilization, graphics processor memory utilization, inference request queue length, edge node load, network round-trip latency, and available cache capacity. S102, Construct the request feature vector The industrial reasoning request is segmented, data type identified, modality identified, entity identified, equipment associated, and industrial knowledge retrieved to construct a request feature vector: in: This represents the input length and predicted output length features; Indicates task type characteristics; Represents input modal features; This represents the characteristics of knowledge retrieval hit rate and knowledge novelty. Indicates the characteristics of equipment operating status; This indicates security risks and data sensitivity. Indicates the characteristics of the deadline; Represents computing resources and queuing status characteristics; This indicates the characteristics of historical cache hits. S103, Calculate reasoning and decision parameters Calculate the task complexity value based on the request feature vector. Industrial risk value Resource pressure value and deadline margin . In one implementation, the task complexity value is calculated according to the following formula: in, For normalization function, These are complexity-related subvectors extracted from the request feature vector. For complexity weights, This is a complexity bias. The industrial risk value is calculated using the following formula: in, The risk value is determined according to industrial safety regulations. This is a risk-related subvector. Resource pressure values ​​are calculated using the following formula: in: To calculate unit utilization; For memory utilization; Normalized queue length; To normalize network latency; Cache utilization; to The weights are denoted by 1, and the sum of the weights is 1. The deadline margin is calculated using the following formula: in, To the maximum allowable response time, The time already consumed. To predict the remaining inference time. S104. Determine the initial reasoning level The initial inference level is determined from the cache reuse layer, lightweight inference layer, standard inference layer, and high-reliability inference layer. For each candidate inference level, the executable conditions of the level are established by combining the device status version, industry knowledge version, process stage, security risk level, industry constraint verification conditions, and deadline margin. The cache reuse layer is denoted as L0, which is used to reuse historical inference results or historical prefix states for non-state-changing requests. Before reuse, it is necessary to verify that the device state version, industry knowledge version, validity period and industry constraint verification status of the cache item meet the preset conditions. The lightweight inference layer, denoted as L1, is used to call lightweight industrial models, or to call a portion of the shared model layer and early exit header of a large industrial model, to complete low-risk, uncontrolled inference with low-level quantization precision. The standard inference layer is denoted as L2. It is used to call the standard model depth of the industrial large model and uses speculative decoding, mixed precision calculation and continuous batch processing to complete the inference that requires conventional industrial constraint verification. The high-reliability inference layer is designated L3. It is used to call the complete industrial model, execute all model layers, enhance knowledge retrieval, perform at least one result verification, and verify industrial constraints. Results involving safety interlocks, state changes, or control commands will not be issued until the control command output conditions are met. Calculate the cost function for each candidate inference level: in: hierarchical The predicted latency; To predict energy consumption; To predict memory usage; To predict computational costs; Penalties are set for violations of accuracy constraints, risk constraints, or deadline constraints. The inference level that satisfies the risk constraint, accuracy constraint, deadline constraint, and hierarchical executability condition and has the minimum cost function is determined as the initial inference level. When the industrial risk value is higher than the first risk threshold, the cache reuse layer is prohibited from being selected; when the industrial risk value is higher than the second risk threshold, the initial inference layer must not be lower than the standard inference layer; when the industrial risk value is higher than the third risk threshold, the initial inference layer is determined to be the high-reliability inference layer. S105, Configuration Hierarchy Calculation Budget The budget is calculated based on the initial inference level configuration, and the calculated budget includes at least two of the following parameters: 1. Number of model layers; 2. Weighted quantization bit width; 3. Activation value quantization bit width; 4. Draft model type; 5. Length of the speculative mark; 6. Maximum number of output tags; 7. Number of documents retrieved; 8. Number of validation models; 9. Number of result verifications; 10. Batch processing category; 11. Inference execution node; 12. Key-value cache capacity. For lightweight inference layers, the number of model layers is set to the first model depth. For the standard inference layer, the number of model layers is set to the second model depth. For high-reliability inference layers, the number of model layers is set to the full model depth. ,in: S106, Perform hierarchical reasoning Based on the determined inference level and computational budget, industrial inference requests are sent to the corresponding inference execution engine. The inference execution engine includes a shared tokenizer, a shared industrial entity encoder, one or more shared prefix model layers, multiple early exit headers, a standard inference model, a validation model, and an industrial rule validator. For inference levels employing speculative decoding, a draft model is used to generate candidate tag sequences, and the target model is used to verify the candidate tag sequences in parallel. Speculative mark length The acceptance rate of candidate tags is dynamically determined based on the latest preset number of decoding windows: in, For candidate marker acceptance rate, This represents the task complexity value. This represents the resource pressure value. S107. Monitor decoding status and calculate level adjustment score. After generating each decoding window, intermediate inference states are collected and input into a hierarchical state machine with upgrade and downgrade thresholds. The decoding window includes one or more consecutive output markers, and the intermediate inference states include at least: Output probability entropy ; The probability interval between the first candidate label and the second candidate label ; Speculative Mark Acceptance Rate ; Consistency of search evidence ; Industrial constraint violation status ; Current industrial risk value ; Current deadline remaining time . Calculate the level upgrade score based on the intermediate reasoning state: in, to The weight is the score. When the level upgrade score is greater than the upgrade threshold, or when the industrial constraint violation status indicates the existence of a high-risk violation, the current inference level will be upgraded by at least one level; when a high-risk violation exists, the downgrade judgment made on the current decoding window will be ignored. When the level upgrade score is lower than the downgrade threshold for multiple consecutive decoding windows, the industrial risk value is lower than the downgrade risk threshold, the deadline margin is greater than the margin threshold, and the currently generated content does not contain the control instructions to be verified, the current inference level is reduced by one level at the semantic segment boundary. The upgrade threshold is greater than the downgrade threshold to form a hierarchical state machine with a hysteresis interval, thus avoiding frequent switching of inference levels. S108, Transfer Reasoning State When the reasoning level changes, the current reasoning state is migrated. The current reasoning state includes the input tag sequence, the generated tag sequence, the retrieved evidence index, the industrial entity state, the shared model layer key-value cache, and the constraint verification state. When the word segmenter version and model embedding layer version of the inference model before and after the upgrade are the same and share at least some model layers, the key-value cache and constraint verification state corresponding to the shared model layer are directly reused, and only the key-value state corresponding to the newly added model layer is calculated. When the inference models before and after the upgrade do not share the key-value cache data structure, the input tag sequence, the generated tag sequence, the retrieved evidence index, and the compressed semantic state are sent to the target inference model, which then reconstructs the unshared key-value cache. The compressed semantic state includes one or more of the following: device entity vector, process stage vector, alarm relationship vector, retrieval evidence vector, and generated text summary vector. S109, Perform hierarchical cache management The inference cache is divided into hot cache, warm cache, and cold cache. The hot cache is located in the inference accelerator memory and is used to store key-value cache blocks corresponding to the current execution request, recent high-frequency industrial prefixes, and high-priority requests. The warm cache is located in main memory and is used to store temporarily suspended requests, low-frequency industrial prefixes, and key-value cache blocks that can be resumed within a predetermined period of time. Cold caches are set up in non-volatile storage media or distributed caches to store historical results, historical retrieval evidence, and reusable industrial prompt templates. The cache identifier includes at least the model version, word segmenter version, device identifier, device status version, industry knowledge version, hint prefix hash value, and industry constraint verification status. When the device status version or industry knowledge version changes, the cached items of historical inference results associated with the old version will be marked as not directly reusable, and can only be updated to a reusable state after passing the industry constraint verification again. S110, Execution Deadline-Aware Batch Scheduling Requests are grouped based on the request inference level, execution graph structure, input length, predicted output length, deadline, risk level, cache similarity, and the type of industrial constraint validation to be performed. Calculate the scheduling priority for each request: in: To request the remaining deadline; Risk level; Benefits of cache reuse; To ensure graph compatibility; To estimate the computing resources required; To prevent positive numbers with a denominator of zero. Requests with the same execution graph, the same inference level, the same type of industrial constraint verification to be executed, and a sequence length difference of less than the length threshold are grouped into dynamic batches. When the prediction completion time of a high-risk request exceeds its deadline, or when a high-risk request switches from a lower level to a high-reliability inference layer during inference, the current batch is split and the high-risk request is executed first. S111, Perform industrial constraint verification Perform industrial constraint verification on the inference results, wherein the industrial constraints include at least one of the following: 1. Upper and lower limits constraints on equipment parameters; 2. Unity consistency constraint; 3. Process material balance constraints; 4. Energy conservation constraint; 5. Equipment state machine transition constraints; 6. Safety interlock constraints; 7. Operation permission constraints; 8. Maintenance procedure sequence constraints; 9. Consistency constraint of industrial knowledge evidence; 10. Prohibited Operational Constraints. When the inference result fails the industrial constraint verification or does not meet the control command output conditions, perform at least one of the following operations: Elevate the level of reasoning; Increase the number of knowledge retrievals; Call the validation model; Regenerate the inference results; Returns a security refusal result; Transfer to manual confirmation; Sending operating commands to industrial control equipment is prohibited; When the inference result passes the industrial constraint verification and meets the control command output conditions, the inference result, inference level, evidence index, confidence level, and constraint verification result are output.

[0008] Device Scheme: The present invention also provides an industrial large-scale model hierarchical dynamic reasoning acceleration device, including a request acquisition module, a hierarchical executable condition determination module, a hierarchical determination and budget configuration module, a cache verification module, a hierarchical reasoning and dynamic monitoring module, a hierarchical state machine module, a state transition module, a cache management module, a batch scheduling module, a constraint verification module, and a result output module; each module works together to execute the above-mentioned hierarchical reasoning method with industrial state version and industrial constraint verification as a closed loop.

[0009] Equipment Solution: The present invention also provides an electronic device, including a processor, an inference accelerator, an inference accelerator memory for storing hot cache, a main memory for storing warm cache, a non-volatile storage medium for storing cold cache, a communication interface, and a computer program stored in the memory and capable of running on the processor; when the processor executes the computer program, it implements the above-mentioned hierarchical dynamic inference acceleration method for large industrial models.

[0010] Beneficial effects: Compared with the scheme of using a fixed complete industrial model, the present invention allocates the computing budget according to the task complexity, industrial risk, deadline, resource status, equipment status version and industrial knowledge version; by setting the industrial status version and industrial constraint verification status for cache reuse, the repeated reasoning or insecure reuse caused by the failure of historical results is reduced.

[0011] During the decoding process, this invention outputs uncertainty, evidence consistency, industrial constraint violation status, and deadline margin into a hierarchical state machine with a hysteresis interval; high-risk violations are prioritized for escalation, and redundant calculations during hierarchical switching are reduced by sharing model layer key-value cache and constraint verification status; This invention performs batch scheduling based on industrial constraint verification types and feeds back the industrial rule verification results to the inference control process. When equipment scope, process relationship or safety interlock rules are violated, or control command output conditions are not met, upgrades, safety rollbacks, manual confirmations or control command blocking are triggered, thereby balancing inference efficiency and industrial safety. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiment description are briefly described below.

[0013] Figure 1 This is a flowchart of the hierarchical dynamic reasoning acceleration method for large industrial models according to the present invention.

[0014] Figure 2 This is a schematic diagram of the overall structure of the hierarchical dynamic reasoning acceleration system for industrial large-scale models of the present invention.

[0015] Figure 3 This is a schematic diagram of the hierarchical structure of the cache reuse layer, lightweight inference layer, standard inference layer, and high-reliability inference layer. Figure 4 This is a schematic diagram illustrating the dynamic upgrade and downgrade state transitions of the reasoning hierarchy. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some embodiments of this application, and not all embodiments.

[0017] Example 1: This example provides a hierarchical dynamic inference acceleration system for large industrial models. The system is deployed on factory edge computing nodes, enterprise private clouds, or edge-cloud collaborative computing environments. The system includes a request access service, an industrial context service, an industrial knowledge retrieval service, a hierarchical inference controller, a lightweight inference engine, a standard inference engine, a highly reliable inference engine, a cache manager, a resource scheduler, an industrial rule validator, and a result output service. The access service receives inference requests from equipment management systems, manufacturing execution systems, data acquisition and monitoring systems, enterprise asset management systems, industrial mobile terminals, or manual operation interfaces. Industrial context services can query equipment operating status, current process stage, real-time alarms, maintenance records, and data sensitivity level based on equipment identifiers; The industrial knowledge retrieval service searches for evidence relevant to the request from equipment manuals, operating procedures, maintenance cases, process knowledge graphs, and safety specifications. The hierarchical inference controller selects the initial inference level based on request characteristics, industry risks, and resource status, and adjusts the level during the inference process; The lightweight inference engine uses a 3B-parameter-level industrial model, or the first 12 layers and early exit head of a 14B-parameter-level industrial model. The standard inference engine uses the first 32 layers or all 40 layers of a 14B parameter-level industrial model, employing 8-bit weights, 16-bit activation values, and speculative decoding. The highly reliable inference engine employs a complete 40-layer industrial model and calls independent verification models to validate conclusions, evidence, and operational procedures.

[0018] Example 2: Initial layer selection: For device nameplate parameter query requests, when the cache semantic similarity is 0.96, the device status version has not changed, the knowledge version is the same, and the cache has not exceeded its validity period, the cache reuse layer L0 is selected; For low-risk knowledge queries such as "what is the rated voltage of a certain type of motor", when there is a cache miss, the task complexity value is 0.18, and the industrial risk value is 0.05, the lightweight inference layer L1 is selected. For equipment diagnostic requests such as "analyze possible causes of continuous increase in compressor vibration value", when the task complexity value is 0.61 and the industrial risk value is 0.42, the standard inference layer L2 is selected; For requests involving safety controls, such as "determining whether an emergency shutdown should be performed immediately", when the industrial risk value is 0.91, regardless of the task complexity and resource pressure values, the high-reliability inference layer L3 is selected.

[0019] Example 3: Dynamic Hierarchical Adjustment: A compressor alarm interpretation request initially enters the lightweight inference layer L1. After generating the first decoding window, the model output probability entropy is 0.31, the evidence consistency is 0.94, and no industrial constraint violation is found, therefore L1 is maintained; After generating the second decoding window, the system detected from real-time data that the exhaust temperature exceeded the safety threshold, the industrial risk value increased from 0.20 to 0.78, and the industrial constraint violation status was set to 1. If the hierarchical inference controller calculates that the level upgrade score exceeds the upgrade threshold, it will upgrade the inference level from L1 to L2. The system reuses the tag sequences, retrieval evidence, device entity vectors, and shared key-value caches of the first 12 layers already generated by the lightweight inference layer, and only calculates the key-value states of layers 13 to 32. When the standard inference layer finds that the alarm combination matches the safety interlock conditions, the inference layer is upgraded to L3, and a shutdown suggestion is generated from the complete model and the verification model, and the result is marked as requiring manual confirmation.

[0020] Example 4: Hierarchical caching: Hot cache stores key-value states in fixed-size cache blocks, with each cache block corresponding to a preset number of tags. For system prompts, enterprise security specifications, and device model descriptions that are shared by multiple requests, the corresponding prefix cache blocks will be set as shared read-only cache blocks. The cache manager calculates a cache retention score based on access frequency, recovery cost, risk level, and probability of invalidation: in: Access frequency; To recalculate the cost; Associating the risk level of the request; This refers to the cache block size. This represents the probability of failure. When the hot cache capacity is insufficient, the cache block with the lowest retention score is migrated to the warm cache; when the warm cache capacity is insufficient, the low-frequency state is digested and written to the cold cache.

[0021] Example 5: Dynamic Batch Processing: The scheduler scans the queue to be executed every 10 milliseconds and selects requests with the same inference level, model execution graph, and quantization precision as candidate batches; When the difference in the predicted remaining output length between two requests does not exceed 64 tags and the difference in the deadline does not exceed 500 milliseconds, the two requests are merged into the same batch. When the deadline margin for a high-risk request is less than 200 milliseconds, the request is split from the original batch and allocated to a reserved high-priority computing stream. When the graphics processor memory utilization exceeds 90%, the scheduler suspends accepting new low-priority requests, migrates some warm cache to main memory, and reduces the maximum batch size of low-risk requests.

[0022] Example 6: Simulation Verification: To verify the technical effects of hierarchical dynamic inference control, state transition, hierarchical caching, and deadline-aware batch scheduling, a discrete event simulation environment is constructed. This simulation is used to illustrate the relative performance trends under different scheduling strategies and does not represent the actual performance of a specific server or a specific large model. The simulation uses a 14B parameter, a 40-layer target industrial model and a 3B parameter draft model, and sets up four levels of inference from L0 to L3, two abstract 80GB AI accelerators, 3000 industrial requests, and task distribution with different complexities and deadlines. The comparison schemes include the fixed complete model scheme, the fixed quantization model and continuous batch processing scheme, and the static small model-large model cascade scheme that does not perform dynamic adjustment at the decoding window level; Simulation results show that, compared to the fixed complete model scheme, the present invention reduces the average end-to-end latency by approximately 55.4%, P95 latency by approximately 39.5%, average first-mark latency by approximately 60.4%, output throughput by approximately 116.6%, peak accelerator memory usage by approximately 29.1%, and relative power consumption by approximately 53.9%. For requests with low confidence, conflicting evidence, high risk, or violations of industry rules, the system can be upgraded to a standard inference layer or a high-reliability inference layer. Average end-to-end delay 2.62 seconds 1.97 seconds 1.79 seconds 1.17 seconds P95 End-to-End Delay 5.46 seconds 4.06 seconds 4.19 seconds 3.30 seconds Average first mark delay 483 milliseconds 350 milliseconds 293 milliseconds 191 milliseconds Output throughput 1247 marks / second 1652 marks / second 1808 marks / second 2701 marks / second Peak accelerator memory usage 63.2GB 47.5GB 52.1GB 44.8GB Relative energy consumption per 100 requests 10.9kJ 8.2kJ 7.5kJ 5.0kJ Deadline fulfillment rate 79.0% 96.1% 94.4% 99.6% Industrial task pass rate 95.4% 94.5% 93.5% 95.7% Industrial constraint pass rate 98.8% 98.8% 98.1% 99.6% .

[0023] Example 7: Electronic device: The electronic device includes a processor 1100, a memory 1200, a communication interface 1300, and an inference accelerator 1400. The memory 1200 stores computer programs, industrial model parameters, industrial rules, scheduling parameters, and cache indexes. The processor 1100 executes the computer program and controls the inference accelerator 1400 to complete the model calculation. The communication interface 1300 is used to communicate with industrial data acquisition systems, equipment management systems, knowledge base servers, and edge computing nodes. The processor may be a central processing unit, a graphics processing unit, a neural network processor, a digital signal processor, or other programmable processor. Without departing from the concept of this invention, any equivalent substitutions or modifications made to the number of inference levels, scoring parameters, model type, caching medium, scheduling cycle, and constraint rules shall fall within the protection scope of this invention.

Claims

1. A method for accelerating hierarchical dynamic reasoning in large industrial models, characterized in that, Executed by the industrial large-scale model inference service system, the process includes: acquiring an industrial inference request, industrial context information associated with the industrial inference request, and computing resource status, wherein the industrial context information is used to characterize at least the equipment status version, industrial knowledge version, process stage, and safety risk level; constructing a request feature vector based on the industrial inference request, industrial context information, and computing resource status, and determining the task complexity value, industrial risk value, resource pressure value, and deadline margin; determining the level executable conditions for the cache reuse layer, lightweight inference layer, standard inference layer, and high-reliability inference layer based on the equipment status version, industrial knowledge version, industrial risk value, industrial constraint verification conditions, and deadline margin, and determining the initial inference level and its computational budget from the candidate inference levels that meet the level executable conditions; when the cache reuse layer hits a historical inference result or historical prefix state, verifying the equipment status version, industrial knowledge version, validity period, and industrial constraint verification status of the historical inference result or historical prefix state, and verifying the pass / fail result. The historical inference results or historical prefix states are reused when they become obsolete. If the verification fails, industrial large model inference is executed according to the initial inference level and calculation budget. After each generation decoding window, intermediate inference states including output probability entropy, retrieval evidence consistency, industrial constraint violation status, current industrial risk value, and current deadline margin are obtained. The intermediate inference states are input into a hierarchical state machine with upgrade and downgrade thresholds to dynamically upgrade or downgrade the inference level. When the inference level changes, the generated tag sequence, retrieval evidence index, and constraint verification status are migrated. When sharing the model layer, the corresponding key-value cache is reused. Industrial constraint verification is performed on the inference results obtained after the level switch. When the industrial constraint verification is passed and the control command output conditions are met, the equipment fault diagnosis results, process parameter analysis results, production anomaly handling results, quality defect analysis results, or safety risk identification results are output. When the industrial constraint verification is not passed or the control command output conditions are not met, a safety rollback, transfer to manual confirmation, or prohibition of sending control commands is executed.

2. The method according to claim 1, characterized in that, The executability conditions for the cache reuse layer include: the current request is a non-state-changing request; the device state version and industrial knowledge version corresponding to the historical inference results are consistent with the current industrial context information; and the historical inference results retain the verification status that has passed the industrial constraint verification. The lightweight inference layer is used to execute a lightweight industrial model or a partially shared model layer of a large industrial model. The standard inference layer is used to execute a standard model depth and employ speculative decoding. The highly reliable inference layer is used to execute a complete large industrial model, enhance knowledge retrieval, and perform at least one result verification. By predicting the inference latency, energy consumption, memory usage, and computational cost corresponding to each candidate inference layer, penalty terms are set for candidate layers that do not meet the risk constraints, accuracy constraints, deadline constraints, or the executability conditions of the layer, and the candidate layer with the lowest overall cost is determined as the initial inference layer.

3. The method according to claim 1, characterized in that, The hierarchical state machine calculates a hierarchical upgrade score based on the normalized output probability entropy, the inverse of the candidate label probability interval, the inverse of the speculative label acceptance rate, the inverse of the consistency of retrieved evidence, the industrial constraint violation status, the current industrial risk value, and the deadline exceeding the limit. When the industrial constraint violation status indicates the existence of a high-risk violation, the downgrade judgment of the current decoding window is ignored and the inference level is upgraded. When the hierarchical upgrade score is greater than the upgrade threshold, the inference level is upgraded. When the hierarchical upgrade score is lower than the downgrade threshold for a consecutive preset number of decoding windows, the current industrial risk value is lower than the risk threshold, the deadline margin is greater than the margin threshold, and the currently generated content does not contain the control instruction to be verified, the inference level is reduced at the semantic segment boundary, wherein the upgrade threshold is greater than the downgrade threshold.

4. The method according to claim 1, characterized in that, When the reasoning level changes, the segmenter version, embedding layer version, and shared model layer identifier of the reasoning model before and after the level change are compared first. When the segmenter version and embedding layer version are consistent and at least part of the model layer is shared, the key-value cache and constraint verification state corresponding to the shared model layer identifier are reused, and only the key-value state of the newly added model layer is calculated. When the key-value cache reuse condition is not met, the input tag sequence, the generated tag sequence, the retrieval evidence index, the industrial entity state, and the compressed semantic state are sent to the target reasoning model to reconstruct the unshared key-value cache.

5. The method according to claim 1, characterized in that, The inference cache is divided into a hot cache located in the inference accelerator memory, a warm cache located in the main memory, and a cold cache located in non-volatile storage media or a distributed cache. Each cache item is associated with a model version, tokenizer version, device identifier, device status version, industry knowledge version, hint prefix hash value, and industry constraint verification status. When the device status version or industry knowledge version changes, the cache item associated with the old version of the historical inference result is marked as not directly reusable. A cache retention score is determined based on the cache access frequency, recalculation cost, associated request risk level, cache block size, cache invalidation probability, and the industry constraint verification status. Cache data is then migrated between the hot cache, warm cache, and cold cache based on the cache retention score.

6. The method according to claim 1, characterized in that, Industrial inference requests are grouped based on the request inference level, model execution graph, prediction output length, deadline, risk level, cache similarity, and type of industrial constraint verification to be executed. Industrial inference requests with the same model execution graph, the same inference level, the same type of industrial constraint verification to be executed, and a prediction output length difference of less than a length threshold are grouped into dynamic batches. When the prediction completion time of a high-risk industrial inference request exceeds the corresponding deadline, or when the high-risk industrial inference request switches from a lower level to a high-reliability inference layer during the inference process, the dynamic batch is split and the high-risk industrial inference request is executed first.

7. The method according to claim 1, characterized in that, The industrial constraints include at least one of the following: upper and lower limit constraints for equipment parameters, unit consistency constraints, process material balance constraints, energy conservation constraints, equipment state machine transition constraints, safety interlock constraints, operation permission constraints, maintenance step sequence constraints, industrial knowledge evidence consistency constraints, and prohibition of operation constraints. The control command output conditions include at least the following: industrial constraint verification passed, retrieval evidence consistency meets a preset threshold, and operation permission verification passed. When the reasoning result fails the industrial constraint verification or does not meet the control command output conditions, at least one of the following operations is executed: raising the reasoning level, increasing the number of knowledge retrievals, calling the verification model, regenerating the reasoning result, returning a safety rejection result, transferring to manual confirmation, and prohibiting the sending of control commands.

8. A hierarchical dynamic reasoning acceleration device for large industrial models, characterized in that, include: The request acquisition module is used to acquire industrial inference requests, industrial context information, and computing resource status. The industrial context information includes equipment status version, industrial knowledge version, process stage, and security risk level. The module for determining hierarchical executable conditions is used to determine the hierarchical executable conditions corresponding to industrial context information for the cache reuse layer, lightweight inference layer, standard inference layer, and high-reliability inference layer. The module for determining the hierarchical level and configuring the budget is used to determine the initial inference level from the candidate inference levels that meet the hierarchical executable conditions and configure the computation budget. The module for verifying the device status version, industrial knowledge version, validity period, and industrial constraint verification status of historical inference results or historical prefix states is used. The module for hierarchical inference and dynamic monitoring is used to perform industrial large model inference and obtain intermediate inference states after the decoding window. The module for hierarchical state machine is used to dynamically adjust the inference level based on the intermediate inference states. The state transition module is used to migrate the inference state when the inference level changes and reuse the shared model layer key-value cache when the version consistency condition is met; the cache and batch scheduling module is used to manage the hierarchical cache according to the industrial constraint verification type and form dynamic batches; the constraint verification and result output module is used to output the inference result when the control command output condition is met, and to perform a safety rollback, manual confirmation or control command blocking when the condition is not met.

9. An electronic device, characterized in that, The device includes a processor, an inference accelerator, a memory, a non-volatile storage medium, and a communication interface. The memory of the inference accelerator is used to store a hot cache, the memory is used to store a warm cache, and the non-volatile storage medium is used to store a cold cache. The memory stores a computer program, which, when executed by the processor, enables the electronic device to implement the hierarchical dynamic inference acceleration method for large industrial models as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the processor performs the hierarchical dynamic reasoning acceleration method for large industrial models as described in any one of claims 1 to 7.

Images