Task execution method and apparatus based on hybrid expert routing

Abstract

The application discloses a task execution method and device based on a hybrid expert routing, the method comprising: receiving an online detection task, extracting the features and service level target of the task; obtaining the current system running state, generating the budget constraint of the task according to the system running state and the service level target of the task; selecting an expert model and an execution mode and an execution path according to the budget constraint, the execution path comprising a fast path, a high-precision path and a degradation path; calling the corresponding expert model according to the selected path to perform reasoning, and outputting a unified evidence field. The scheme can ensure the high precision and reliability of the task output while ensuring the system response speed, adapt to various complex online scene requirements, and have good abnormal processing capability and task tracing capability.

Description

Technical Field

[0001] This application relates to the field of industrial vision inspection, and more specifically, to a task execution method and apparatus based on hybrid expert routing. Background Technology

[0002] In industrial vision inspection scenarios, the complexity, latency requirements, and resource availability of tasks are highly dynamic, making it difficult for fixed paths or single-model execution methods to meet diverse service needs.

[0003] "Expert models" or "mixed-expert models" (MoE) are an important model structure that can selectively activate some experts based on task characteristics, improving system efficiency and reducing inference costs. Traditional MoE methods mainly focus on routing optimization at the model level, but in actual deployment, there is still the problem of how to combine system resource status and service level objectives (SLOs) for dynamic task scheduling and model path selection.

[0004] Many current task execution systems lack unified scheduling logic and execution path strategies when facing different quality of service requirements, resulting in either excessive consumption of computing resources or inability to meet the requirements of high-precision services. Therefore, an intelligent, flexible, and service-oriented task execution mechanism is needed. Summary of the Invention

[0005] The main objective of this application is to provide a task execution method, apparatus, computing device, and storage medium based on hybrid expert routing. By adopting a hybrid expert routing mechanism, flexible and efficient computing resource scheduling is achieved, while ensuring the accuracy of detection and the real-time response of the system.

[0006] To achieve the above objectives, the first aspect of this application proposes a task execution method based on hybrid expert routing, comprising: receiving an online detection task and extracting the task's features and service level objectives; obtaining the current system operating status and generating a budget constraint for the task based on the system operating status and the task's service level objectives; selecting an expert model, execution method, and execution path based on the budget constraint, wherein the execution path includes a fast path, a high-precision path, and a degraded path; and invoking the corresponding expert model to perform inference based on the selected path and outputting a unified evidence field.

[0007] Furthermore, receiving online detection tasks and extracting task features and service level objectives include: registering task metadata information and writing the task into a pending queue when the task is accessed; obtaining the service level objective parameters of the task from the task metadata, including maximum latency, minimum accuracy, and priority.

[0008] Furthermore, the current system operating status is obtained, and budget constraints for the task are generated based on the system operating status and the service level target of the task, including: real-time monitoring of the system's GPU / CPU utilization, memory usage, and the length of the current pending task queue; and based on the current system load status and the service level target requirements of the task, the available latency window for the current task, the computing resources that the current task can be allocated, whether degradation is allowed, and the maximum number of retries allowed.

[0009] Furthermore, based on budget constraints, the expert model, execution method, and execution path are selected. The execution paths include fast paths, high-precision paths, or degraded paths, including: executing high-precision paths for tasks requiring complete decision evidence; executing fast paths for tasks with high-frequency requests, low accuracy requirements, or those requiring rapid initial screening; switching to high-precision paths when the confidence level of the lightweight model output is lower than a preset threshold; and executing degraded paths when tasks time out, systems malfunction, or resources are insufficient.

[0010] Furthermore, based on the selected path, the corresponding expert model is invoked for inference, and the unified evidence fields output include: according to the selected path, the corresponding set of expert models is scheduled, and the models already resident on the current device are loaded or reused; if a fast path is used, the confidence level is determined, and if the confidence level is higher than a preset threshold, the result is output; if the confidence level is lower than the preset threshold, the high-precision path is switched; if the task execution is abnormal or times out, a degraded path is triggered, and the output result is marked as degraded inference.

[0011] Furthermore, the evidence fields include model inference results, confidence scores, actual execution paths, latency, and the amount of computing resources used.

[0012] Furthermore, the execution methods include parallel execution and cascaded execution. Parallel execution involves running multiple models simultaneously and fusing the model results. Cascaded execution uses the result of the current model as the input of the next model, gradually improving the accuracy.

[0013] A second aspect of this application provides a task execution device based on hybrid expert routing, comprising: a task access module for receiving online detection tasks and extracting task features and service level objectives; a budget evaluation module for obtaining the current system operating status and generating budget constraints for the task based on the system operating status and the task's service level objectives; a path decision module for selecting an expert model, execution method, and execution path based on the budget constraints, wherein the execution path includes a fast path, a high-precision path, or a degraded path; and a model reasoning module for calling the corresponding expert model to perform reasoning based on the selected path and outputting a unified evidence field.

[0014] According to a third aspect of this disclosure, a computing device is provided, including at least one processor and at least one memory storing a computer program. The processor executes the task execution method based on hybrid expert routing from the first aspect of this disclosure.

[0015] According to a fourth aspect of this disclosure, a computer-readable storage medium storing a computer program is provided, wherein the computer program, when executed by a processor, implements the steps of the task execution method based on hybrid expert routing according to a first aspect of this disclosure.

[0016] As can be seen from the above scheme, the task execution method and apparatus based on hybrid expert routing provided in this application improves the efficiency and accuracy of task execution by flexibly selecting execution paths and expert models during task execution through the introduction of hybrid expert routing. Appropriate paths and execution methods are dynamically selected based on the task's service level objectives, system resource status, and task priority. Simultaneously, a unified evidence field ensures the traceability and transparency of task execution results. This approach is suitable for the automated execution of online detection tasks and can effectively cope with changing scenarios such as resource constraints, high-frequency requests, and high-precision requirements. Attached Figure Description

[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings: Figure 1 A flowchart illustrating the task execution method based on hybrid expert routing provided in this application; Figure 2 A schematic diagram of the structure of the task execution device based on hybrid expert routing provided in this application; Figure 3 This is a schematic block diagram of a computing device according to embodiments of the present disclosure. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] This solution combines task orchestration, multi-expert dynamic routing mechanisms, standardized evidence output, and policy governance assurance mechanisms to ensure that the entire system has high scalability (plug-in expert model access), interpretability (unified evidence generation and semantic summarization), and maintainability (policy versioning and replay mechanisms, policy support for canary deployment, version tracking, and rapid rollback). Specifically, firstly, image data and its metadata are aggregated across multiple dimensions, including batch number, product model, process parameters, and panel type. A context-aware task entry point is constructed, defining the task context by combining image metadata, detection targets, and the surrounding environment, providing an information foundation for subsequent task planning and model invocation. Then, the overall detection process is deconstructed into a standardized task chain (DAG), and the task execution sequence and toolchain plan are determined. Scheduling includes sub-tasks such as image preprocessing, model invocation, result merging, and interpretation generation, ensuring a reasonable, efficient, and monitorable process. Based on the constraints of cost (computation / invocation cost) and cycle time (processing latency) for different detection tasks, the optimal combination of expert models is dynamically selected. The expert model outputs structured detection results, packaged in a unified format into an evidence package, including: original and intermediate processed images, model output results, confidence assessment and contextual information, model invocation chain, and version information. By introducing Large Language Models (LLM) or Multimodal Language Models (VLM), semantic interpretations of the model results are generated based on evidence packages to interpret natural language. Combined with historical context, task summaries are generated to initially formulate subsequent processing suggestions or process strategies. Under the constraint mechanism of "evidence + consensus," illusion and reasoning drift are avoided, improving interpretability and credibility. The generated strategy draft undergoes rule verification, conflict detection, and other legality checks, historical replay evaluation (simulated operation compared with historical results), and canary release (gradual online verification) to generate reusable strategy versions. Multi-version management and rollback mechanisms are supported to achieve governance and controllability.

[0021] This disclosure proposes a task execution method based on hybrid expert routing to address the problem of efficiently and flexibly scheduling multiple expert models to complete online detection tasks under different service level objectives and system load conditions. Hybrid expert routing is a strategy that combines multiple expert models and dynamically selects the appropriate model based on different input features. Its main objective is to dynamically switch between multiple expert models, ensuring high accuracy while maximizing computational efficiency and real-time performance. This method is particularly suitable for online scenarios requiring high accuracy and real-time response.

[0022] By dynamically selecting task execution paths, expert models, and execution methods, this solution can optimize the utilization of computing resources, improve the efficiency of task execution, and ensure high accuracy and reliability of task execution results.

[0023] Figure 1 This is a flowchart illustrating the task execution method based on hybrid expert routing provided in this application. (Refer to...) Figure 1 As shown, in step S102, the online detection task is received, and the features and service level targets of the task are extracted.

[0024] Specifically, upon task access, task metadata information is registered, and the task is written to a pending queue. Service level target parameters (SLPs) for the task are obtained from the task metadata, including maximum latency, minimum accuracy, and priority. This information provides the foundation for subsequent task scheduling, expert selection, and path selection.

[0025] Subsequently, in step S104, the current system operating status is obtained, and the budget constraints for the task are generated based on the system operating status and the service level target of the task.

[0026] To rationally select expert models and execution paths, it is necessary to monitor the system's operational status in real time, such as current CPU and memory usage, and the length of the pending queue. Based on this information, the adequacy of current computing resources can be assessed. Service Level Objectives (SLOs), such as response time and throughput, are used to evaluate whether the current request's requirements can be met. Based on the current system load and the task's SLO requirements, the available latency window for the current task, the computing resources that can be allocated to the current task, whether degradation is allowed, and the maximum number of retries allowed are calculated. For example, when the system load is low, more resources can be allocated to ensure high-precision computation; while when the load is high, computing resource allocation needs to be optimized to prioritize the completion of critical tasks.

[0027] Next, in step S106, an expert model, execution method, and execution path are selected based on budget constraints. The execution path includes a fast path, a high-precision path, or a degraded path.

[0028] For example, the fast path uses lightweight models (such as YOLO-Nano and MobileNet variants) designed to quickly process input samples and perform rapid screening. In most scenarios, lightweight models can efficiently complete preliminary classification or detection tasks and quickly provide initial results. When the fast path's confidence is low, such as failing to effectively determine the type of defect in a sample or not providing sufficient evidence, it triggers entry into the high-precision path for further processing. This mechanism ensures that speed is maintained without sacrificing accuracy.

[0029] The high-precision path employs more complex and accurate expert models (such as Unet and HRNet). These models typically involve higher computational costs but provide more detailed analysis and greater accuracy. The high-precision path works in conjunction with the fast path, forming a dual guarantee system that ensures both rapid processing and high-precision processing.

[0030] Degradation paths are designed to handle system malfunctions or computation timeouts. When the processing time of a detection task in the fast or high-precision path exceeds a predetermined threshold, or when a system anomaly occurs, the degradation path is activated, executing degradation strategies to ensure the task is completed within a very short time. The purpose of degradation path design is to ensure the normal operation of the production line even when the system encounters problems or is overloaded, preventing the entire system from shutting down due to the anomaly of a single task. Degradation paths can use coarse models or other simplified algorithms to quickly obtain basic results, ensuring the production line is not affected. All tasks passing through the degradation path are logged in detail for subsequent analysis. This not only helps optimize degradation strategies but also allows for the identification and repair of potential system bottlenecks, hardware, or software problems.

[0031] By dynamically selecting different paths, we can adapt to the needs of different tasks, ensure task execution efficiency, and ensure that tasks can be properly processed under various system loads.

[0032] Finally, in step S108, the corresponding expert model is invoked according to the selected path to perform reasoning and output a unified evidence field.

[0033] Specifically, based on the selected path, the corresponding set of expert models is scheduled, and models already resident on the current device are loaded or reused. If a fast path is used, the output confidence level is determined. If the confidence level is higher than a preset threshold, the result is output. If the confidence level is lower than the preset threshold, the system switches to a high-precision path. If the task execution is abnormal or times out, a degraded path is triggered, and the output result is marked as degraded inference.

[0034] The reasoning results for each task will output a uniform evidence field, including the following information: Model inference results: The final output of the task.

[0035] Confidence score: The confidence level of the model's inference results, indicating the reliability of the results.

[0036] Actual execution path: The actual execution path selected by the task (fast, high-precision, or degraded path).

[0037] Delay time: The time taken to execute a task.

[0038] Amount of computing resources used: The computing resources used during task execution (such as GPU / CPU time, memory consumption, etc.).

[0039] To improve task execution efficiency, this solution supports two execution modes: parallel execution, where multiple expert models can execute tasks in parallel to accelerate result generation, suitable for high-frequency, low-latency tasks; and cascaded execution, where tasks are executed sequentially, with the output of each step serving as the input for the next, suitable for tasks requiring complex reasoning.

[0040] As can be seen from the above scheme, this disclosure improves the efficiency and accuracy of task execution by introducing hybrid expert routing and flexibly selecting execution paths and expert models during task execution. The system can dynamically select appropriate paths and execution methods based on the task's service level objectives, system resource status, and task priority. Simultaneously, it ensures the traceability and transparency of task execution results through a unified evidence field. This makes it suitable for the automated execution of online detection tasks and can effectively cope with changing scenarios such as resource constraints, high-frequency requests, and high-precision requirements.

[0041] Figure 2 A schematic diagram of the task execution device based on hybrid expert routing provided in this application. (Refer to...) Figure 2 As shown, the device 200 includes: a task access module 210, a budget evaluation module 220, a path decision module 230, and a model reasoning module 240.

[0042] The task access module 210 can receive online detection tasks and extract their features and service level objectives (SLAs). Specifically, upon task access, the task access module 210 registers the task's metadata information and writes the task into a processing queue. It retrieves the SLA target parameters from the task metadata, including maximum latency, minimum accuracy, and priority. This information provides the basis for subsequent task scheduling, expert selection, and path selection.

[0043] The budget assessment module 220 can obtain the current system operating status and generate budget constraints for the task based on the system operating status and the service level objectives (SLOs) of the task. To rationally select expert models and execution paths, it is necessary to monitor the system's operating status in real time, such as current CPU and memory usage, and the current queue length for pending tasks. Based on this information, it assesses whether the current computing resources are sufficient. It evaluates whether the current request requirements can be met by setting service level objectives (SLOs) such as response time and throughput. Based on the current system load and the service level objective requirements of the task, it calculates the available latency window for the current task, the computing resources that can be allocated to the current task, whether degradation is allowed, and the maximum number of retries allowed. For example, when the system load is low, more resources can be allocated to ensure high-precision computation; while when the load is high, it is necessary to optimize the allocation of computing resources and prioritize the completion of critical tasks.

[0044] The path decision module 230 can select the expert model, execution method, and execution path based on budget constraints. Execution paths include a fast path, a high-precision path, or a degraded path. For example, the fast path uses a lightweight model designed to quickly process input samples and perform rapid screening. In most scenarios, a lightweight model can efficiently complete preliminary classification or detection tasks and quickly provide preliminary results. When the confidence level of the fast path is low, such as failing to effectively determine the type of defect in the sample or failing to provide sufficient evidence, it triggers entry into the high-precision path for further processing. This mechanism ensures that speed is maintained without sacrificing accuracy.

[0045] High-precision paths employ more complex and accurate expert models, which typically involve greater computational demands but offer more refined analysis and higher accuracy. High-precision paths work in conjunction with fast paths, forming a dual-guarantee system that ensures both rapid and high-precision processing.

[0046] Degradation paths are designed to handle system malfunctions or computation timeouts. When the processing time of a detection task in the fast or high-precision path exceeds a predetermined threshold, or when a system anomaly occurs, the degradation path is activated, executing degradation strategies to ensure the task is completed within a very short time. The purpose of degradation path design is to ensure the normal operation of the production line even when the system encounters problems or is overloaded, preventing the entire system from shutting down due to the anomaly of a single task. Degradation paths can use coarse models or other simplified algorithms to quickly obtain basic results, ensuring the production line is not affected. All tasks passing through the degradation path are logged in detail for subsequent analysis. This not only helps optimize degradation strategies but also allows for the identification and repair of potential system bottlenecks, hardware, or software problems.

[0047] The model inference module 240 can invoke the corresponding expert model to perform inference based on the selected path and output a unified evidence field. Specifically, depending on the selected path, the model inference module 240 can schedule the corresponding set of expert models and load or reuse models already resident on the current device. If a fast path is used, the confidence level is determined. If the confidence level is higher than a preset threshold, the result is output. If the confidence level is lower than the preset threshold, the system switches to a high-precision path. If the task execution is abnormal or times out, a degraded path is triggered, and the output result is marked as degraded inference.

[0048] The reasoning results for each task will output a uniform evidence field, including the following information: Model inference results: The final output of the task.

[0049] Confidence score: The confidence level of the model's inference results, indicating the reliability of the results.

[0050] Actual execution path: The actual execution path selected by the task (fast, high-precision, or degraded path).

[0051] Delay time: The time taken to execute a task.

[0052] Amount of computing resources used: The computing resources used during task execution (such as GPU / CPU time, memory consumption, etc.).

[0053] As can be seen, the task execution based on hybrid expert routing provided in this application improves the efficiency and accuracy of task execution by flexibly selecting execution paths and expert models during the task execution process through the introduction of hybrid expert routing. The system can dynamically select appropriate paths and execution methods based on the service level objectives of the task, system resource status, and task priority. At the same time, it ensures the traceability and transparency of task execution results through a unified evidence field. It is suitable for the automated execution of online detection tasks and can effectively cope with changing scenarios such as resource constraints, high-frequency requests, and high-precision requirements.

[0054] Figure 3 This is a schematic block diagram of a computing device according to embodiments of the present disclosure. Figure 3 As shown, the computing device 300 may include a processor 310 and a memory 320 storing a computer program. When the computer program is executed by the processor 310, the computing device 300 is able to perform the steps of a production line image defect detection strategy generation method.

[0055] In embodiments of this disclosure, processor 310 may be, for example, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a processor based on a multi-core processor architecture, etc. Memory 320 may be any type of memory implemented using data storage technologies, including but not limited to random access memory, read-only memory, semiconductor-based memory, flash memory, disk storage, etc.

[0056] Furthermore, in embodiments of this disclosure, the computing device 300 may also include an input device 330, such as a keyboard or mouse. Additionally, the computing device 300 may also include an output device 340, such as a display.

[0057] In other embodiments of this disclosure, a computer-readable storage medium storing a computer program is also provided, wherein the computer program, when executed by a processor, is capable of implementing the steps of a task execution method based on hybrid expert routing.

[0058] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0059] Obviously, those skilled in the art should understand that the various units or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0060] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A task execution method based on hybrid expert routing, characterized in that, include: Receive online detection tasks and extract the task's features and service level objectives; Obtain the current system operating status, and generate budget constraints for the task based on the system operating status and the service level target of the task; Based on the budget constraints, an expert model, execution method, and execution path are selected, including fast path, high-precision path, and degraded path. The corresponding expert model is invoked based on the selected path to perform reasoning and output a unified evidence field.

2. The task execution method based on hybrid expert routing according to claim 1, characterized in that, The process of receiving online detection tasks and extracting task features and service level objectives includes: When a task is received, the task metadata information is registered and the task is written to the pending queue. Obtain the service level target parameters of the task from the task metadata. The service level target parameters include maximum latency, minimum accuracy, and priority.

3. The task execution method based on hybrid expert routing according to claim 1, characterized in that, The process of obtaining the current system operating status and generating budget constraints for the task based on the system operating status and the task's service level objective includes: Real-time monitoring of system GPU / CPU utilization, memory usage, and current queue length of pending tasks; Based on the current system load status and the service level target requirements of the task, calculate the available latency window for the current task, the computing resources that the current task can be allocated, whether degradation is allowed, and the maximum number of retries allowed.

4. The task execution method based on hybrid expert routing according to claim 1, characterized in that, The step of selecting an expert model, execution method, and execution path based on the budget constraints, wherein the execution path includes a fast path, a high-precision path, or a degraded path, includes: For tasks requiring complete decision-making evidence, execute high-precision paths; For tasks with high-frequency requests, low accuracy requirements, or those requiring rapid initial screening, execute a fast path. When the confidence level of the lightweight model output is lower than a preset threshold, it will switch to the high-precision path. Execute a fallback path when a task times out, the system malfunctions, or resources are insufficient.

5. The task execution method based on hybrid expert routing according to claim 1, characterized in that, The step of calling the corresponding expert model to perform reasoning based on the selected path and outputting unified evidence fields includes: Based on the selected path, schedule the corresponding set of expert models and load or reuse the models already resident on the current device; If the fast path is used, the output confidence level is determined. If the confidence level is higher than the preset threshold, the result is output. If the confidence level is lower than the preset threshold, the high-precision path is switched. If the task execution is abnormal or times out, the degraded path is triggered, and the output result is marked as degraded inference.

6. The task execution method based on hybrid expert routing according to claim 1, characterized in that, The evidence fields include model inference results, confidence scores, actual execution paths, latency, and the amount of computing resources used.

7. The task execution method based on hybrid expert routing according to claim 1, characterized in that, The execution methods include parallel execution and cascaded execution. Parallel execution involves running multiple models simultaneously and fusing the model results. Cascaded execution uses the result of the current model as the input of the next model to gradually improve accuracy.

8. A task execution device based on hybrid expert routing, characterized in that, include: The task access module is used to receive online detection tasks and extract the task's features and service level objectives. The budget assessment module is used to obtain the current system operating status and generate budget constraints for tasks based on the system operating status and the service level objectives of the tasks. The path decision module is used to select an expert model, execution method, and execution path based on the budget constraints. The execution path includes a fast path, a high-precision path, or a degraded path. The model reasoning module is used to call the corresponding expert model to perform reasoning based on the selected path and output a unified evidence field.

9. A computing device, characterized in that, include: At least one processor; as well as At least one memory storing a computer program; When the computer program is executed by the at least one processor, the computing device performs the steps of the task execution method based on hybrid expert routing according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the task execution method based on hybrid expert routing according to any one of claims 1 to 7.

Images