Inspection task adaptive arrangement system

Abstract

The invention relates to an inspection task adaptive arrangement system, which adopts a multi-agent collaborative architecture and comprises a task definition layer, an analysis and planning layer, a sensing and reasoning layer, a verification and fusion layer and a summarization and analysis layer, and the layers work cooperatively through multiple agents to realize a closed loop from a task script to execution feedback. Different agents cooperate to complete the whole inspection process, information is shared through message transmission, and a closed loop inspection task adaptive arrangement system is formed. Compared with a single model for processing routing inspection task arrangement, confusion caused by simultaneous processing of all tasks by the single model is avoided. Meanwhile, by adopting the hierarchical multi-agent mode, delay can be reduced, the number of model calling times can be reduced, and better auditing performance can be provided. According to the multi-agent cooperation architecture provided by the invention, high modularization and robustness are brought.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection technology, specifically to an adaptive scheduling system for inspection tasks. Background Technology

[0002] Intelligent inspection is a new type of inspection method that utilizes advanced information technology, automation technology, and artificial intelligence technology to conduct regular or irregular inspections, monitoring, and management of specific areas, equipment, and systems. It aims to improve the efficiency, accuracy, and reliability of inspections, promptly identify potential problems, and provide early warnings and solutions to ensure the normal operation of production and other activities.

[0003] Intelligent inspection also includes store inspection. Stores can install various sensors, such as door and window sensors to monitor whether the store's doors and windows are opened abnormally; smoke sensors and temperature and humidity sensors to monitor the safety of the store environment in real time; and customer flow statistics cameras to analyze customer flow at different times to help merchants understand their business situation.

[0004] Headquarters or management personnel can use a mobile app or computer system to assign inspection tasks to inspectors, specifying the inspection content, standards, and timeframes. Upon arrival at the store, inspectors will check various aspects of the store according to system prompts, such as merchandise display, hygiene, equipment operation (e.g., air conditioning, lighting), and employee service standards. They will record the inspection results through photos, text descriptions, etc., and upload them to the system in real time.

[0005] The system will organize and analyze the inspection data, generating detailed reports and charts to help managers understand the operational status of each store. When abnormalities are detected (such as a store failing to meet hygiene standards or equipment malfunctions), the system will promptly issue an early warning notification so that managers can take timely measures to resolve the problem.

[0006] However, traditional store inspections rely primarily on manual labor or monitoring systems based on fixed rules, resulting in high costs, poor real-time performance, and incomplete coverage. Manual inspections, due to limited manpower, often struggle to identify all issues promptly, and the frequency and quality of inspections are inconsistent. Some existing computer vision solutions can quickly convert shelf photos into structured data, enabling the detection of stockouts or display errors. For example, some manufacturers use image recognition to identify stockouts and misplacement within seconds and issue alerts to staff. However, these systems are mostly designed for single tasks, lacking flexible task orchestration capabilities and difficulty in adapting to different stores and scenarios. Furthermore, traditional systems often operate with a single intelligent agent or module, lacking mechanisms for collaboration among multiple intelligent modules, and failing to fully utilize the capabilities of the latest large-scale multimodal models to improve the robustness and intelligence of inspections. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to provide an adaptive scheduling system for inspection tasks, so as to solve the problems of rigid and poor adaptability of inspection task scheduling in the prior art.

[0008] According to a first aspect of the present invention, an adaptive scheduling system for inspection tasks is provided, comprising: The task definition layer is used to receive the inspection task orchestration script input by the user. The inspection task orchestration script includes a task declaration, user-defined inspection steps under the task declaration, and control parameters configured for each inspection step. The parsing and planning layer is used to parse the inspection task orchestration script into an executable intermediate representation through the built-in planning agent, and adjust the parsing parameters or modify the inspection task orchestration script when it cannot be parsed into an executable intermediate representation. The perception and reasoning layer is used to execute each subtask contained in the intermediate representation through the built-in execution agent, and to obtain the execution result of each subtask by calling the perception tool bound to each subtask through the built-in perception agent. The verification and fusion layer is used to fuse and verify the execution results of each subtask through a built-in summary agent; The summary and analysis layer is used to aggregate the fusion verification results of all subtasks through the built-in summary agent and output the inspection results.

[0009] Preferably, the parsing and planning layer is used to parse the inspection task orchestration script into an executable intermediate representation through a built-in planning agent, including: After receiving the inspection task scheduling script, the built-in planning agent constructs corresponding prompts and sends them to the pre-trained large language model; the prompts carry the grammatical information of the inspection task scheduling script, as well as the format requirements for the parsing results. Based on the prompts, the large language model parses the inspection task orchestration script into an executable intermediate representation, which includes: a task declaration, a list of inspection steps under the task declaration, and corresponding control parameters.

[0010] Preferably, when the parsing and planning layer cannot parse into an executable intermediate representation, it adjusts the parsing parameters or modifies the inspection task orchestration script, specifically as follows: When the built-in planning agent cannot resolve to an executable intermediate representation, it adjusts the prompts or prompts the user to modify the inspection task orchestration script through user interaction.

[0011] Preferably, the perception and inference layer is used to execute each subtask contained in the intermediate representation through a built-in execution agent, and to call the perception tool bound to each subtask to obtain the execution result of each subtask, including: Each inspection step in the inspection step list is defined as a subtask. If the subtask is to obtain the inspection status information, call the multimedia device bound to the subtask to obtain it; If the subtask involves comparing inspection data, the structured query tool bound to that subtask will be invoked to retrieve the data.

[0012] Preferably, the perception and reasoning layer obtains the execution result of each subtask by calling the perception tool bound to each subtask through the built-in perception agent, specifically as follows: The built-in perception agent calls a pre-trained multimodal model to process the perception information acquired by the sensing tool and outputs a judgment about the environmental state.

[0013] Preferably, the verification and fusion layer is used to perform fusion verification of the execution results of each subtask through a built-in summary agent, including: If the subtask is to obtain the inspection status information, the multimedia device bound to the subtask is invoked to collect status information at multiple consecutive times before and after the inspection time; based on the status information at multiple different sampling times, the final execution result is determined; or, If the subtask is to obtain the status information of an important inspection target, it calls multiple multimedia devices in different locations to collect the status information of the important inspection target; based on the status information from multiple different perspectives, it determines the final execution result.

[0014] Preferably, determining the final execution result based on state information at multiple different sampling times includes: Based on the confidence level of the state information at different sampling times, perform DS evidence fusion, or confidence-weighted fusion, and determine the fusion result as the final execution result; or... The process of determining the final execution result based on state information from multiple different perspectives includes: Based on the confidence level of the state information from different perspectives, perform DS evidence fusion, or confidence-weighted fusion, and determine the fusion result as the final execution result.

[0015] Preferably, the summary and analysis layer is used to aggregate the fusion verification results of all subtasks through a built-in summary agent and output the inspection results, including: If the execution result of a subtask that is greater than or equal to the preset number of items fails, the inspection result is determined to be a failure, and a summary report including the inspection result is output by calling the pre-trained intelligent model.

[0016] Preferably, the summary and analysis layer, used to aggregate the fusion verification results of all subtasks through a built-in summary agent and output the inspection results, further includes: If the execution results of any two or more subtasks contradict each other, the summary agent will mark the corresponding execution results. If the confidence level of the execution result of any subtask is lower than the threshold, the summary agent notifies the execution agent to re-execute the subtask, or notifies the planning agent to re-plan the subtask.

[0017] Preferably, the inspection task adaptive scheduling system further includes: The learning module stores the results of each inspection in a knowledge base for model training. The adjustment module is used to dynamically adjust the inspection frequency and model parameters based on historical inspection results.

[0018] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: A multi-agent collaborative architecture is adopted, including a task definition layer, a parsing and planning layer, a perception and reasoning layer, a verification and fusion layer, and a summarizing and analyzing layer. Each layer works collaboratively with multiple agents to achieve a closed loop from task script to execution feedback. Different agents cooperate to complete the entire inspection process, sharing information through message passing, forming a closed-loop adaptive orchestration system for inspection tasks. Compared to a single model handling inspection task orchestration, this avoids the chaos caused by a single model handling all tasks simultaneously. Furthermore, this layered multi-agent model reduces latency, decreases the number of model calls, and provides better auditability. The multi-agent collaborative architecture provided by this invention offers high modularity and robustness.

[0019] Furthermore, through a reasonable division of roles (planning, execution, and summarizing) and communication protocols, the adaptive orchestration system for inspection tasks provided by this invention can achieve a more powerful problem-solving capability than a single agent. The planning agent formulates the plan, executes the actions, and the summarizing agent verifies and summarizes the findings. In retail store inspection scenarios, multi-agent collaboration enables the system to flexibly respond to real-time changes and anomalies. For example, if a step fails, the summarizing agent can immediately request re-execution or plan adjustment, rather than causing the entire process to collapse. This solves the problems of rigid and poorly adaptive inspection task orchestration in existing technologies.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0022] Figure 1 This is a schematic block diagram illustrating an adaptive scheduling system for inspection tasks according to an exemplary embodiment; Figure 2 This is a schematic diagram illustrating the division of agent functions at each layer and the collaborative relationship among multiple agents according to an exemplary embodiment; Figure 3 This is a data flow timing diagram illustrating the collaborative operation of various agent layers according to an exemplary embodiment. Detailed Implementation

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0024] Figure 1 This is a schematic block diagram illustrating an adaptive scheduling system 100 for inspection tasks according to an exemplary embodiment, such as... Figure 1 As shown, the system 100 includes: The task definition layer 101 is used to receive the inspection task arrangement script input by the user. The inspection task arrangement script includes a task declaration, user-defined inspection steps under the task declaration, and control parameters configured for each inspection step. The parsing and planning layer 102 is used to parse the inspection task orchestration script into an executable intermediate representation through the built-in planning agent, and adjust the parsing parameters or modify the inspection task orchestration script when it cannot be parsed into an executable intermediate representation. The perception and reasoning layer 103 is used to execute each subtask contained in the intermediate representation through the built-in execution agent, and to obtain the execution result of each subtask by calling the perception tool bound to each subtask through the built-in perception agent. The verification and fusion layer 104 is used to perform fusion verification on the execution results of each subtask; The summary and analysis layer 105 is used to summarize the fusion verification results of all subtasks through the built-in summary agent and output the inspection results.

[0025] It should be noted that in the technical solution provided in this embodiment, each agent is responsible for different functions and communicates with each other through clearly defined interfaces.

[0026] The sensing tools include, but are not limited to, multimedia devices (such as cameras and recording devices), structured query tools, etc.

[0027] It is understood that the technical solution provided in this embodiment adopts a multi-agent collaborative architecture, including a task definition layer, a parsing and planning layer, a perception and reasoning layer, a verification and fusion layer, and a summary and analysis layer. Each layer works collaboratively through multiple agents to achieve a closed loop from task script to execution feedback. Different agents cooperate to complete the entire inspection process, sharing information through message passing to form a closed-loop adaptive orchestration system for inspection tasks. Compared to a single model handling inspection task orchestration, this avoids the chaos caused by a single model handling all tasks simultaneously. Furthermore, this layered multi-agent model can reduce latency, decrease the number of model calls, and provide better auditability. The multi-agent collaborative architecture provided in this embodiment brings high modularity and robustness.

[0028] Furthermore, through reasonable role division (planning, execution, and summarizing) and communication protocols, the adaptive orchestration system for inspection tasks provided in this embodiment can achieve more powerful problem-solving capabilities than a single agent. The planning agent formulates the plan, executes the agent's actions, and the summarizing agent verifies and summarizes the results. In retail store inspection scenarios, multi-agent collaboration enables the system to flexibly respond to real-time changes and anomalies. For example, if a step fails, the summarizing agent can immediately request re-execution or plan adjustment, rather than causing the entire process to collapse. This solves the problems of rigid inspection task orchestration and poor adaptability in existing technologies.

[0029] The task definition layer 101 mentioned above, also known as the DSL (Domain-Specific Language) layer, is used to describe inspection tasks in a scripting language. Taking a store inspection scenario as an example, the DSL provides a set of syntax and instructions for store inspection scenarios, allowing users to configure the inspection process without complex programming. For example, a simple DSL script example is as follows: TASK "OpenStoreCheck" { USE_CAMERA("FrontDoorCam"); STEP 1: VERIFY("DoorOpen") >= 0.9; STEP 2: VERIFY("LightsOn"); STEP 3: VERIFY("OpenSignVisible"); CONFIG interval = daily(09:00), tolerance = 5m; } In the script above, TASK "OpenStoreCheck" represents a task declaration; USE_CAMERA("FrontDoorCam") means using the front door camera; STEP 1: VERIFY("DoorOpen") >= 0.9 means using the front door camera to perform the first verification step and check if the store door is open; where >= 0.9 indicates that the confidence level for the "DoorOpen" check must be no less than 0.9. STEP 2: VERIFY("LightsOn"), which means using the front door camera to perform the second verification step and check if the lights are on; STEP 3: VERIFY("OpenSignVisible") means using the front door camera to perform the third verification step and check if the sign is lit up; CONFIG interval = daily(09:00), tolerance = 5m, represents the time interval for task execution (executed once daily at 9:00) and the tolerance parameter 5m, allowing a delay of up to 5 minutes (i.e., the check must be completed no later than 09:05). After the user deploys this task, the system will automatically trigger the inspection process at the specified time each day.

[0030] It should be noted that DSL scripts also support other commands and syntax, such as conditional statements, wait delays (WAIT), and parallel execution (PARALLEL), to adapt to the inspection needs of complex scenarios. Through DSL scripts, users can easily program inspection tasks in a near-natural language manner, greatly reducing the workload of hard-coding rules.

[0031] like Figure 2 As shown, the custom DSL script is parsed into a task plan by the LLM, the Executor Agent performs the perception action and sends the evidence to the fusion module, and the Summarizer Agent outputs the final report. The various intelligent agent modules work together to form a closed loop.

[0032] It is understood that the technical solution and multi-agent collaborative architecture provided in this embodiment are also applicable to the implementation of other tasks in other inspection scenarios. Users can refer to the above module pseudocode to customize corresponding logic for different scenarios. For example, "store opening inspection" can be extended to 24 / 7 scheduled store inspections, or to check whether personnel are on duty, whether there is debris piled on the ground, whether designated promotional advertisements are being used, etc. Only the schedule configuration of the DSL needs to be changed and the required inspection items added; another example is the inspection of shelf merchandise placement, where a CHECK command can be defined in the DSL in conjunction with a merchandise list, the Executor Agent calls OCR to identify shelf labels and the Summarizer Agent compares them with the inventory database. All new requirements can be quickly integrated on the basis of the existing framework. This fully demonstrates the flexibility and scalability of the architecture of this invention, providing the retail industry with an intelligent, efficient, and continuously evolving store inspection solution.

[0033] In practice, the parsing and planning layer 102 is used to parse the inspection task orchestration script into an executable intermediate representation through a built-in planning agent, including: After receiving the inspection task scheduling script, the built-in planning agent constructs corresponding prompts and sends them to the pre-trained large language model; the prompts carry the grammatical information of the inspection task scheduling script, as well as the format requirements for the parsing results. Based on the prompts, the large language model parses the inspection task orchestration script into an executable intermediate representation, which includes: a task declaration, a list of inspection steps under the task declaration, and corresponding control parameters.

[0034] To better understand the working principle of the parsing and planning layer 102, let's take the aforementioned store inspection scenario as an example. When the day arrives at 09:00, the Planning Agent reads the OpenStoreCheck DSL script and begins parsing and planning. The pseudocode implementation is as follows: # Planner Agent Pseudocode: function plan_and_dispatch(task_script): prompt = build_prompt("ParseDSL", task_script) IR = LLM.call(prompt) # Call LLM to resolve the DSL to IR. if not validate(IR): log("IR verification failed, task parsing error") return plan = IR.plan # Extract the list of subtasks # Distribute subtasks to the Executor for execution For subtasks in the plan: ExecutorAgent.execute(subtask) Here, prompt = build_prompt("ParseDSL", task_script) means first building a ParseDSL prompt, embedding the DSL script in the prompt, and requesting the Large Language Model (LLM) to output the Intermediate Representation (IR).

[0035] After obtaining the IR, basic structure validation is performed (such as whether the JSON format is complete, and whether each validation step VERIFY has a target). Once passed, the subtask list is extracted. Then, the Planner Agent calls the Executor Agent in sequence to execute each subtask.

[0036] In practice, the Planner Agent can publish subtasks to multiple Executor Agent worker threads via message queues, thus supporting parallel execution (in this example, the three verification steps VERIFY can be performed in parallel because they are independent of each other). For simplicity, the pseudocode here represents a sequential loop.

[0037] In practice, when the parsing and planning layer 102 cannot parse into an executable intermediate representation, it adjusts the parsing parameters or modifies the inspection task orchestration script, specifically as follows: When the built-in planning agent cannot resolve to an executable intermediate representation, it adjusts the prompts or prompts the user to modify the inspection task orchestration script through user interaction.

[0038] Understandably, the Planner Agent determines the flow and strategy of task execution, such as whether tasks should be executed sequentially or in parallel, and whether certain steps have preconditions. Once the plan is finalized, the Planner Agent assigns subtasks to the Executor Agent and monitors key nodes during execution (e.g., whether replanning is needed if a step fails). The Planner Agent is also responsible for managing task metadata, such as task priority, scheduling frequency, and coordination between multiple tasks (preventing resource conflicts, etc.).

[0039] In practice, the perception and reasoning layer 103 is used to execute each subtask contained in the intermediate representation through a built-in execution agent, and to call the perception tool bound to each subtask to obtain the execution result of each subtask, including: Each inspection step in the inspection step list is defined as a subtask. If the subtask is to obtain the inspection status information, call the multimedia device bound to the subtask to obtain it; If the subtask involves comparing inspection data, the structured query tool bound to that subtask will be invoked to retrieve the data.

[0040] Preferably, the perception and reasoning layer 103 obtains the execution result of each subtask by calling the perception tool bound to each subtask through the built-in perception agent, specifically as follows: The built-in perception agent calls a pre-trained multimodal model to process the perception information acquired by the sensing tool and outputs a judgment about the environmental state.

[0041] Understandably, the Executor Agent is responsible for executing individual subtasks, acting as the system's "worker" or executor. It can be viewed as the Executor Agent carrying out each step assigned by the Planner Agent. Upon receiving a subtask instruction, the Executor Agent invokes the corresponding perception tool or model to obtain the result. For example, for the "VERIFY('DoorOpen')" task to verify whether a door is open, the Executor Agent calls the Perception Agent to obtain the door status; for tasks requiring comparison with a database, the Executor Agent can invoke structured query tools, etc.

[0042] The Executor Agent typically encapsulates the larger model around the perception and inference layers. It selects appropriate methods to acquire evidence data based on different task types and submits the results (such as Boolean values / confidence scores) to the Summarizer Agent for aggregation. During execution, the Executor Agent can also perform simple logical checks and error handling. For example, if the model's confidence score is too low in a single identification, it can try adjusting the camera angle or re-acquiring the image after a short delay. This agent design allows each Executor Agent to focus on completing a specific sub-task, and multiple Executor Agents can work in parallel, thereby improving overall efficiency.

[0043] In this embodiment, the large model for the perception and inference layers preferably adopts a multimodal large model based on the Transformer architecture, which can fuse visual features and textual information for inference. The model can recognize objects and scenes in images, and can also read text within images (using OCR capabilities) and understand its meaning in context. For example, the model can recognize the opening and closing status of doors in camera footage, the brightness of lights inside stores, or read the text "OPEN / CLOSED" on signs, thereby determining whether the sign is lit. To improve the efficiency and availability of this stage, the system can use a vLLM (Vectorized Large Language Model) high-throughput service engine designed specifically for large language model inference for efficient deployment. vLLM allows large models to be served on GPUs with high throughput, processing multiple requests in parallel, thus supporting the simultaneous inspection needs of multiple cameras and multiple tasks. Through vLLM, the Perception Agent can host the multimodal model on a single server and utilize batch inference and asynchronous mechanisms to accelerate the response speed of image analysis.

[0044] After receiving subtasks from the Planner Agent, the Executor Agent takes appropriate actions based on the task type. For Verify VERIFY type tasks, the Executor Agent needs to interact with the environment to obtain the perception results. Its pseudocode logic can be represented as follows: # Executor Agent Pseudocode (for VERIFY tasks) function execute(subtask): target = subtask.target #eg, "DoorOpen" cam_id = subtask.camera #eg, "FrontDoorCam" # Get camera image image = Camera.capture(cam_id) # Construct a perception model prompt and query the target state vision_prompt = build_prompt("Detect", target, image) result = LLM.call(vision_prompt) # Call the multimodal model # Parse the model's returned results, such as Yes / No + confidence state, confidence = parse_detection(result) # Record Results record = { "target": target, "state": state, "confidence": confidence, "time": now() } # Send results to Summarizer SummarizerAgent.collect(record) For each VERIFY subtask, the Executor Agent first determines the target to be checked and the cam_id of the camera to be used (this information is defined in each subtask item of IR). Then, it captures the current image through the camera interface. Next, it constructs a vision_prompt for the multimodal LLM, for example, the vision_prompt could be: "Image description: {…}. Question: {Is the door open?}. Please answer yes or no, and give the confidence level.", along with the image. This is equivalent to having the model perform visual question answering (Visual QA). The LLM's returned results need to be parsed to extract the relevant state and confidence level; for example, the returned text might be: "Yes, the door is open. (Confidence level 0.95)". The Executor Agent compiles the results into a record, including the check item, the judgment state (yes / no or normal / abnormal), the confidence level, and the timestamp, and immediately sends it to the Summarizer Agent for summary processing via internal communication.

[0045] In practical implementations, the Executor Agent can perform more detailed operations as needed. For example, if the model confidence is too low (below a preset threshold), it can trigger a retake or adjust camera parameters and retry; if the camera cannot capture the image, it can record the abnormal state and continue to the next task. These robust measures ensure that the entire execution process completes all subtasks as much as possible and is not interrupted by the failure of a single step. Multiple Executor Agents can also execute different subtasks in parallel, improving efficiency when there are many subtasks.

[0046] In the example above, the Executor Agent will retrieve three results in sequence, for example: DoorOpen -> Store door detection result: Status "Open", confidence level 0.98, time 09:00.

[0047] LightsOn -> Light detection result: Status "On", confidence level 0.92, time 09:00.

[0048] OpenSignLit -> Sign detection result: Status "Not lit", confidence level 0.85, time 09:01.

[0049] Assuming the first two items are normal, but the third item may have a problem (the sign is not lit), all these results will be transmitted to the Summarizer Agent in real time.

[0050] In practice, the verification and fusion layer 104 is used to perform fusion verification of the execution results of each subtask through a built-in summary agent, including: If the subtask is to obtain the inspection status information, the multimedia device bound to the subtask is invoked to collect status information at multiple consecutive times before and after the inspection time; based on the status information at multiple different sampling times, the final execution result is determined; or, If the subtask is to obtain the status information of an important inspection target, it calls multiple multimedia devices in different locations to collect the status information of the important inspection target; based on the status information from multiple different perspectives, it determines the final execution result.

[0051] Preferably, determining the final execution result based on state information at multiple different sampling times includes: Based on the confidence level of the state information at different sampling times, perform DS evidence fusion, or confidence-weighted fusion, and determine the fusion result as the final execution result; or... The process of determining the final execution result based on state information from multiple different perspectives includes: Based on the confidence level of the state information from different perspectives, perform DS evidence fusion, or confidence-weighted fusion, and determine the fusion result as the final execution result.

[0052] Understandably, the verification and fusion layer is a key component in ensuring the reliability of the system's results. This embodiment employs multiple strategies to fuse perception results from different times and perspectives, including cross-temporal verification, multi-perspective cross-comparison, consistency determination, and uncertainty fusion based on Dempster-Shafer (DS) evidence theory.

[0053] Cross-time sequence verification: For checks that require continuity over time, the system collects data multiple times at different times for comparison. For example, if the store door is required to open at 9:00, photos of the store entrance can be taken consecutively at 8:59, 9:00, and 9:01 to verify the door's status changes. If the door is closed the first time but opens the second time, the Summarizer Agent will consider it normal (opening on time); if the door fails to open after multiple consecutive checks, an alarm will be generated. This redundant sampling across time reduces the impact of momentary misjudgments.

[0054] Multi-view cross-comparison: For important targets, the system can schedule multiple cameras or various sensors to observe simultaneously, acquiring evidence from different perspectives. For example, when a shop sign is lit, the system can simultaneously observe the light emission from the lamps using an indoor camera and the brightness of the sign using an outdoor camera. The results from multiple perspectives are summarized and compared by a summarizer agent: if all perspectives support the same conclusion, the confidence level is significantly increased; if there are conflicts, the system is marked as requiring manual review or further analysis.

[0055] Consistency Check: Before merging data, the Summarizer Agent applies a logical consistency check. For example, according to business logic, "door open" should usually be accompanied by "light on" (when the store opens); if the detection result shows that the door is open but the light is not on, the Summarizer Agent will identify this contradiction and attempt to explain it (e.g., possible light malfunction) or lower the confidence level of the conclusion and request human intervention. Similarly, if there is a causal relationship between the results of different subtasks, the agent will perform rule validation to ensure that the conclusions in the final report are consistent.

[0056] Dempster-Shafer Evidence Theory Fusion: For probabilistic evidence from independent sources, this system introduces Dempster-Shafer evidence theory for mathematical fusion. Dempster-Shafer theory is a framework for handling uncertain information, allowing the combination of belief functions from multiple sources to calculate a comprehensive confidence level. In this system, detection results from different perspectives or at different times can be considered independent evidence, each with its own confidence level assignment for a certain proposition (e.g., "the sign is lit"). The Summarizer Agent synthesizes this evidence into a comprehensive confidence level using Dempster's combination rules. For example, if camera A has an 80% confidence level that the sign is lit, and camera B has a 70% confidence level, the fused evidence will have a higher comprehensive confidence level (considering mutual support). If the two conclusions conflict (e.g., one is lit and the other is dark), Dempster-Shafer theory also provides a mechanism to handle the impact of conflicting evidence. Through evidence theory, the system can quantify the confidence level after fusion at the probabilistic level, providing a more solid quantitative basis for alarm / pass determination.

[0057] Confidence-Weighted Fusion: In addition to the DS theory, this embodiment also employs an intuitive confidence-weighted strategy as a supplement. That is, based on the historical reliability of each sensor / model, different weights are assigned to its output results for participation in the final decision. For example, if a camera has a high false detection rate in low-light conditions at night, the evidence weight of that camera is reduced at night; conversely, a model with consistently high accuracy can have its output given greater influence. The Summarizer Agent can maintain a weight table and dynamically adjust the weights of each evidence source based on the context, thereby achieving experience-adaptive fusion decision-making. This weighting strategy can be seen as a practical enhancement to the DS theory, incorporating prior knowledge into the fusion process and further improving the robustness of the final judgment.

[0058] By combining the above strategies, the verification and fusion layer maximizes the value of multi-source data, ensuring the reliability and credibility of the inspection conclusions provided by the system. Regardless of potential errors in a single-point model, they can be detected and corrected through cross-validation across multiple dimensions—space, time, and logic. For example, if one camera detects an anomaly, evidence fusion from another camera can still lead to a correct judgment, thus avoiding missed or false alarms caused by misjudgments from a single perspective. This multi-evidence fusion method is particularly suitable for the complex and ever-changing situations in retail environments, making AI inspections more akin to the human process of verifying decisions from multiple angles.

[0059] In practice, the summary and analysis layer 105 is used to aggregate the fusion verification results of all subtasks through a built-in summary agent and output the inspection results, including: If the execution result of a subtask that is greater than or equal to the preset number of items fails, the inspection result is determined to be a failure, and a summary report including the inspection result is output by calling the pre-trained intelligent model.

[0060] Preferably, the summary and analysis layer 105, used to summarize the fusion verification results of all subtasks through a built-in summary agent and output the inspection results, further includes: If the execution results of any two or more subtasks contradict each other, the summary agent will mark the corresponding execution results. If the confidence level of the execution result of any subtask is lower than the threshold, the summary agent notifies the execution agent to re-execute the subtask, or notifies the planning agent to re-plan the subtask.

[0061] It should be noted that the threshold is set based on historical experience or experimental data.

[0062] Understandably, the Summarizer Agent is responsible for verifying, integrating, and summarizing all execution results, essentially acting as a summarizer / evaluator. Once the execution agent returns the detection results for each sub-item of an inspection task, the Summarizer Agent aggregates this data for global analysis. For example, if there are contradictions in the results of some sub-tasks, the Summarizer Agent will identify and mark them; or if the confidence level of a sub-task's result is low, the Summarizer Agent can decide whether to require a re-execution. The Summarizer Agent applies pre-defined business rules (such as determining which items fail the inspection) and can also call the LLM (Limited Learning Model) to generate a report using natural language summarization (the prompt includes the task context and a list of detection results, allowing the model to output a summary description). In complex scenarios, the Summarizer Agent can also act as an Evaluator, triggering the Planner Agent to replan for low-confidence results (such as adjusting the inspection order or adding extra inspection steps). Finally, the Summarizer Agent outputs human-readable inspection reports or alert notifications and archives key information for continuous learning.

[0063] Understandably, the Summarizer Agent continuously collects results from the execution agent, and its main logic pseudocode is as follows: # Summarizer Agent Pseudocode collected_results = [] # Store the collected results records function collect(record): collected_results.append(record) if len(collected_results) == expected_count: analyze_and_report() # This means: The Summarizer Agent maintains a list to collect the results of all subtasks. When the number of results reaches a preset number (3 in this example), analyze_and_report() is triggered to perform analysis and summarization, entering the fusion judgment and summary stage.

[0064] function analyze_and_report(): # 1. Cross-step consistency check inconsistencies = check_consistency(collected_results) # means: First, perform a consistency check (check_consistency) to scan for contradictions between different results (such as inconsistencies like a door being open but a light being off).

[0065] # 2. Fusion of Multi-Source Evidence fused_results = [] for target in get_unique_targets(collected_results): evidences = [r for r in collected_results if r.target ==target] fused = fuse_evidences(evidences) # DS fusion or weighted average fused_results.append(fused) #This means: perform multi-source evidence fusion for each type of inspection target. If an inspection target corresponds to multiple pieces of evidence (e.g., multiple cameras or multiple samples), the `fuse_evidences` function is called to fuse them into a single result. `fuse_evidences` can be implemented as DS evidence combination, weighted average, or logical rules, outputting the fused state judgment and overall confidence level. If only a single piece of evidence is available, its result is used directly.

[0066] # 3. Determine the final state and generate a conclusion. conclusions = {} for res in fused_results: conclusions[res.target] = (res.state if res.confidence >=threshold else "Uncertain") #Meaning: Based on the fusion results, the final conclusions are: For each inspection item, if the overall confidence level is higher than the threshold and the status is normal, it is judged as passing; if the status is abnormal and the confidence level is high, a corresponding abnormal conclusion is also made; if the confidence level is insufficient, the conclusion is marked as "uncertain" to indicate that manual review is required.

[0067] # 4. Use LLM to generate natural language reports report_prompt = build_prompt("SummarizeReport", conclusions,inconsistencies) report_text = LLM.call(report_prompt) # 5. Output Report output(report_text) # 6. If there are significant anomalies or uncertainties, notify the Planner to reschedule. if need_recheck(conclusions, inconsistencies): PlannerAgent.replan(task, conclusions) #This means: The Summarizer Agent calls the LLM to generate an inspection report text: report_prompt contains a list of confirmed conclusions, explanations of inconsistencies, etc., allowing the model to output a summary statement. For example, the final report might be: "Store opening inspection results: The store door was opened on time, and the indoor lights are normal. However, the OPEN sign outside the store is not lit. Please check the sign's power supply as soon as possible." After obtaining the report text returned by the LLM, the Summarizer Agent outputs it to the user or management system.

[0068] Finally, the Summarizer Agent decides whether to notify the Planner Agent for adaptive adjustments based on the conclusions. If there are significant anomalies (such as the sign not lighting up in this example) or "uncertain" conclusions, the Summarizer Agent can invoke the Planner Agent's replan interface. For example, it can request the Planner Agent to schedule a follow-up check at a later time, or temporarily add a task step (such as switching to a backup camera to reconfirm the sign's status). This feedback mechanism ensures that the system proactively takes further action when problems are detected, rather than passively waiting for manual intervention.

[0069] Figure 3This diagram illustrates the communication flow between agents at each layer when a user or scheduling system triggers a store visit task. The Planning Agent translates the DSL task into an execution plan and assigns it to the Executor Agent for item-by-item checking. At each check step, the Executor Agent invokes a perception tool (camera) and uses a Large Language Model (LLM) for image analysis, sending the results to the Summarizer Agent. The Summarizer Agent performs evidence fusion and summarization, feeding the results back to the user and, if necessary, notifying the Planning Agent to adjust subsequent plans. This sequence diagram clearly depicts the data flow and collaborative relationships within the multi-agent framework.

[0070] Understandably, the "store opening inspection" task was successfully automated through the above process. In this embodiment, the system detected the anomaly of the store sign not being lit, automatically generated an alert report, and notified relevant personnel, thus facilitating timely problem resolution. If everything is normal, the system generates a report confirming that the store is operating normally and all preparations are complete. The entire process only takes less than tens of seconds to run in the background, and personnel only need to check the results, greatly improving the efficiency of store inspections.

[0071] Preferably, the inspection task adaptive scheduling system 100 further includes: The learning module stores the results of each inspection in a knowledge base for model training. The adjustment module is used to dynamically adjust the inspection frequency and model parameters based on historical inspection results.

[0072] It should be noted that, in order to continuously improve the performance of the inspection task adaptive orchestration system provided in this embodiment over time, this embodiment also designs a continuous learning and parameter adaptation mechanism, including model learning and strategy parameter adjustment.

[0073] On one hand, the system supports the use of new data collected during inspections for continuous learning. The results of each inspection task (including images, model judgments, and actual human feedback) are stored in a knowledge base. Periodically, this data can be used to fine-tune the multimodal model or update the LLM's prompting strategy, allowing the model to gradually adapt to environmental changes in specific stores. For example, if a store changes its signboard design, the model may initially fail to recognize it accurately. However, after several inspections that collect new signboard images and correct annotations, the model's recognition capabilities can be fine-tuned, reducing future misclassification rates. For LLM, the system can also accumulate common task scripts and parsing results for few-shot prompts or to train a dedicated small parsing model, improving the reliability of DSL parsing.

[0074] On the other hand, the system's key operating parameters are adaptively optimized, including sampling frequency, confidence threshold, and tool routing selection. Regarding sampling frequency, the system can dynamically adjust task scheduling frequency based on historical inspection results: if a certain inspection consistently passes, the system reduces its frequency to conserve resources; conversely, if related problems occur frequently during a certain period, the frequency is increased, or even real-time monitoring mode is activated. Regarding confidence threshold, the Summarizer Agent statistically analyzes the relationship between model predictions and actual results, automatically adjusting the confidence threshold for pass / alarm. For example, if a model's 0.8 confidence level for "normal" still indicates a high probability of actual anomalies, the threshold can be raised; conversely, if a 0.6 confidence level is almost certain, the threshold can be appropriately lowered to reduce false alarms. Self-optimization of tool routing means that when multiple perception tools are available, the Executor Agent learns from feedback to select the optimal tool combination. For example, for tasks involving reading price tag information, the system can compare the performance of an OCR engine and a multimodal model, automatically favoring the one with higher accuracy. As data accumulates, the agent can learn which tool is most effective in which environment, forming a self-optimizing strategy routing.

[0075] Furthermore, the system incorporates self-reflection and reinforcement mechanisms to enhance continuous learning. After each task, the agent reviews its decisions, summarizing the reasons for success and failure. If a check generates multiple false positives, the SummarizerAgent triggers the Planner Agent to update the DSL script or prompts, or adjust the fusion strategy, improving from the source. This self-reflective optimization allows the agent to continuously improve, much like humans summarizing lessons learned in their work, thus achieving so-called "experience-driven adaptation." In engineering implementation, techniques such as RLHF (Human Feedback Reinforcement Learning) can be used to introduce simulated reward signals (e.g., positive feedback for accurate task completion) to fine-tune the model's decision preferences, enabling the entire system to evolve towards reducing false positives and false negatives and improving efficiency.

[0076] Through continuous learning and parameter adaptation, the store inspection system provided in this embodiment will not remain static, but will become "smarter with use." As deployment time increases, the system will gradually reduce its reliance on human intervention, achieving more accurate identification and more rational scheduling configuration. This adaptive capability is invaluable in real-world retail environments because store displays, equipment conditions, and lighting conditions are constantly changing; only an AI system capable of learning can maintain good performance over the long term.

[0077] In summary, the adaptive inspection task orchestration system provided by this invention is applicable to multiple application scenarios, including intelligent store inspections in the retail industry. It introduces a custom inspection task orchestration language (DSL) for efficiently defining inspection tasks and achieves adaptive, robust, and efficient store inspection processes through multi-agent collaborative execution (including LLM planning, perception and recognition, result fusion, and summary analysis). This system architecture leverages the reasoning and planning capabilities of LLM and the perception capabilities of multimodal models, combined with evidence theory fusion strategies, significantly reducing manual configuration costs and improving the accuracy of cross-camera and multi-time-series verification.

[0078] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0079] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0080] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0081] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0082] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0083] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0084] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0085] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0086] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An adaptive scheduling system for inspection tasks, characterized in that, include: The task definition layer is used to receive the inspection task orchestration script input by the user. The inspection task orchestration script includes a task declaration, user-defined inspection steps under the task declaration, and control parameters configured for each inspection step. The parsing and planning layer is used to parse the inspection task orchestration script into an executable intermediate representation through the built-in planning agent, and adjust the parsing parameters or modify the inspection task orchestration script when it cannot be parsed into an executable intermediate representation. The perception and reasoning layer is used to execute each subtask contained in the intermediate representation through the built-in execution agent, and to obtain the execution result of each subtask by calling the perception tool bound to each subtask through the built-in perception agent. The verification and fusion layer is used to fuse and verify the execution results of each subtask through a built-in summary agent; The summary and analysis layer is used to aggregate the fusion verification results of all subtasks through the built-in summary agent and output the inspection results.

2. The adaptive scheduling system for inspection tasks according to claim 1, characterized in that, The parsing and planning layer is used to parse the inspection task orchestration script into an executable intermediate representation through a built-in planning agent, including: After receiving the inspection task scheduling script, the built-in planning agent constructs corresponding prompts and sends them to the pre-trained large language model; the prompts carry the grammatical information of the inspection task scheduling script, as well as the format requirements for the parsing results. Based on the prompts, the large language model parses the inspection task orchestration script into an executable intermediate representation, which includes: a task declaration, a list of inspection steps under the task declaration, and corresponding control parameters.

3. The adaptive scheduling system for inspection tasks according to claim 2, characterized in that, When the parsing and planning layer cannot parse into an executable intermediate representation, it adjusts the parsing parameters or modifies the inspection task orchestration script, specifically as follows: When the built-in planning agent cannot resolve to an executable intermediate representation, it adjusts the prompts or prompts the user to modify the inspection task orchestration script through user interaction.

4. The adaptive scheduling system for inspection tasks according to claim 2, characterized in that, The perception and reasoning layer is used to execute each subtask contained in the intermediate representation through a built-in execution agent, and to call the perception tool bound to each subtask to obtain the execution result of each subtask, including: Each inspection step in the inspection step list is defined as a subtask. If the subtask is to obtain the inspection status information, call the multimedia device bound to the subtask to obtain it; If the subtask involves comparing inspection data, the structured query tool bound to that subtask will be invoked to retrieve the data.

5. The adaptive scheduling system for inspection tasks according to claim 1, characterized in that, The perception and reasoning layer obtains the execution results of each subtask by calling the perception tool bound to each subtask through the built-in perception agent. Specifically: The built-in perception agent calls a pre-trained multimodal model to process the perception information acquired by the sensing tool and outputs a judgment about the environmental state.

6. The inspection task adaptive scheduling system according to claim 4, characterized in that, The verification and fusion layer is used to perform fusion verification on the execution results of each subtask through a built-in summary agent, including: If the subtask is to obtain the inspection status information, the multimedia device bound to the subtask is invoked to collect status information at multiple consecutive times before and after the inspection time; based on the status information at multiple different sampling times, the final execution result is determined; or, If the subtask is to obtain the status information of an important inspection target, it calls multiple multimedia devices in different locations to collect the status information of the important inspection target; based on the status information from multiple different perspectives, it determines the final execution result.

7. The adaptive scheduling system for inspection tasks according to claim 6, characterized in that, The step of determining the final execution result based on state information at multiple different sampling times includes: Based on the confidence level of the state information at different sampling times, perform DS evidence fusion, or confidence-weighted fusion, and determine the fusion result as the final execution result; or... The process of determining the final execution result based on state information from multiple different perspectives includes: Based on the confidence level of the state information from different perspectives, perform DS evidence fusion, or confidence-weighted fusion, and determine the fusion result as the final execution result.

8. The inspection task adaptive scheduling system according to any one of claims 1 to 7, characterized in that, The summary and analysis layer is used to aggregate the fusion verification results of all subtasks through a built-in summary agent and output the inspection results, including: If the execution result of a subtask that is greater than or equal to the preset number of items fails, the inspection result is determined to be a failure, and a summary report including the inspection result is output by calling the pre-trained intelligent model.

9. The inspection task adaptive scheduling system according to claim 8, characterized in that, The summary and analysis layer is used to aggregate the fusion verification results of all subtasks through a built-in summary agent and output the inspection results. It also includes: If the execution results of any two or more subtasks contradict each other, the summary agent will mark the corresponding execution results. If the confidence level of the execution result of any subtask is lower than the threshold, the summary agent notifies the execution agent to re-execute the subtask, or notifies the planning agent to re-plan the subtask.

10. The inspection task adaptive scheduling system according to claim 8, characterized in that, Also includes: The learning module stores the results of each inspection in a knowledge base for model training. The adjustment module is used to dynamically adjust the inspection frequency and model parameters based on historical inspection results.

Images