Multi-modal large model-based autonomous generation and scheduling method and system for pipe gallery inspection machine-dog cooperative task

Abstract

The invention provides a multi-modal large model-based autonomous generation and scheduling method for a pipe gallery inspection machine-dog cooperative task. The method comprises the following steps: fusing multi-modal data through a multi-modal large model to generate a task graph with a dependency relationship; performing dimension reduction on task features by adopting an LSTM embedded auto-encoder, and predicting resource requirements and uncertainty intervals thereof in combination with a speculation sampling strategy; on the basis, a weighted priority model is constructed to divide task queues, and a collaborative execution sequence of the robot dog and the unmanned aerial vehicle is generated by adopting an operator selection mechanism driven by reinforcement learning; a dynamic time slice is set in the task execution process, and preemptive scheduling is triggered based on confidence evaluation; meanwhile, monitoring a key value cache state of an unexecuted task, and realizing dynamic migration between the GPU and a CPU memory by adopting a delay updating mechanism; and dynamically updating the task graph and the scheduling strategy based on the real-time returned data. According to the invention, intelligent generation and adaptive scheduling of the inspection task are realized, and the system resource utilization rate and the inspection efficiency are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection technology, and in particular to a method for autonomous generation and scheduling of collaborative tasks between a robot and a patrol dog for pipe gallery inspection based on a multimodal large model. Background Technology

[0002] With the acceleration of urbanization, underground utility tunnels, as an important component of urban infrastructure, are facing increasing importance for safety inspection. Traditional tunnel inspections mainly rely on manual methods, which suffer from low efficiency, high risk, and limited coverage. In recent years, with the development of robotics technology, automated inspections using robotic dogs and drones have become a trend. However, existing inspection systems still face the following technical bottlenecks: First, traditional task generation methods rely on preset rules, making it difficult to adapt to the complex and ever-changing environment of utility tunnels. The internal structure of utility tunnels is complex, containing various types of pipes, cables, and equipment. Inspection tasks need to consider multiple factors such as spatial layout, equipment type, and safety standards. Existing methods lack the ability to deeply integrate multi-source information.

[0003] Secondly, existing scheduling systems lack sufficient accuracy in predicting resource demand. Inspection tasks are affected by environmental interference, changes in equipment status, and other factors during execution, leading to significant uncertainty in resource requirements. Traditional scheduling methods employ fixed resource allocation strategies, which cannot effectively address such dynamic changes, easily resulting in resource waste or task failure.

[0004] Third, the collaborative scheduling of multiple devices lacks an intelligent decision-making mechanism. Robot dogs and drones have different motion characteristics and functional advantages. How to dynamically adjust task allocation and execution order according to real-time environmental conditions to achieve optimal collaborative effects is a technical problem that has not yet been well solved.

[0005] Furthermore, existing systems suffer from inefficient memory management when processing large-scale multimodal data. Particularly when using large models for environmental perception and decision-making, key-value caching consumes significant amounts of GPU memory, limiting the system's deployment capabilities on resource-constrained devices.

[0006] Therefore, there is an urgent need for a utility tunnel inspection solution that can achieve intelligent task generation, accurate resource prediction, multi-device collaborative scheduling, and efficient memory management, so as to improve inspection efficiency and quality and ensure the safe operation of urban utility tunnels. Summary of the Invention

[0007] This invention provides a method for autonomous generation and scheduling of collaborative tasks for pipe gallery inspection drones based on a multimodal large model, aiming to solve technical problems such as unintelligent task generation, inaccurate resource prediction, and low efficiency of multi-device collaboration in existing technologies.

[0008] In a first aspect, embodiments of this application provide a method for autonomous generation and scheduling of collaborative tasks for pipe gallery inspection drones based on a multimodal large model, the method comprising: By using a multimodal large model, text procedures, images, and 3D point clouds of the utility tunnel inspection scenario are jointly processed to generate a task graph with dependencies. Based on the task graph, an autoencoder with embedded LSTM is used to reduce the dimensionality of the task features, and a speculative sampling strategy is combined to predict the task resource requirements, outputting the total resource requirements and their uncertainty range. Based on the total resource requirements and their uncertainty range, a weighted priority model is constructed by combining task urgency and equipment status. The tasks are divided into multiple priority queues, and a reinforcement learning-driven operator selection mechanism is used to generate a collaborative execution sequence between the robot dog and the drone. During the operation of the collaborative execution sequence, dynamic time slices are set for each priority queue. When a task execution times out or a low-priority task waits for more than a threshold, preemptive scheduling based on confidence assessment is triggered, and the task queues are reallocated in combination with the optimal computational allocation strategy. During the task queue reallocation process, the key-value cache status of unexecuted tasks is monitored, and a delayed update mechanism is used to dynamically migrate the KV cache of future unscheduled tasks between GPU and CPU memory. Based on the multimodal data transmitted back by the robot dog and drone during the execution process, the system status is evaluated in real time through a multimodal large model, and the dependencies of the task graph and the weighted priority model are dynamically updated.

[0009] Secondly, embodiments of this application provide a system for autonomous generation and scheduling of collaborative tasks for tunnel inspection drones based on a multimodal large model, applied to the method for autonomous generation and scheduling of collaborative tasks for tunnel inspection drones based on a multimodal large model as described in the first aspect. The system includes: The task graph generation module is used to jointly process text procedures, images, and 3D point clouds of the utility tunnel inspection scenario using a multimodal large model to generate a task graph with dependencies. The resource demand prediction module is used to predict the task resource demand based on the task graph by using an autoencoder with embedded LSTM to reduce the dimensionality of the task features and combining it with a speculative sampling strategy, and output the total resource demand and its uncertainty range. The task scheduling module is used to construct a weighted priority model based on the resource requirements and their uncertainty range, combined with the task urgency and equipment status, divide the tasks into multiple priority queues, and generate a cooperative execution sequence between the robot dog and the drone using a reinforcement learning-driven operator selection mechanism. The dynamic scheduling and adjustment module is used to set dynamic time slices for each priority queue during the operation of the collaborative execution sequence. When the task execution times out or the waiting time of a low-priority task exceeds the threshold, it triggers preemptive scheduling based on confidence evaluation and reallocates the task queues in combination with the optimal computational allocation strategy. The cache management module is used to monitor the key-value cache status of unexecuted tasks during the task queue reallocation process, and to dynamically migrate the KV cache of future unscheduled tasks between GPU and CPU memory using a delayed update mechanism. The online learning optimization module is used to evaluate the system status in real time through a large multimodal model based on the multimodal data returned by the robot dog and drone during the execution process, and dynamically update the dependency relationship of the task graph and the weighted priority model.

[0010] Thirdly, embodiments of this application provide an electronic device, including: processor; Memory used to store processor-executable instructions; The processor is configured to implement the autonomous generation and scheduling method for collaborative tasks of pipe gallery inspection drones based on a multimodal large model as described in the first aspect when executing the instructions.

[0011] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that instructs a device to execute the autonomous generation and scheduling method for collaborative tasks of a pipe gallery inspection robot dog based on a multimodal large model as described in the first aspect.

[0012] The main technical solution of this invention includes: First, a multimodal large model is used to jointly process the text procedures, images, and 3D point clouds of the pipeline inspection scenario to generate a task graph with dependencies; then, an autoencoder with embedded LSTM is used to reduce the dimensionality of the task features, and a speculative sampling strategy is combined to predict the task resource requirements and their uncertainty range; based on the prediction results, a weighted priority model is constructed to divide the tasks into multiple priority queues, and a reinforcement learning-driven operator selection mechanism is used to generate a cooperative execution sequence between the robot dog and the drone; during task execution, a dynamic time slice is set for each priority queue, and preemptive scheduling based on confidence evaluation is triggered when the task execution times out or the waiting exceeds the threshold; at the same time, the key-value cache status of unexecuted tasks is monitored, and a delayed update mechanism is used to realize dynamic migration between GPU and CPU memory; finally, the task graph and scheduling strategy are updated in real time based on the multimodal data returned by the robot dog and the drone.

[0013] The core innovations of this invention are: 1) intelligent generation and dynamic updating of inspection tasks through a multimodal large model; 2) accurate prediction of resource requirements using an autoencoder embedded with LSTM and a speculative sampling strategy; 3) intelligent collaborative scheduling of multiple devices through a reinforcement learning-driven operator selection mechanism; and 4) improved system resource utilization by designing a preemptive scheduling and KV cache dynamic migration mechanism based on confidence evaluation. Compared with existing technologies, this invention has the following advantages: it realizes the autonomous generation and adaptive scheduling of utility tunnel inspection tasks, significantly improves inspection efficiency and system resource utilization, effectively copes with complex and ever-changing utility tunnel environments, and ensures the reliable execution of inspection tasks. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of a method for autonomous generation and scheduling of collaborative tasks for pipe gallery inspection robots based on a multimodal large model, provided in an embodiment of this application.

[0015] Figure 2 The system architecture diagram for autonomous generation and scheduling of collaborative tasks of pipeline inspection robot dogs based on a multimodal large model provided in this application.

[0016] Figure 3 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0018] It should be noted that in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0019] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] Example 1

[0021] Figure 1 This is a schematic diagram illustrating the process of an autonomous generation and scheduling method for collaborative tasks of a utility tunnel inspection robot based on a multimodal large model, provided as an embodiment of this application. Figure 1 As shown, a method for autonomous generation and scheduling of collaborative tasks for utility tunnel inspection drones based on a multimodal large model includes: Step (1) Intelligent task generation: The text procedures, images, and 3D point clouds of the utility tunnel inspection scenario are jointly processed by a multimodal large model to generate a task graph with dependencies. By fusing text, image, and point cloud data through a multimodal large model, a task graph with logical dependencies is automatically constructed.

[0022] Specifically, in this embodiment, step (1) of generating a task graph with dependencies includes: The multimodal large model employs a visual encoder to extract equipment status features and meter readings from images, a point cloud encoder to extract spatial geometric features and deformation data of the pipe rack structure, and a text encoder to parse operational constraints and safety standards from inspection procedures. The visual encoder, a neural network module within the multimodal large model, is specifically designed to process image data, extracting meaningful visual features from images captured during inspections. For example, it can identify whether a valve is open or closed from an image of pipe rack equipment, or read the pointer reading (e.g., 1.5 MPa) on a pressure gauge. The point cloud encoder, a neural network module within the multimodal large model, is specifically designed to process 3D point cloud data, extracting spatial and geometric features of the pipe rack structure from point cloud data obtained through 3D laser scanning. This includes detecting depressions or protrusions on the pipe rack wall, calculating the precise location and direction of pipelines, or measuring the spatial distance between two pieces of equipment. The text encoder, a neural network module within the multimodal large model, is specifically designed to process text data, converting written inspection procedures and standards into structured information that computers can understand and process. For example, after inspecting the water pumps in area A according to the analytical procedure, it is necessary to check the water pressure in area B. From this, we can extract that there is a time dependency relationship (i.e., the order of priority) between the two tasks of checking the water pumps in area A and checking the water pressure in area B.

[0023] The aforementioned multimodal features are fused at the fusion layer through cross-modal alignment and attention weighting to identify temporal dependencies, spatial proximity relationships, and logical causal relationships between tasks. Different features from images, point clouds, and text are mapped to the same semantic space, enabling them to be correlated. For example, a valve identified in an image is associated with its 3D position in the point cloud and information about valve operation procedures in the text. The importance of different modal information is automatically determined. For example, when assessing equipment safety status, pressure gauge readings (image features) are given higher weight; when planning inspection routes, spatial distance (point cloud features) is given higher weight.

[0024] Based on the identified dependencies, a task graph structure is constructed with inspection tasks as nodes and dependencies as directed edges. Node attributes include task type, target location, and expected time, while edge attributes include dependency type and predecessor strength. The task graph structure is a mathematical model, typically a directed graph, used to represent all inspection tasks and their relationships. It formally describes the entire inspection plan and serves as a blueprint for subsequent scheduling and execution. Its components are as follows: Nodes: Represent a specific inspection task. Node A: {Task type: Read table, Target location: (X1, Y1, Z1), Expected time: 2 minutes}. Node B: {Task type: Take photo, Target location: (X2, Y2, Z2), Expected time: 1 minute}. Directed edges: Represent dependencies between tasks. Edge A->B: {Dependency type: Temporal dependency, Predecessor strength: High} / / Indicates that the table must be read before taking a photo.

[0025] The above steps achieve deep fusion of multimodal perception information, and based on this, intelligently and automatically construct a structured inspection task blueprint (task graph) containing rich related information, laying a solid foundation for subsequent intelligent scheduling. It transforms the originally isolated environmental data (graphs, text, 3D models) and procedure texts into a structured knowledge model that can be directly used by computers for planning and decision-making.

[0026] Step (2) Accurate Resource Prediction: Based on the task graph, an autoencoder with embedded LSTM is used to reduce the dimensionality of the task features. Combined with a speculative sampling strategy, the task resource requirements are predicted, and the total resource requirements and their uncertainty interval are output. An LSTM autoencoder is used for feature dimensionality reduction, and combined with a speculative sampling strategy, a resource requirement prediction including the uncertainty interval is output.

[0027] Specifically, in this embodiment, step (2) of predicting task resource requirements specifically includes: The node and edge features of the task graph are input into an autoencoder embedded in an LSTM. The LSTM layers capture the temporal dependencies between tasks, and the encoder-decoder structure maps the high-dimensional task features to a low-dimensional latent space. The LSTM-embedded autoencoder is a hybrid neural network structure combining a Long Short-Term Memory (LSTM) network and an autoencoder. The LSTM layers capture the temporal characteristics arising from dependencies in the task graph. For example, task B can only begin after task A is completed; this order affects the coherence of resource usage. The autoencoder compresses high-dimensional, sparse task features (such as task type, location, dependencies, etc.) into a low-dimensional, dense latent feature space, removing redundant information and retaining core features.

[0028] In a low-dimensional latent space, a speculative sampling strategy is employed to perform Monte Carlo sampling on multiple possible paths during task execution, predicting the computational resource consumption and communication overhead for each path. The speculative sampling strategy is a Monte Carlo sampling method used to predict multiple possible future scenarios through stochastic simulation in the presence of uncertainty. It simulates different paths and situations that may occur during task execution, thereby assessing various possibilities for resource requirements. For example, it simulates whether the robot dog will encounter obstacles on the inspection path, causing detours (increasing time and computational resources), or whether data transmission will be retransmitted due to signal interference (increasing communication overhead). Computational resource consumption and communication overhead refer to specific quantitative indicators of the hardware and network resources required for task execution, providing specific resource requirement data for scheduling decisions. Examples of computational resource consumption include: GPU computing power (e.g., 2 TFLOPs) and memory usage (e.g., 512MB) required to execute image recognition algorithms. Communication overhead includes: the amount of data transmission required for the robot dog to send the high-definition video stream back to the edge server (e.g., 500MB) and network latency (e.g., <100ms).

[0029] A probability distribution model of resource demand is established based on the sampling results. By calculating the mean and variance of this distribution, the estimated total resource demand for the task and its uncertainty interval are output, where the uncertainty interval is expressed as a confidence interval. The probability distribution model is a mathematical model used to describe the uncertainty of resource demand, such as a Gaussian distribution (normal distribution). It does not simply provide a single, definitive resource value, but rather a range of possible values ​​and their probabilities, more accurately reflecting the uncertainty in reality. The uncertainty interval (confidence interval) refers to a numerical range with a certain level of confidence, given based on the probability distribution model. It quantitatively represents the risk and fluctuation range of resource prediction, providing a basis for subsequent robust scheduling. For example, if the estimated total resource demand is 100 units, its 95% confidence interval is [85, 115]. This means there is a 95% certainty that the actual resource demand will fall between 85 and 115 units.

[0030] The core function of this step is to achieve accurate and reliable prediction of task resource requirements. It extracts essential task features using deep learning methods, simulates various possible scenarios through random sampling, and ultimately outputs a result containing the predicted value and its fluctuation range. This changes the traditional approach of providing only a fixed predicted value, enabling the scheduling system to anticipate and respond to uncertainties in resource requirements, thus making more robust decisions. For example, for a task with a predicted requirement of 100±15, the scheduler can reserve 115 resources to avoid task failure due to a sudden increase in demand.

[0031] Specifically, the use of a speculative sampling strategy to perform Monte Carlo sampling on multiple possible paths during task execution includes: A random walk model based on a task graph structure is constructed, generating multiple alternative execution paths at each decision node of the task graph based on historical execution data. The random walk model refers to a probabilistic path generation algorithm based on the task graph structure. At each decision point in the task graph, a subsequent path is randomly selected, generating a large number of possible complete execution sequences. For example, in pipe gallery inspection, facing the task chain of detecting leaks → taking photos → reporting results, a random walk might generate: Path A: Detection → High-quality photo → Immediate reporting (high resource requirement). Path B: Detection → Standard photo → Batch reporting after caching (medium resource requirement). Path C: Detection → Low-quality photo → Delayed reporting (low resource requirement).

[0032] For each alternative execution path, the required computing resource consumption and communication overhead are sampled and calculated. The computing resource consumption includes CPU / GPU utilization and memory usage, while the communication overhead includes data transfer volume and network latency.

[0033] Importance sampling techniques are used to weight low-probability paths with high resource consumption, and bootstrap aggregation is employed to integrate the sampling results, thereby improving the robustness of resource demand prediction. Importance sampling refers to a statistical method that weights low-probability but high-impact paths, preventing rare but important execution paths from being ignored and improving the comprehensiveness of the prediction. For example, a robot dog might encounter a severe obstacle requiring emergency obstacle avoidance with a probability of only 5%, but its resource demand is three times that of normal conditions; importance sampling would assign a higher weight to this path. Bootstrap aggregation is a machine learning technique that uses sampling with replacement to build multiple sub-models and integrate the results, reducing the prediction variance caused by sampling randomness and improving robustness. For example, from 100 random walks: Sub-model 1: predicts resource demand based on paths sampled 1-30; Sub-model 2: predicts resource demand based on paths sampled 25-55; ... The final prediction = the average of the predictions from each sub-model.

[0034] Based on the integrated sampling results, a nonparametric probability distribution model of resource demand is established using the kernel density estimation method. The pre-set confidence interval of this distribution is then calculated as the uncertainty interval of resource demand. The kernel density estimation method is a nonparametric probability density function estimation technique that does not assume resource demand follows a specific distribution; it directly learns the distribution shape from the data. The formula is: Where K is the kernel function and h is the bandwidth. These are sampled observations. For example, based on the sampled resource demand data {95,103,88,112,...}, kernel density estimation can fit a probability density curve, rather than simply assuming a normal distribution.

[0035] Step (3) Dynamic Task Scheduling: Based on the total resource requirements and their uncertainty range, a weighted priority model is constructed by combining task urgency and equipment status. Tasks are divided into multiple priority queues, and a reinforcement learning-driven operator selection mechanism is used to generate a collaborative execution sequence between the robot dog and the drone. Based on resource prediction results and equipment status, a priority model is constructed, and reinforcement learning is used to realize the collaborative task allocation between the robot dog and the drone.

[0036] Specifically, in this embodiment, step (3) of constructing the weighted priority model and generating the cooperative execution sequence specifically includes: A multi-objective optimization function is established, comprehensively considering the uncertainty range of resource requirements, task urgency weighting factors, equipment availability state matrix, and inter-task dependency constraints. This multi-objective optimization function formalizes the scheduling problem as a decision model that weighs multiple key factors. Examples include Minimize[total energy consumption, total time consumption, resource waste] and Subjectto[deadline, dependencies, equipment capacity]. Objectives include minimizing total energy consumption, minimizing maximum completion time, and maximizing equipment utilization; these objectives are often conflicting.

[0037] Task prioritization is transformed into a constrained optimization problem, incorporating the following factors through a weighted summation: the normalized value of the mean resource requirement, the width penalty term for the uncertainty interval, the urgency coefficient of the task deadline, and the equipment load balancing index. For example, the total priority score = w1 × mean resource requirement + w2 × uncertainty width + w3 × urgency coefficient + w4 × load balancing. Urgent tasks may be given a higher weight for the urgency coefficient (w3 = 0.6), while routine tasks prioritize resource efficiency (w1 = 0.5). The width penalty term is a negative evaluation term for the width of the uncertainty interval in the optimization function. It encourages the selection of tasks with more certain predictions and lower risks. For example, if task A has a resource requirement of [90, 110] and task B has [80, 120], even with the same mean, task A will receive a higher priority due to its narrower uncertainty interval. The urgency coefficient is a quantitative indicator of urgency calculated based on the task deadline. It ensures that time-sensitive tasks are processed promptly. For example, the urgency coefficient = 1 / (deadline - current time). A task with only 30 minutes left until the deadline has a higher urgency factor than a task with 2 hours left. Load balancing is a metric that measures the evenness of workload distribution across multiple devices, preventing some devices from being overloaded while others are idle. For example, load balancing = 1 - (standard deviation of load for each device / average load). A device with 90% load and device B with 20% load has poor load balancing, while a device with both at 55% load has better load balancing.

[0038] A reinforcement learning algorithm is used to maintain an operator selection policy network. The network's input states include real-time system load characteristics, device collaborative capability evaluation vectors, and task dependency matrices. Its output action space contains various scheduling operators. The policy network is trained using a delayed update mechanism and an optimal computational allocation strategy. At each decision, a scheduling operator is selected based on the ε-greedy policy, dynamically generating a collaborative execution sequence between the robot dog and the drone. The ε-greedy policy is a reinforcement learning action selection strategy that balances exploration and exploitation. New strategies are tried with probability ε (exploration), and the current optimal strategy is selected with probability 1-ε (exploitation). For example, setting ε=0.1 means that the known optimal scheduling operator is selected 90% of the time, and other potentially better operators are randomly tried 10% of the time.

[0039] By establishing an intelligent multi-factor task priority evaluation system and achieving dynamic optimization scheduling decisions through reinforcement learning, it upgrades traditional rule-based scheduling into an intelligent scheduling system capable of adapting to environmental changes and balancing multiple conflicting objectives, ensuring efficient and reliable inspection task execution in complex and uncertain utility tunnel environments.

[0040] Furthermore, the method of maintaining the operator selection policy network using reinforcement learning algorithms includes: A policy network architecture based on Deep Q-Network (DQN) is constructed, where the state space is defined as a six-tuple (current system load rate, device battery level matrix, task resource demand vector, task dependency matrix, queue waiting time series, and environmental uncertainty index). The DQN policy network architecture is implemented using the Q-learning algorithm based on deep neural networks, learning the complex mapping relationship from system state to scheduling operator value. For example, given the current system state as input, the output is the Q-value (expected cumulative reward) of each of the eight scheduling operators. The six-tuple state space describes the six-dimensional feature vectors of the complete system state, providing comprehensive environmental awareness information for reinforcement learning.

[0041] The action space includes eight scheduling operators: task migration, resource reallocation, sequence reorganization, device switching, path replanning, priority boosting, task splitting, and load balancing. These eight scheduling operators represent eight basic scheduling actions that the reinforcement learning agent can choose from, providing a rich variety of scheduling strategy combinations. Specific operators include: Task Migration: Transferring tasks from overloaded devices to idle devices. Resource Reallocation: Dynamically adjusting the CPU / GPU resource ratio allocated to tasks. Sequence Reorganization: Reordering the task execution sequence on the same device. Device Switching: Redistributing tasks between different types of devices (robot dog ↔ drone). Path Replanning: Optimizing the robot dog's mobile inspection path. Priority Boosting: Elevating important tasks to a higher priority queue. Task Splitting: Breaking down large tasks into parallel executable subtasks. Load Balancing: Redistributing task load across multiple devices.

[0042] The design incorporates a composite reward function, comprising the following components: a negative exponential reward for task completion time, a logarithmic reward for resource utilization, a squared reward for device load balancing, a linear reward for task priority satisfaction, and a regularization penalty for scheduling stability. The composite reward function is a weighted evaluation function composed of multiple sub-rewards, guiding the reinforcement learning agent towards multi-objective optimization.

[0043] Training is performed using a dual-Q network structure and an experience replay mechanism. Training stability is improved through delayed updates of the target network and prioritized experience sampling. An ε-decay strategy is used to balance exploration and utilization. The dual-Q network structure and experience replay are two key techniques for improving the training stability of DQN. The dual-Q network uses two networks to reduce the problem of overestimation of Q-values, while experience replay stores and randomly reuses historical experience, breaking data correlation. For example, batches of data (states, actions, rewards, new states) are randomly drawn from the experience pool for training. The ε-decay strategy is a scheduling strategy that gradually reduces the exploration rate over time, allowing for sufficient exploration in the early stages of training and focusing on utilizing learned knowledge in the later stages.

[0044] The core function of the above steps is to build a complete reinforcement learning scheduling decision system. Through carefully designed state space, action space and reward mechanism, the system can learn the optimal scheduling strategy autonomously from historical experience, adapt to the complex and ever-changing working environment of pipeline inspection, and achieve continuous optimization of the collaborative scheduling efficiency of multiple devices.

[0045] Step (4) Adaptive scheduling adjustment: During the operation of the collaborative execution sequence, dynamic time slices are set for each priority queue. When a task execution times out or a low-priority task waits beyond a threshold, preemptive scheduling based on confidence assessment is triggered, and the task queues are reallocated in conjunction with the optimal computational allocation strategy. Through confidence assessment and preemption mechanism, the scheduling strategy is dynamically adjusted during task execution.

[0046] Specifically, in this embodiment, triggering preemptive scheduling based on confidence assessment in step (4) includes: An adaptive dynamic time slice is set for each priority queue. The time slice length is dynamically adjusted based on the average execution time of tasks in the queue and the uncertainty of resource requirements. The adaptive dynamic time slice is a time allocation unit that is dynamically adjusted according to task characteristics and system status, allocating appropriate execution time windows for tasks of different priorities. For example, the time slice for high-priority queues is set to 500ms, and the time slice for low-priority queues is set to 200ms. When the system load is high, the time slices of each queue are automatically reduced.

[0047] Establish a real-time monitoring mechanism for task waiting time. When the waiting time of a low-priority task exceeds its dynamic threshold, a confidence assessment process is initiated. This threshold is calculated based on the task's urgency and historical execution success rate. A monitoring system continuously tracks task waiting time and execution status to promptly detect potential task starvation and timeout issues. For example, if a low-priority task has been waiting for more than 300 seconds, an alert is triggered. The dynamic threshold refers to the upper limit of the waiting time dynamically calculated based on task characteristics and historical data. Intelligently determine when preemptive scheduling needs to be initiated. Threshold = Base waiting time × Urgency coefficient × Historical success rate correction factor. For example, the dynamic threshold for an urgent task might be only 60 seconds, while for a normal task it could reach 600 seconds.

[0048] A Bayesian update method is used to assess the confidence level of a task's execution status, determining the reliability index of task completion by calculating the posterior probability distribution. The Bayesian update method is a probabilistic update mechanism based on Bayes' theorem, continuously updating the confidence level assessment of the task's execution status based on new evidence. Posterior probability = (prior probability × likelihood) / evidence factor. For example, if the initial confidence level of task completion is 90%, after receiving a report of execution delay, it is reduced to 70% through a Bayesian update.

[0049] When the confidence level falls below a preset threshold, preemptive scheduling is triggered. Combined with an optimal computational allocation strategy, the task is reassigned to a higher-priority queue or its execution order is adjusted. Confidence assessment is a quantitative evaluation of the probability of a task's successful completion, providing an objective basis for preemptive decisions. For example, if the confidence threshold is set at 60%, a task below this value is considered to have excessively high risk. Preemptive scheduling is a mechanism that interrupts currently executing tasks to run higher-priority tasks, ensuring that critical tasks are processed promptly. For instance, pausing a routine inspection task to prioritize an emergency leak detection task.

[0050] During the task redistribution process, load balancing and task dependency constraints among devices should be considered to ensure that scheduling changes do not cause system deadlocks or resource conflicts. When redistributing tasks, avoid assigning interdependent tasks to different devices while maintaining load balancing across all devices.

[0051] Establish an intelligent anti-starvation and dynamic priority adjustment mechanism. Through real-time monitoring, confidence assessment, and constrained preemptive scheduling, ensure that the system guarantees timely response to high-priority tasks while also taking into account the basic rights of low-priority tasks, avoiding situations where tasks wait indefinitely, thereby improving the fairness and reliability of the entire inspection system.

[0052] Step (5) Memory Optimization Management: During the task queue reallocation process, the key-value cache status of unexecuted tasks is monitored, and a delayed update mechanism is used to dynamically migrate the KV cache of future unscheduled tasks between GPU and CPU memory. The delayed update mechanism realizes intelligent migration of KV cache between GPU / CPU, improving memory utilization efficiency.

[0053] Specifically, in this embodiment, the dynamic migration in step (5) includes: Establish a task-KV cache mapping table to record in real time the size, access frequency, and estimated scheduling time of the key-value cache corresponding to each unexecuted task. The task-KV cache mapping table refers to a database or data structure that records the correspondence between each unexecuted task and its corresponding key-value cache generated in the larger model. Serving as a ledger for cache management, it provides data support for subsequent migration decisions.

[0054] Based on a reinforcement learning-driven prediction model, the scheduling probability of each task within a future time window is calculated. When a task's scheduling probability is lower than a dynamic threshold, it is marked as a task to be migrated. The reinforcement learning-driven prediction model refers to a machine learning model used to predict the probability of a task being scheduled for execution in the near future. It intelligently identifies which tasks have cached cold data (not needed in the short term) and thus become candidates for migration. Based on system queues, task priorities, and other information, the model predicts that the probability of task T5 being scheduled within the next 60 seconds is less than 10%. The dynamic threshold is a probability threshold that is dynamically adjusted based on the overall system memory pressure. It automatically adapts to different load scenarios, determining the aggressiveness of the migration; for example, when the GPU memory idle rate is >30%, the threshold is set to 5%; when the GPU memory idle rate is <10%, the threshold is set to 20%, meaning more task caches will be moved out.

[0055] A two-stage migration strategy is employed: First, the key-value cache for tagged tasks is asynchronously transferred from GPU global memory to a buffer in CPU memory. Then, based on system memory pressure, a decision is made regarding whether to further compress and store it on a solid-state drive (SSD). This two-stage migration strategy is a step-by-step, gradual cache offloading strategy, striking a balance between freeing up valuable GPU memory and ensuring task restart speed. Stage 1 (GPU→CPU): The cache is asynchronously moved to the slower but larger-capacity CPU memory for rapid recall. Stage 2 (CPU→SSD): If system memory pressure remains high, the cache is further compressed and stored on disk as a last resort. For example, the cache for task T5 is first moved to CPU memory; if system pressure is not alleviated, it is subsequently compressed and stored on an SSD.

[0056] This paper designs an LRU-K-based cache replacement algorithm that comprehensively considers cache access frequency, task priority, and migration cost to dynamically maintain the active cache set in GPU memory. The LRU-K-based cache replacement algorithm is an improved version of the LRU (Least Recently Used) algorithm. It considers not only the time of the most recent access but also the historical frequency of the past K accesses. This more accurately identifies truly cold data and avoids mistakenly deleting cache entries that are about to be accessed. For example, if a task has not been accessed recently but has been frequently accessed in the past, it is not considered cold data and will be retained in the GPU.

[0057] Before a task is about to be executed, its associated key-value cache is reloaded into GPU memory in priority order via a prefetch mechanism to ensure the continuity of task execution. The prefetch mechanism refers to the process of loading the necessary key-value cache back into GPU memory in advance before a task is scheduled to execute. This ensures that tasks can begin execution immediately without waiting when scheduled, avoiding performance degradation due to cache misses. For example, if the scheduler determines that task T5 will execute in 5 seconds, it triggers a prefetch operation, loading its 150MB cache from CPU memory back into GPU memory.

[0058] The core function of the above steps is to implement an intelligent, hierarchical key-value cache lifecycle management system. Through prediction and dynamic decision-making, it ensures that the hottest data remains in the fastest GPU memory, while moving cold data to cheaper, larger-capacity storage tiers as needed, thereby maximizing the throughput of the entire system with limited memory resources.

[0059] The calculation of scheduling probabilities by the reinforcement learning-driven prediction model includes: Construct a state feature vector, which includes task priority weights, resource requirement matching degree, equipment availability indicators, queue waiting time, historical execution success rate, and sensitivity to environmental changes. The state feature vector refers to the multi-dimensional set of features input into the prediction model that describes the current state of a task. It provides the model with all the contextual information needed to make accurate predictions.

[0060] A deep neural network prediction model is designed, which learns the mapping relationship between state features and scheduling probabilities through a three-layer fully connected network. The output layer uses a softmax activation function to generate multi-step scheduling probability predictions. This deep neural network prediction model is a three-layer fully connected neural network used to learn the complex nonlinear mapping from state features to scheduling probabilities. It serves as the core brain for predicting task scheduling probabilities.

[0061] The temporal difference learning algorithm is used to train the prediction model. The reward function includes rewards for scheduling accuracy and resource utilization efficiency, and a prediction stability regularization term is also included. The temporal difference learning algorithm trains the prediction model so that its prediction results not only reflect immediate benefits but also consider long-term benefits. For example, the model not only learns that improving resource utilization can obtain immediate rewards, but also learns that avoiding frequent cache migrations to maintain system stability can bring greater long-term rewards.

[0062] Establish a dynamic threshold adjustment mechanism to automatically adjust the scheduling probability threshold based on the system's real-time load rate and memory usage. When the system load rate exceeds a first threshold (e.g., 75%), a strict threshold policy is activated; when memory usage exceeds a first threshold (e.g., 80%), aggressive cache migration is initiated. A set of rules automatically modifies the migration threshold based on real-time system metrics (load rate, memory usage). This enables the system's behavior to adaptively adjust according to its current health status.

[0063] The predictive model parameters are continuously updated through an online learning mechanism, using the difference between the actual scheduling results and the predicted results as a loss function to continuously improve the accuracy of scheduling probability prediction. During actual system operation, the model continuously updates its parameters using newly generated data (actual scheduling results). This allows the predictive model to adapt to changes in the utility tunnel environment and continuously optimize its predictive accuracy, achieving continuous evolution. For example, new model parameters = old model parameters - learning rate × (predicted probability - 0 / 1 label of whether or not the system was actually scheduled).

[0064] We construct an intelligent predictive core capable of self-evolution and environmental adaptation. This ensures that the decisions of the entire cache management system are not based on static, fixed rules, but on a dynamic model that can continuously learn and adjust from actual operational experience, thereby giving the system true long-term adaptability and robustness.

[0065] Step (6) Continuous System Optimization: Based on the multimodal data transmitted back by the robot dog and drone during execution, the system status is evaluated in real time through a large multimodal model, and the dependencies of the task graph and the weighted priority model are dynamically updated. The task graph and scheduling model are dynamically updated based on real-time feedback data to form a closed-loop optimization system.

[0066] Specifically, in this embodiment, step (6) includes: establishing a multimodal data real-time analysis pipeline to synchronously process the infrared thermal imaging and vibration data returned by the robot dog and the visible light video and laser point cloud returned by the drone; constructing a system health assessment model to calculate the comprehensive system status score based on the device battery power, communication quality indicators, and environmental disturbance factors; dynamically correcting the task graph dependencies through an online learning mechanism, automatically adding or deleting task nodes or adjusting the dependency edge weights when environmental changes or device anomalies are detected; designing a priority feedback closed loop to adjust the weight coefficients in the weighted priority model in reverse according to the deviation between the actual completion time and the expected time of the task and the accuracy of resource estimation; and establishing a simulation verification environment based on digital twins to conduct virtual simulations before implementing the scheduling strategy update and evaluate the expected effect of the updated strategy.

[0067] The system enables autonomous decision-making throughout the entire process, from task generation and resource allocation to execution optimization, solving key technical challenges in traditional inspection systems such as unintelligent task generation, inaccurate resource allocation, and low efficiency of multi-device collaboration.

[0068] Example 2

[0069] like Figure 2 As shown in the figure, this application provides an architecture diagram of a collaborative task autonomous generation and scheduling system for pipe gallery inspection drones based on a multimodal large model. It is applied to the collaborative task autonomous generation and scheduling system for pipe gallery inspection drones based on a multimodal large model as described in Embodiment 1, including: a task graph generation module 210, a resource demand prediction module 220, a task scheduling module 230, a dynamic scheduling adjustment module 240, a cache management module 250, and an online learning optimization module 260.

[0070] The task graph generation module 210 is used to jointly process the text procedures, images and 3D point clouds of the pipeline inspection scenario through a multimodal large model to generate a task graph with dependencies.

[0071] The resource demand prediction module 220 is used to predict the task resource demand based on the task graph by using an autoencoder with embedded LSTM to reduce the dimensionality of the task features and combining a speculative sampling strategy, and output the total resource demand and its uncertainty range.

[0072] The task scheduling module 230 is used to construct a weighted priority model based on the resource requirements and their uncertainty range, combined with the task urgency and equipment status, divide the tasks into multiple priority queues, and generate a cooperative execution sequence between the robot dog and the drone using a reinforcement learning-driven operator selection mechanism.

[0073] The dynamic scheduling adjustment module 240 is used to set dynamic time slices for each priority queue during the operation of the collaborative execution sequence. When the task execution times out or the waiting time of a low-priority task exceeds the threshold, it triggers preemptive scheduling based on confidence evaluation and reallocates the task queues in combination with the optimal computational allocation strategy.

[0074] The cache management module 250 is used to monitor the key-value cache status of unexecuted tasks during the task queue reallocation process, and to dynamically migrate the KV cache of future unscheduled tasks between GPU and CPU memory using a delayed update mechanism.

[0075] The online learning optimization module 260 is used to evaluate the system status in real time through a large multimodal model based on the multimodal data returned by the robot dog and drone during the execution process, and dynamically update the dependency relationship of the task graph and the weighted priority model.

[0076] Figure 3 This is an electronic device provided in one embodiment of this application. For example... Figure 3 As shown, the electronic device includes at least the following components: processor 301 and memory 300, communication interface 303, and bus 302.

[0077] In this embodiment of the application, memory 300 is used to store executable instructions of processor 301, which, when configured to execute instructions, implements the method as described in the first aspect.

[0078] In embodiments of this application, a computer-readable storage medium includes instructions that instruct a device to perform the method as described in the first aspect. For example, the instructions instruct the device to perform... Figure 1 The method is shown in the process steps.

[0079] In one embodiment of this application, the program operating in the electronic device may be a program that controls a central processing unit (CPU) or similar device to achieve the functions of the above-described embodiments of the present invention (a program that enables the computer to function). Information processed by these systems is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (FlashROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.

[0080] It should be noted that a portion of the electronic device described in the above embodiments can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.

[0081] It should be noted that the computer mentioned here refers to a computer built into an electronic device, employing hardware including an operating system and peripheral devices. Furthermore, computer-readable recording media refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage systems such as hard drives built into the computer.

[0082] Furthermore, computer-readable recording media can include: media that dynamically stores programs for short periods of time, such as communication lines used when transmitting programs via networks like the Internet or communication lines like telephone lines; and media that store programs for fixed periods of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining them with programs already recorded in the computer.

[0083] Furthermore, the electronic device in the above embodiments can also be implemented as an assembly (system group) composed of multiple systems. Each system constituting the system group can possess some or all of the functions or functional blocks of the electronic device in the above embodiments. As a system group, it is sufficient to have all the functions or functional blocks of the electronic device.

[0084] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. A method for autonomous generation and scheduling of collaborative tasks for utility tunnel inspection drones based on a multimodal large model, characterized in that, Includes the following steps: By using a multimodal large model, text procedures, images, and 3D point clouds of the utility tunnel inspection scenario are jointly processed to generate a task graph with dependencies. Based on the task graph, an autoencoder with embedded LSTM is used to reduce the dimensionality of the task features, and a speculative sampling strategy is combined to predict the task resource requirements, outputting the total resource requirements and their uncertainty range. Based on the total resource requirements and their uncertainty range, a weighted priority model is constructed by combining task urgency and equipment status. The tasks are divided into multiple priority queues, and a reinforcement learning-driven operator selection mechanism is used to generate a collaborative execution sequence between the robot dog and the drone. During the operation of the collaborative execution sequence, dynamic time slices are set for each priority queue. When a task execution times out or a low-priority task waits for more than a threshold, preemptive scheduling based on confidence assessment is triggered, and the task queues are reallocated in combination with the optimal computational allocation strategy. During the task queue reallocation process, the key-value cache status of unexecuted tasks is monitored, and a delayed update mechanism is used to dynamically migrate the KV cache of future unscheduled tasks between GPU and CPU memory. Based on the multimodal data transmitted back by the robot dog and drone during the execution process, the system status is evaluated in real time through a multimodal large model, and the dependencies of the task graph and the weighted priority model are dynamically updated.

2. The method according to claim 1, characterized in that, The generation of a task graph with dependencies specifically includes: The system extracts equipment status features and meter readings from images using a multimodal large-scale visual encoder, extracts spatial geometric features and deformation data of the pipe gallery structure using a point cloud encoder, and parses operational constraints and safety standards in the inspection procedures using a text encoder. Multimodal features are aligned across modalities and weighted by attention in the fusion layer to identify temporal dependencies, spatial proximity relationships, and logical causal relationships between tasks; Based on the identified dependencies, a task graph structure is constructed with inspection tasks as nodes and dependencies as directed edges. The node attributes include task type, target location, and expected time, while the edge attributes include dependency type and predecessor strength.

3. The method according to claim 1, characterized in that, The resource requirements for the prediction task specifically include: The node and edge features of the task graph are input into the autoencoder embedded in the LSTM. The temporal dependencies between tasks are captured through the LSTM layer, and the high-dimensional task features are mapped to the low-dimensional latent space through the encoder-decoder structure. In the low-dimensional potential space, a speculative sampling strategy is used to perform Monte Carlo sampling on multiple possible paths in the task execution process to predict the computational resource consumption and communication overhead for each path. Based on the sampling results, a probability distribution model of resource demand is established. By calculating the mean and variance of this distribution, the estimated total resource demand of the task and its uncertainty interval are output, where the uncertainty interval is expressed in the form of a confidence interval.

4. The method according to claim 3, characterized in that, The method of using a speculative sampling strategy to perform Monte Carlo sampling on multiple possible paths during task execution includes: Construct a random walk model based on a task graph structure, and generate multiple alternative execution paths at each decision node of the task graph based on historical execution data; For each alternative execution path, the required computing resource consumption and communication overhead are sampled and calculated. The computing resource consumption includes CPU / GPU utilization and memory usage, and the communication overhead includes data transfer volume and network latency. We use importance sampling techniques to weight low-probability paths with high resource consumption, and then use a bootstrap aggregation method to integrate the sampling results. Based on the integrated sampling results, a nonparametric probability distribution model of resource demand is established using the kernel density estimation method. The pre-set confidence interval of this distribution is then used as the output of the uncertainty interval of resource demand.

5. The method according to claim 1, characterized in that, The construction of the weighted priority model and the generation of the cooperative execution sequence specifically include: Establish a multi-objective optimization function that comprehensively considers the uncertainty range of resource requirements, the task urgency weight factor, the equipment availability state matrix, and the constraints of inter-task dependencies. The task priority calculation is transformed into a constrained optimization problem, and the following factors are incorporated by weighted summation: the normalized value of the mean resource requirement, the width penalty term of the uncertainty interval, the urgency coefficient of the task deadline, and the equipment load balancing index. A reinforcement learning algorithm is used to maintain an operator selection policy network. The input state of the network includes the real-time load characteristics of the system, the evaluation vector of the device coordination capability, and the task dependency matrix. The output action space contains a variety of scheduling operators. The policy network is trained using a delayed update mechanism and an optimal computational allocation strategy. At each decision, a scheduling operator is selected based on the ε-greedy policy to dynamically generate a cooperative execution sequence between the robot dog and the drone.

6. The method according to claim 5, characterized in that, The network for maintaining the operator selection strategy using reinforcement learning algorithms includes: Construct a policy network architecture based on a deep Q-network, where the state space is defined as a six-tuple; Construct an action space containing eight scheduling operators; A composite reward function is designed, and a double-Q network structure and an experience replay mechanism are used for training. The training stability is improved by delaying the target network update and prioritizing experience sampling. An ε-decay strategy is used to balance exploration and exploitation.

7. The method according to claim 1, characterized in that, The triggering of preemptive scheduling based on confidence assessment specifically includes: Each priority queue is assigned an adaptive dynamic time slice, the length of which is dynamically adjusted based on the average execution time and resource requirement uncertainty of the tasks in the queue. Establish a real-time monitoring mechanism for task waiting time. When the waiting time of a low-priority task exceeds its dynamic threshold, initiate a confidence assessment process. This threshold is calculated based on the task's urgency and historical execution success rate. The Bayesian update method is used to evaluate the confidence of the task's execution status, and the reliability index of task completion is determined by calculating the posterior probability distribution. When the confidence level is lower than the preset threshold, preemptive scheduling is triggered. Combined with the optimal computational allocation strategy, the task is reassigned to a higher priority queue or its execution order is adjusted. During the reallocation process, load balancing and task dependency constraints among devices are considered to ensure that scheduling changes do not cause system deadlocks or resource conflicts.

8. The method according to claim 7, characterized in that, The dynamic migration specifically includes: Establish a task-KV cache mapping table to record the key-value cache size, access frequency, and estimated scheduling time for each unexecuted task in real time. Based on a reinforcement learning-driven prediction model, the scheduling probability of each task within a future time window is calculated. When the scheduling probability of a task is lower than a dynamic threshold, it is marked as a task to be transferred. A two-stage migration strategy is adopted: first, the KV cache of the marking task is asynchronously transferred from the GPU global memory to the CPU memory buffer, and then the system memory pressure determines whether to further compress and store it to the solid-state drive. Design an LRU-K-based cache replacement algorithm that comprehensively considers cache access frequency, task priority, and migration cost, and dynamically maintains the active cache set in GPU memory; Before a task is about to be executed, its associated key-value cache is reloaded into GPU memory in priority order through a prefetching mechanism to ensure the continuity of task execution.

9. The method according to claim 8, characterized in that, The reinforcement learning-driven prediction model calculates the scheduling probability of each task within a future time window, including: Construct a state feature vector, which includes task priority weight, resource requirement matching degree, device availability index, queue waiting time, historical execution success rate, and sensitivity to environmental changes; A deep neural network prediction model is designed. The mapping relationship between state features and scheduling probability is learned through a three-layer fully connected network. The output layer uses the softmax activation function to generate multi-step scheduling probability prediction. The prediction model is trained using a temporal difference learning algorithm. The reward function includes a scheduling accuracy reward and a resource utilization efficiency reward, and a prediction stability regularization term is also added. Establish a dynamic threshold adjustment mechanism to automatically adjust the scheduling probability threshold based on the real-time system load rate and memory usage. When the system load rate is higher than the first prediction threshold, a strict threshold strategy is enabled, and when the memory usage rate exceeds the second prediction threshold, an aggressive cache migration is initiated. By continuously updating the prediction model parameters through an online learning mechanism and using the difference between the actual scheduling results and the prediction results as a loss function, the accuracy of scheduling probability prediction is continuously improved.

10. A system for autonomous generation and scheduling of collaborative tasks for pipe gallery inspection drones based on a multimodal large model, applied to the method for autonomous generation and scheduling of collaborative tasks for pipe gallery inspection drones based on a multimodal large model as described in any one of claims 1 to 9, characterized in that, The system includes: The task graph generation module is used to jointly process text procedures, images, and 3D point clouds of the utility tunnel inspection scenario using a multimodal large model to generate a task graph with dependencies. The resource demand prediction module is used to predict the task resource demand based on the task graph by using an autoencoder with embedded LSTM to reduce the dimensionality of the task features and combining it with a speculative sampling strategy, and output the total resource demand and its uncertainty range. The task scheduling module is used to construct a weighted priority model based on the resource requirements and their uncertainty range, combined with the task urgency and equipment status, divide the tasks into multiple priority queues, and generate a cooperative execution sequence between the robot dog and the drone using a reinforcement learning-driven operator selection mechanism. The dynamic scheduling and adjustment module is used to set dynamic time slices for each priority queue during the operation of the collaborative execution sequence. When the task execution times out or the waiting time of a low-priority task exceeds the threshold, it triggers preemptive scheduling based on confidence evaluation and reallocates the task queues in combination with the optimal computational allocation strategy. The cache management module is used to monitor the key-value cache status of unexecuted tasks during the task queue reallocation process, and to dynamically migrate the KV cache of future unscheduled tasks between GPU and CPU memory using a delayed update mechanism. The online learning optimization module is used to evaluate the system status in real time through a large multimodal model based on the multimodal data returned by the robot dog and drone during the execution process, and dynamically update the dependency relationship of the task graph and the weighted priority model.

Images