Low-altitude robot-oriented end-side large model inference method and system

Abstract

The application discloses an end-side large model inference method and system for low-altitude robots, and particularly relates to the technical field of edge computing, which comprises the following steps: collecting the computing resources, communication parameters and target motion and occlusion data of the low-altitude robot, and fusing them into a system state vector; inferring the probability of hidden states such as target position offset, confidence noise and link interruption based on the vector; dynamically adjusting the pruning ratio, token reservation number and end-cloud split point of the end-side large model according to the hidden states; monitoring the request rate, and enabling nonlinear damping adjustment of the queuing priority when the request rate exceeds a threshold; finally, performing counterfactual reasoning according to the actual response delay and tracking success rate, calculating the causal utility value and updating the adjustment strategy; the application can effectively avoid resource deadlock and cascading avalanche, improve the multi-target tracking success rate, and realize long-term self-adaptation to the dynamic environment by dynamically adjusting the inference granularity and end-cloud split, suppressing the request storm through nonlinear damping, and utilizing counterfactual causal feedback closed-loop optimization strategy.

Description

Technical Field

[0001] This invention relates to the field of edge computing technology, and more specifically, to an edge-side large model inference method and system for low-altitude robots. Background Technology

[0002] Low-altitude robots, such as multi-rotor drones, typically need to process multimodal data, including visual images and point clouds, in real time on the onboard device when autonomously performing tasks such as target tracking, power line inspection, and emergency rescue. However, due to limitations in the weight, power consumption, and heat dissipation of the onboard computing platform, running large-scale deep learning models directly on the device faces challenges such as insufficient computing resources and excessive inference latency. To address this, existing technologies have developed edge-cloud collaborative inference architectures: the low-altitude robot performs preliminary processing of the collected data using a lightweight onboard model, and then uploads intermediate features or keyframes to a large cloud model via a wireless communication link for refined inference, thus balancing real-time response on the device with high-precision recognition in the cloud.

[0003] In actual flight conditions, the communication link quality of low-altitude robots fluctuates drastically due to obstructions from buildings, mountains, or electromagnetic interference. Simultaneously, the movement of tracked targets often exhibits nonlinear characteristics such as sudden acceleration and sharp turns. Furthermore, partial target occlusion caused by environmental factors like tree shade and smoke leads to frequent oscillations in the detection confidence output of the lightweight end-side model. When these adverse factors combine, existing end-to-cloud collaborative systems are prone to a typical and difficult-to-cure problem: uncontrolled growth in the request transmission rate. Specifically, a target's short-term occlusion causes the end-side confidence to repeatedly exceed the upload threshold, resulting in a continuous transmission of a large number of low-quality intermediate features or keyframes to the cloud within seconds, quickly consuming the limited uplink bandwidth. This behavior not only drastically increases the actual tracking delay of the target but also blocks requests from other normal targets in the same communication channel, triggering computationally intensive processes such as simultaneous re-identification of multiple targets in the cloud. Ultimately, this leads to a surge in overall system response latency, a sharp drop in tracking success rate, and even mission failure. Therefore, this invention proposes an end-side large-model inference method for low-altitude robots to address the aforementioned problems. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides the following technical solution: An end-side large-scale model inference method for low-altitude robots includes the following steps: The system collects data on the onboard computing resource utilization rate of the low-altitude robot, the available bandwidth and communication latency of the communication link, as well as the movement speed and occlusion ratio of the tracked target, and fuses the collected data into a system state vector. Based on the system state vector, the probability of occurrence of concealed states that cannot be directly measured is calculated using a probabilistic inference method. Concealed states include the target's true position offset, confidence measurement noise, and the probability of communication link interruption. Based on the probability of the occurrence of the concealed state, the inference granularity parameters of the edge-side large model and the computation task splitting point between the low-altitude robot and the cloud are dynamically determined. The inference granularity parameters include the model pruning ratio and the number of tokens retained. During the dynamic determination process, the request sending rate changes of multiple tracked targets or multiple low-altitude robots are monitored; when the increase in the request sending rate is detected to exceed the preset rate increase threshold, a nonlinear damping mechanism is activated to adjust the queuing priority of the requests. Based on the response delay and target tracking success rate generated by the system after the actual execution of dynamic determination and nonlinear damping mechanism, counterfactual reasoning is performed: the actual response is compared with the virtual response that might have been generated if the current dynamic determination had not been executed, the causal utility value of the current dynamic determination is calculated, and the adjustment strategy of the inference granularity parameter is updated using the causal utility value.

[0005] In a preferred embodiment, the step of fusing the collected data into a system state vector specifically includes: The onboard computing resource utilization, available bandwidth of the communication link, communication latency of the communication link, movement speed of the tracked target, and occlusion ratio of the tracked target are linearly normalized so that the value of each component is mapped to the interval between zero and one. Then, they are arranged into a five-dimensional vector according to a preset order. This five-dimensional vector is the system state vector.

[0006] In a preferred embodiment, the step of calculating the probability of occurrence of a concealed state that cannot be directly measured using a probabilistic inference method includes the following calculation process: The first step is to use the speed of the tracked target to make a one-step linear prediction of the target's current position, thus obtaining the predicted position; By using the occlusion ratio of the tracked target, the absolute value of the difference between the observed position and the predicted position directly measured by the sensor is corrected. The absolute value of the difference is multiplied by one minus the difference in the occlusion ratio of the tracked target to obtain the observation error value. The second step is to set the prior probability distribution of the target's true position offset as a normal distribution, with the expected value of the normal distribution being zero and the variance being a preset value. The third step is to update the posterior probability of the target's true position offset based on the observation error value and the onboard computing resource utilization rate. The calculation formula is as follows: The posterior probability value = the function value of the probability density function of the prior probability distribution at the observation error value multiplied by the preset scaling factor, and then divided by the preset normalization constant. The fourth step involves using the same computational logic as the first to third steps to calculate the posterior probability of the confidence measurement noise and the posterior probability of the communication link interruption. When calculating the posterior probability of the confidence measurement noise, the variance of the target detection confidence output by the large end-side model at three consecutive time points is used as the observation input. When calculating the posterior probability of the communication link interruption, the available bandwidth and communication delay of the communication link are used to calculate the rate of change of available bandwidth and the rate of change of communication delay within two consecutive time windows, and the weighted sum of the rate of change of available bandwidth and the rate of change of communication delay is used as the observation input.

[0007] In a preferred embodiment, the steps for dynamically determining the inference granularity parameters of the edge-side large model and the computational task splitting point between the low-altitude robot and the cloud are as follows: The posterior probability of the target's true position offset in the probability of the concealed state is compared with a first preset threshold: When the posterior probability of the target's true location offset is greater than the first preset threshold, increase the model pruning ratio and reduce the number of tokens retained. When the posterior probability of the target's true location offset is less than the first preset threshold, reduce the model pruning ratio and increase the number of tokens retained. When the posterior probability of the target's true location offset is equal to the first preset threshold, the current model pruning ratio and the current number of tokens retained remain unchanged. The increase or decrease is determined by multiplying the difference between the posterior probability of the target's true position offset and the first preset threshold by a preset step size coefficient. The probability of communication link interruption in the probability of the occurrence of the concealed state is compared with a second preset threshold: When the probability of communication link interruption is greater than the second preset threshold, the calculation task splitting point will be adjusted so that all calculation tasks are executed on the end side of the low-altitude robot. When the probability of communication link interruption is less than the second preset threshold, the calculation task splitting point is adjusted so that the first part of the calculation task is executed on the end side of the low-altitude robot and the second part of the calculation task is executed in the cloud. When the probability of communication link interruption is equal to the second preset threshold, the current computing task split point remains unchanged.

[0008] In a preferred embodiment, the preset rate increase threshold is a pre-stored value representing the maximum allowed increase in the number of request transmissions per unit time; the specific method for detecting that the increase in the request transmission rate exceeds the preset rate increase threshold is as follows: The average request sending rate within the first time window before the current time is recorded as the first rate reference value, and the average request sending rate within the second time window before the current time is recorded as the second rate reference value, wherein the duration of the second time window is shorter than the duration of the first time window; Calculate the difference between the second rate reference value and the first rate reference value. When the difference is greater than the preset rate growth threshold, it is determined that the increase in the request sending rate exceeds the preset rate growth threshold.

[0009] In a preferred embodiment, the specific implementation of enabling the nonlinear damping mechanism to adjust the queuing priority of requests is as follows: Assign a dynamic priority value to each tracked target, with the initial value of the dynamic priority value being a preset value; Whenever the increase in the request sending rate for the same tracked target exceeds the preset rate increase threshold, the dynamic priority value of the tracked target is reduced by a preset step value. In the request queuing queue, the requests of each tracked target are sorted in descending order of dynamic priority value, with requests with lower dynamic priority values ​​placed at the back of the request queuing queue.

[0010] In a preferred embodiment, the step of performing counterfactual reasoning and calculating causal utility values ​​includes the following calculation process: Record the first real value of the response delay generated by the system after the actual execution of the dynamic determination steps and the second real value of the target tracking success rate; A virtual scene is constructed and it is assumed that the adjustment actions in the dynamic determination step are not executed in the virtual scene. Instead, the inference granularity parameters and calculation task split points of the previous moment are used. The subsequent input data with the same real environment is used for simulation calculation to obtain the virtual response delay value and the virtual target tracking success rate value. The difference between the first real value and the virtual response delay value is calculated as the delay change, and the difference between the second real value and the virtual target tracking success rate value is calculated as the success rate change. The delayed change is transformed into a positive indicator that contributes in the same direction as the success rate change, and then the two are weighted and summed to generate a causal utility value.

[0011] In a preferred embodiment, the adjustment strategy of updating the inference granularity parameter using the causal utility value is specifically implemented as follows: When the causal utility value is positive, increase the first preset threshold and decrease the second preset threshold; When the causal utility value is negative, decrease the first preset threshold and increase the second preset threshold; The increase or decrease is equal to the absolute value of the causal utility value multiplied by a preset adjustment factor, and then superimposed on the initial values ​​of the first preset threshold and the second preset threshold. When the causal utility value is zero, no update operation is performed, and the updated inference granularity parameter is limited to the corresponding preset value range.

[0012] In a preferred embodiment, an end-side large model inference system for low-altitude robots includes: The data acquisition and fusion module is used to collect the onboard computing resource utilization rate of the low-altitude robot, the available bandwidth and communication latency of the communication link, as well as the movement speed and occlusion ratio of the tracked target, and fuse the collected data into a system state vector. The probability inference module is used to calculate the probability of occurrence of concealed states that cannot be directly measured based on the system state vector using a probability inference method. Concealed states include the target's true position offset, confidence measurement noise, and the probability of communication link interruption. The dynamic determination module is used to dynamically determine the inference granularity parameters of the edge-side large model and the computation task splitting point between the low-altitude robot and the cloud based on the probability of the occurrence of the concealed state. The inference granularity parameters include the model pruning ratio and the number of tokens retained. The damping control module is used to monitor the changes in the request sending rate of multiple tracked targets or multiple low-altitude robots during the dynamic determination process; when the increase in the request sending rate is detected to exceed the preset rate increase threshold, a nonlinear damping mechanism is activated to adjust the queuing priority of the requests. The counterfactual reasoning module is used to perform counterfactual reasoning based on the response delay and target tracking success rate generated by the system after the actual execution of dynamic determination and nonlinear damping mechanism: compare the real response with the virtual response that might have been generated if the current dynamic determination had not been executed, calculate the causal utility value of the current dynamic determination, and use the causal utility value to update the adjustment strategy of the inference granularity parameter.

[0013] The technical effects and advantages of this invention are as follows: This invention, upon detecting that the request sending rate of multiple tracked targets or multiple low-altitude robots increases by more than a preset rate increase threshold, promptly activates a nonlinear damping mechanism to adjust the queuing priority of requests. This mechanism does not rely on static judgments of the authenticity of request content, but rather proactively breaks the positive feedback chain by reducing the queuing priority of abnormally high-volume targets, preventing invalid requests from a single target from consuming communication bandwidth. Compared to existing solutions that use fixed-threshold uploads, this invention can quickly smooth out abnormal spikes in request rates under sudden conditions of severe communication link fluctuations or brief target obstruction, thereby maintaining effective uplink throughput and avoiding resource deadlock caused by queue congestion and retransmissions.

[0014] This invention dynamically adjusts the model pruning ratio and the number of tokens retained based on the posterior probability of the inferred true target position offset. Simultaneously, it dynamically adjusts the task splitting point for edge computing based on the posterior probability of communication link interruption. When the target maneuvers drastically or is severely occluded, the pruning ratio is automatically increased and the number of tokens reduced to decrease edge-side computation and compress uplink data. When communication quality deteriorates, more computational tasks are automatically migrated to the edge. This joint adjustment based on the probability of concealed states ensures that the system always operates at the optimal operating point allowed by current resource constraints while maintaining tracking accuracy. Compared to fixed splitting or single threshold switching schemes, this invention significantly reduces target loss rate and improves global tracking success rate in complex tasks with multiple targets and multiple interferences.

[0015] This invention records the actual response latency and target tracking success rate after each adjustment action, and constructs a virtual scenario—assuming no current adjustment was performed but the inference granularity parameters and split points from the previous moment are still used—to perform simulation calculations, obtaining virtual response latency and virtual success rate, and then calculating a causal utility value. This utility value clearly distinguishes whether the performance improvement comes from the effectiveness of the adjustment action or from accidental changes in the external environment. When the causal utility value is positive, a first preset threshold is increased and a second preset threshold is decreased, causing the system to evolve towards a better threshold; when it is negative, the adjustment is reversed. This adaptive mechanism based on causal inference avoids the large amount of random trial and error in traditional reinforcement learning, enabling learning while executing in actual flight missions, allowing the adjustment strategy to continuously converge to the optimal configuration, significantly enhancing the generalization ability and long-term robustness of this invention across different aircraft types and geographical environments. Attached Figure Description

[0016] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings; Figure 1 This is a schematic diagram of the end-side large model inference method for low-altitude robots in this invention.

[0017] Figure 2 This is a schematic diagram of the end-side large model inference system for low-altitude robots in this invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] Reference Figure 1-2 The following examples were obtained: Example 1: A large-scale end-side inference method for low-altitude robots, comprising the following steps: The system collects data on the onboard computing resource utilization rate of the low-altitude robot, the available bandwidth and communication latency of the communication link, as well as the movement speed and occlusion ratio of the tracked target, and fuses the collected data into a system state vector. Based on the system state vector, the probability of occurrence of concealed states that cannot be directly measured is calculated using a probabilistic inference method. Concealed states include the target's true position offset, confidence measurement noise, and the probability of communication link interruption. Based on the probability of the occurrence of the concealed state, the inference granularity parameters of the edge-side large model and the computation task splitting point between the low-altitude robot and the cloud are dynamically determined. The inference granularity parameters include the model pruning ratio and the number of tokens retained. During the dynamic determination process, the request sending rate changes of multiple tracked targets or multiple low-altitude robots are monitored; when the increase in the request sending rate is detected to exceed the preset rate increase threshold, a nonlinear damping mechanism is activated to adjust the queuing priority of the requests. Based on the response delay and target tracking success rate generated by the system after the actual execution of dynamic determination and nonlinear damping mechanism, counterfactual reasoning is performed: the actual response is compared with the virtual response that might have been generated if the current dynamic determination had not been executed, the causal utility value of the current dynamic determination is calculated, and the adjustment strategy of the inference granularity parameter is updated using the causal utility value.

[0020] The specific steps involved in fusing the collected data into a system state vector include: First, the onboard computing resource utilization rate is obtained. This utilization rate is the ratio of the resources used by the low-altitude robot processor to the total resources at the current moment, ranging from zero to one. A typical value, such as 0.3, indicates 30% utilization. Simultaneously, the available bandwidth of the communication link is obtained, measured in megabits per second (Mbps) by the wireless communication module in real time, typically ranging from one to one hundred Mbps. Next, the communication latency of the communication link is obtained, measured in milliseconds, the round-trip time from sending to receiving a data packet, typically ranging from ten to five hundred milliseconds. Then, the movement speed of the tracked target is obtained, calculated by fusion of visual odometry and inertial measurement unit (IMU), measured in meters per second, typically ranging from zero to thirty meters per second. Finally, the occlusion ratio of the tracked target is obtained, the proportion of the target's visible area obscured by obstacles, output by a deep learning model, ranging from zero to one, with a typical value of 0.4 indicating 40% occlusion.

[0021] The five original values ​​are linearly normalized. The normalization formula is (original value minus minimum value) divided by (maximum value minus minimum value), where the minimum and maximum values ​​are preset based on experimental data or the equipment manual. For airborne computing resource utilization, the minimum value is set to zero and the maximum value to one, so the normalized value is the original value itself. For available bandwidth, the minimum value is set to 0 megabits per second as determined by the equipment manual, and the maximum value is set to 100 megabits per second as determined by the equipment manual, resulting in a bandwidth normalized value between zero and one. For communication latency, the minimum value is set to 10 milliseconds as determined by statistical results, and the maximum value is set to 500 milliseconds as determined by statistical results, resulting in a latency normalized value. Note that the smaller the latency, the closer the normalized value is to zero. For movement speed, the minimum value is set to 0 meters per second, and the maximum value is set to 30 meters per second as determined by the maximum tracking capability of the UAV, resulting in a speed normalized value. For occlusion ratio, the minimum value is set to zero and the maximum value is set to one, so the normalized value is the original occlusion ratio.

[0022] The five normalized values ​​are arranged into a five-dimensional vector in a fixed preset order. This preset order is as follows: normalized value of onboard computing resource utilization, normalized value of available bandwidth, normalized value of communication latency, normalized value of motion speed, and normalized value of occlusion ratio. The resulting five-dimensional vector is the system state vector, which fully describes the current state of the low-altitude robot's computing resources, communication status, and the combined situation of target motion and environmental occlusion.

[0023] Based on the system state vector, a probabilistic inference method is used to calculate the probability of occurrence of concealed states that cannot be directly measured. Concealed states include the target's true position offset, confidence measurement noise, and the probability of communication link interruption. These three states cannot be directly obtained by sensors and require probabilistic inference based on existing observations. The first step is to perform a one-step linear prediction of the target's current position using its velocity, thus obtaining the predicted position. One-step linear prediction refers to calculating the estimated position of the target at the next moment (i.e., the future step) based on its current position and velocity, following a uniform linear motion model. The velocity is the original velocity value corresponding to the normalized velocity value in the system state vector. For example, if the current velocity is ten meters per second and the target position at the previous moment was (100, 200), then the predicted position is (100 plus ten multiplied by the sampling interval, 200). The sampling interval is determined based on the visual sensor's frame rate, typically 0.05 seconds. The absolute value of the difference between the observed position directly measured by the sensor and the predicted position is corrected using the occlusion ratio of the tracked target. This absolute value is multiplied by one and subtracted from the difference in the occlusion ratio of the tracked target to obtain the observation error value. For example, if the occlusion ratio is 0.3, then 1 minus 0.3 equals 0.7. If the absolute value of the difference between the observed position and the predicted position is 2 meters, then the observation error value is 1.4 meters. The higher the occlusion ratio, the smaller the corrected observation error value, indicating a lower level of confidence in the sensor measurement.

[0024] The second step is to set the prior probability distribution of the target's true position offset as a normal distribution. The expected value of the normal distribution is zero, and the variance is a preset value. The preset value is determined based on the historical statistical error of the low-altitude robot positioning system. The typical value is 0.5 square meters, which means that in the absence of observation information, the true position offset has a 68% probability of falling within the range of ±0.7 meters.

[0025] The third step involves updating the posterior probability of the target's true position offset based on the observation error value and the onboard computing resource occupancy rate. The calculation formula is: Posterior probability value = (Probability density function of the prior probability distribution at the observation error value) multiplied by a preset scaling factor, then divided by a preset normalization constant. The probability density function of the prior probability distribution follows a normal distribution formula, and substituting the observation error value yields a positive number. The preset scaling factor is set according to the onboard computing resource occupancy rate. For example, when the onboard computing resource occupancy rate is 0.2 (relatively idle), the preset scaling factor is 0.8; when the occupancy rate is 0.8 (relatively strained), it is 0.2. The two are linearly negatively correlated, and the specific value is determined based on experimental data. The preset normalization constant is a scaling factor that makes the sum of all possible posterior probabilities equal to one, usually obtained through numerical integration.

[0026] The fourth step uses the same computational logic as steps one through three to calculate the posterior probability of the confidence measurement noise and the posterior probability of the communication link interruption. When calculating the posterior probability of the confidence measurement noise, the variance of the target detection confidence output from the large end-side model at three consecutive time points is used as the observation input. For example, if the confidence output at three consecutive time points is 0.9, 0.7, and 0.8, its variance is calculated to a certain value; the larger the variance, the greater the confidence measurement noise, and the posterior probability reflects the likelihood of the current noise level. When calculating the posterior probability of the communication link interruption, the available bandwidth and communication delay of the communication link are used to calculate the rate of change of available bandwidth and the rate of change of communication delay within two consecutive time windows, and the weighted sum of these two rates of change is used as the observation input.

[0027] The time window length is set according to the sampling period of the communication module. For example, the first time window length is two seconds, and the second time window length is 0.5 seconds. The available bandwidth change rate is (the average bandwidth of the current window minus the average bandwidth of the previous window) divided by the average bandwidth of the previous window. The communication delay change rate is calculated similarly. The weight coefficients in the weighted sum are set according to empirical values. For example, the bandwidth change rate weight is 0.6, and the delay change rate weight is 0.4. The sum of the two yields the observed input value, which is used to update the posterior probability of the communication link interruption probability.

[0028] The specific steps for dynamically determining the inference granularity parameters of the large-scale model on the edge and the computational task splitting points between the low-altitude robot and the cloud are as follows: The posterior probability of the target's true position offset in the probability of the concealed state is compared with a first preset threshold. The first preset threshold is set based on experimental data, with a typical value of 0.5, indicating that when the posterior probability exceeds half, the target position offset is considered large, and the model inference granularity needs to be adjusted. This threshold can also be adjusted according to the tracking accuracy requirements of the low-altitude robot in the equipment manual, with a value range between 0.3 and 0.7.

[0029] When the posterior probability of the target's true location offset is greater than a first preset threshold, the model pruning ratio is increased and the number of tokens retained is decreased. For example, if the posterior probability is 0.7, which is greater than the first preset threshold of 0.5, the model pruning ratio increases from 0.2 to 0.3, and the number of tokens retained decreases from one hundred to seventy. The model pruning ratio refers to the proportion of redundant channels or neurons pruned in the large end-side model, and the number of tokens retained refers to the number of image patches retained in the visual Transformer model.

[0030] When the posterior probability of the target's true location offset is less than a first preset threshold, the model pruning ratio is reduced and the number of tokens retained is increased. For example, if the posterior probability is 0.3, which is less than 0.5, the model pruning ratio is reduced from 0.2 to 0.1, and the number of tokens retained is increased from 100 to 120.

[0031] When the posterior probability of the target's true location offset equals the first preset threshold, the current model pruning ratio and the current number of tokens retained remain unchanged. For example, if the posterior probability is exactly 0.5, the model pruning ratio remains unchanged at 0.2, and the number of tokens retained remains unchanged at 100.

[0032] The increase or decrease is determined by multiplying the difference between the posterior probability of the target's true position offset and a first preset threshold by a preset step size coefficient. The preset step size coefficient is set based on experience, typically 0.5. For example, if the difference is 0.2 (0.7 minus 0.5), multiplying by the step size coefficient of 0.5 gives 0.1, thus increasing the model pruning ratio by 0.1 (from 0.2 to 0.3). Similarly, the decrease in the number of tokens retained is calculated by multiplying the difference by the step size coefficient and then by the original number of tokens. For example, if the original number of tokens is one hundred, the decrease is two hundred multiplied by 0.1, resulting in twenty tokens, meaning a reduction of twenty tokens.

[0033] The probability of communication link interruption in the probability of occurrence of the concealed state is compared with the second preset threshold. The second preset threshold is set according to the statistical results of the communication module. The typical value is 0.3, which means that when the interruption probability exceeds 30%, the link is considered unreliable and all computing tasks need to be moved to the end side. The value of this threshold is between 0.2 and 0.5.

[0034] When the probability of communication link interruption exceeds the second preset threshold, the computation task splitting point is adjusted so that all computation tasks are executed on the edge of the low-altitude robot. For example, if the probability of communication link interruption is 0.4, which is greater than the second preset threshold of 0.3, then all layers of the large model on the edge are computed locally on the low-altitude robot, and the cloud does not participate in any inference.

[0035] When the probability of communication link interruption is less than a second preset threshold, the computation task splitting point is adjusted so that the first part of the computation task is executed on the edge of the low-altitude robot, and the second part of the computation task is executed in the cloud. For example, if the interruption probability is 0.1, which is less than 0.3, then the first few layers of the model (such as the three Transformer blocks) are executed on the edge, and the subsequent layers are executed in the cloud. The specific location of the splitting point is preset according to the edge computing power and communication bandwidth. Common splitting points are located at one-third or half of the total number of layers in the model.

[0036] When the probability of communication link interruption is equal to the second preset threshold, the current computing task splitting point remains unchanged. For example, if the interruption probability is exactly 0.3, the existing end-to-cloud task allocation method will not be changed.

[0037] The first part of the computational task and the second part of the computational task refer to: The computation process of the large-scale edge model is divided into two consecutive segments according to the model's hierarchical structure or computational modules. The first part of the computation task involves the first few layers of the model (e.g., the first three Transformer coding blocks), executed on the edge processor of the low-altitude robot to complete preliminary feature extraction. The second part of the computation task involves the remaining layers of the model (e.g., the last nine Transformer coding blocks). The output of the first part is uploaded to the cloud server via a communication link for execution, completing the final inference. The two parts together constitute the complete edge-side large-scale model computation flow. The location of the split point is determined by the probability of communication link interruption: when the interruption probability is low (good network), the split point is earlier, with a lighter burden on the edge and a heavier burden on the cloud; when the interruption probability is high, the split point moves towards the end of the model until all parts are executed on the edge. A typical split point is: one-third of the total number of model layers is executed on the edge, and two-thirds are executed on the cloud.

[0038] The preset rate increase threshold is a pre-stored value that represents the maximum allowable increase in the number of requests sent per unit time. This threshold is obtained based on historical data of the low-altitude robot under normal tracking conditions. For example, it is the 95th percentile of the change in the request sending rate in 100 tasks. The typical value is 5 times per second, and the range is from 2 times per second to 10 times per second.

[0039] The specific method for detecting an increase in the request sending rate exceeding a preset rate increase threshold is as follows: The average request sending rate within a first time window prior to the current moment is recorded as a first rate reference value. Simultaneously, the average request sending rate within a second time window prior to the current moment is recorded as a second rate reference value. The duration of the second time window is shorter than that of the first time window. The length of the first time window is set to 10 seconds based on experience to smooth out long-term fluctuations; the length of the second time window is set to 1 second to capture short-term sudden changes.

[0040] The difference between the second rate reference value and the first rate reference value is calculated. When this difference is greater than a preset rate increase threshold, it is determined that the increase in the request sending rate exceeds the preset rate increase threshold. For example, if the average request sending rate in the 10 seconds before the current time is 20 times per second, and the average request sending rate in the 1 second before the current time is 28 times per second, then the difference is 8 times per second, which is greater than the preset rate increase threshold of 5 times per second. Therefore, it is determined that the threshold is exceeded. If the difference is 3 times per second, then it is not exceeded.

[0041] The specific implementation method for adjusting the queuing priority of requests by enabling the nonlinear damping mechanism is as follows: Each tracked target is assigned a dynamic priority value, with an initial value set by a preset threshold. This preset value is set based on experience, typically 100, with a range of 50 to 200; a higher value indicates a higher initial priority. Whenever the rate of request transmission to the same tracked target exceeds a preset rate increase threshold, the dynamic priority value of that target is reduced by a preset step size. This preset step size is set based on experimental data, typically 10, with a range of 5 to 20. Reducing the step size decreases the target's priority, resulting in slower processing for subsequent requests. For example, a tracked target with an initial dynamic priority of 100 might decrease by 10 to 90 the first time it exceeds the preset rate increase threshold, and then decrease by another 10 to 80 the second time. In the request queue, requests for each tracked target are sorted in descending order of their dynamic priority values, with requests having lower dynamic priority values ​​placed at the back of the queue. For example, if there are three targets in the queue with dynamic priority values ​​of 90, 70 and 110 respectively, then after sorting, the request with priority 110 will be placed first, followed by 90, and then 70. The request with a value of 70 will be delayed.

[0042] In this invention, when the increase in the request sending rate of a tracked target exceeds a preset rate increase threshold, the system will reduce the dynamic priority value of the target by one step, causing it to move to the back of the queue, i.e., delaying its processing. This operation is actually based on the core defect that this invention aims to solve—chain triggering and task avalanche.

[0043] In the large-scale inference process of low-altitude robots, the rapid increase in the request sending rate is often not caused by abnormal movement of the actual target, but by "false positive" misjudgments of the small-scale model on the edge under dynamic, nonlinear, and concealed conditions. For example, if a target briefly enters a shaded area, the edge confidence level oscillates between 0.4 and 0.6, resulting in dozens of invalid upload requests per second. If the system provides positive incentives for this rate increase (such as increasing priority), the invalid requests from that target will quickly fill the communication bandwidth, blocking requests from other normal targets. The cloud, unable to receive normal data, misjudges the loss of other targets, thus initiating high-load tasks such as re-identification, triggering a cascading amplification effect and ultimately causing the entire system to crash.

[0044] Therefore, this invention employs a damping control approach: by reducing the priority of requests from abnormal targets, a delay is proactively created, forming negative feedback. This delay causes backlog in the target's uplink queue. Upon detecting this backlog, the endpoint automatically reduces the transmission rate (e.g., from 50 times per second to 10 times per second), thus breaking the positive feedback chain. Simultaneously, the freed bandwidth allows requests from other healthy targets to be transmitted normally, maintaining overall system stability. Similar to the deceleration mechanism in congestion control, it does not punish anomalies but rather prevents a single fault source from spreading globally. In the context of the concealed and collective defects of this invention, this nonlinear damping control is a necessary means to ensure system robustness.

[0045] The steps for performing counterfactual reasoning and calculating causal utility values ​​include the following calculation process: The system records the first true value of the response latency and the second true value of the target tracking success rate after the dynamic determination step is executed. The response latency, measured in milliseconds, is the time elapsed from the time the low-altitude robot sends a request to the time it receives the result from the cloud; it can be measured using the system clock. The target tracking success rate is the proportion of targets correctly tracked within a preset time window out of the total number of tracked targets; its value ranges from 0 to 1 and is calculated by comparing the target positions output by the system with the manually labeled actual positions. For example, in a certain flight mission, the first true value of the response latency after executing the dynamic determination step was 120 milliseconds, and the second true value of the target tracking success rate was 0.92.

[0046] A virtual scenario is constructed, assuming that the adjustment actions in the dynamic determination step are not performed, and the inference granularity parameters and computation task split points from the previous time step are retained. The inference granularity parameters from the previous time step include the model pruning ratio and the number of tokens retained; for example, the model pruning ratio is 0.2 (representing the removal of 20% of redundant channels), and the number of tokens retained is 100. The computation task split point from the previous time step is that the first three Transformer coding blocks are executed on the edge, and the last nine coding blocks are executed on the cloud. Simultaneously, using the same subsequent input data as the real environment, including the same sensor image sequences, the same communication link bandwidth and delay measurements, and the same target motion trajectory, simulation calculations are performed to obtain virtual response delay values ​​and virtual target tracking success rate values. Simulation calculations can be performed during idle periods on the low-altitude robot's edge or offline at the ground station; for example, the simulated virtual response delay value is 180 milliseconds, and the virtual target tracking success rate value is 0.85.

[0047] The difference between the first true value and the virtual response delay value is calculated as the delay change, and the difference between the second true value and the virtual target tracking success rate value is calculated as the success rate change. The delay change equals the first true value minus the virtual response delay value, i.e., 120 milliseconds minus 180 milliseconds equals -60 milliseconds. A negative value indicates that the true delay is 60 milliseconds less than the virtual delay, indicating that the adjustment improved the response speed. The success rate change equals the second true value minus the virtual target tracking success rate value, i.e., 0.92 minus 0.85 equals 0.07. A positive value indicates that the true success rate is 7% higher than the virtual success rate, indicating that the adjustment improved the tracking performance.

[0048] Finally, the change in delay is transformed into a positive indicator that contributes in the same direction as the change in success rate. The two are then weighted and summed to generate a causal utility value. Since a negative change in delay indicates improvement (delay reduction) and a positive change indicates deterioration (delay increase), it needs to be multiplied by a negative coefficient to convert it into a positive contribution. The specific transformation method is as follows: calculate the negative change in delay, i.e., take the opposite number, to obtain the delay improvement. For example, the opposite of negative 60 milliseconds is positive 60 milliseconds. Then, multiply the delay improvement by a first weighting coefficient, and multiply the success rate change by a second weighting coefficient. The two are then added together to obtain the causal utility value. The first weighting coefficient is set to 0.02 (per millisecond) based on empirical values, ranging from 0.01 to 0.05, indicating that each 1-millisecond reduction in delay generates a utility contribution of 0.02. The second weighting coefficient is set to 2.0 (per hundredth) based on experimental data, ranging from 1.0 to 3.0, indicating that each 1-percentage-point increase in success rate generates a utility contribution of 2.0. The causal utility value is equal to 60 milliseconds multiplied by 0.02 plus 0.07 multiplied by 2.0, which is 1.2 plus 0.14 equals 1.34. A positive causal utility value indicates that the adjustment action was effective; a negative value indicates that the adjustment action was ineffective or harmful, and a reverse adjustment strategy is needed.

[0049] The specific method for adjusting the inference granularity parameters using causal utility values ​​is as follows: When the causal utility value is positive, it indicates that the adjustment action performed is effective (i.e., the adjustment action reduces response latency and improves target tracking success rate). Therefore, the first preset threshold is increased and the second preset threshold is decreased. The first preset threshold is used to determine whether the posterior probability of the target's true position offset is too high. Increasing this threshold means that the system becomes more tolerant of position offsets and will not easily trigger an increase in the model pruning ratio and a decrease in the number of tokens retained, thereby reducing unnecessary computational adjustments. The second preset threshold is used to determine whether the probability of communication link interruption is too high. Decreasing this threshold means that the system becomes more sensitive to link interruption. When the interruption probability exceeds a smaller value, it will proactively switch all computational tasks to the edge side to avoid communication risks in advance. For example, when the causal utility value is positive (1.34), the first preset threshold increases from 0.5 to 0.6, and the second preset threshold decreases from 0.3 to 0.2.

[0050] When the causal utility value is negative, it indicates that the adjustment action performed is ineffective or harmful (i.e., the adjustment action increases response latency and reduces target tracking success rate). Therefore, the first preset threshold is reduced and the second preset threshold is increased. Reducing the first preset threshold makes the system more sensitive to positional offsets, triggering an increase in the model pruning ratio and a decrease in the number of tokens retained earlier to improve tracking accuracy. Increasing the second preset threshold makes the system less sensitive to link interruptions, switching to edge execution only when the probability of interruption is higher, to avoid prematurely abandoning cloud computing capabilities. For example, when the causal utility value is negative -0.8, the first preset threshold decreases from 0.5 to 0.4, and the second preset threshold increases from 0.3 to 0.4.

[0051] The increase or decrease is equal to the absolute value of the causal utility multiplied by a preset adjustment factor, and then added to the initial first or second preset threshold value. The preset adjustment factor is set based on experience, typically 0.1, with a range of 0.05 to 0.2. For example, if the absolute value of the causal utility is 1.34, multiplying it by 0.1 gives 0.134, so the first preset threshold increases by 0.134 (or decreases by 0.134 if it is negative). To ensure system stability, the updated first and second preset thresholds are both limited to their corresponding preset value ranges. The first preset threshold ranges from 0.2 to 0.8, and the second preset threshold ranges from 0.1 to 0.6. When the causal utility value is zero, no update operation is performed, and the current first and second preset thresholds remain unchanged.

[0052] The above update logic and the positive / negative correspondence of causal utility values ​​are as follows: A positive utility value indicates that the current adjustment strategy improves system performance. Therefore, the first preset threshold is appropriately increased (to reduce overreaction to position offsets) and the second preset threshold is decreased (to more proactively address link degradation), further optimizing the system in the direction of effective strategies. A negative utility value indicates that the current adjustment strategy harms system performance. Therefore, the opposite adjustment is made: the first preset threshold is decreased (to increase sensitivity to position offsets) and the second preset threshold is increased (to reduce overreaction to link interruptions), thereby correcting harmful strategies.

[0053] Example 2: A large-scale end-side inference system for low-altitude robots, comprising: The data acquisition and fusion module is used to collect the onboard computing resource utilization rate of the low-altitude robot, the available bandwidth and communication latency of the communication link, as well as the movement speed and occlusion ratio of the tracked target, and fuse the collected data into a system state vector. The probability inference module is used to calculate the probability of occurrence of concealed states that cannot be directly measured based on the system state vector using a probability inference method. Concealed states include the target's true position offset, confidence measurement noise, and the probability of communication link interruption. The dynamic determination module is used to dynamically determine the inference granularity parameters of the edge-side large model and the computation task splitting point between the low-altitude robot and the cloud based on the probability of the occurrence of the concealed state. The inference granularity parameters include the model pruning ratio and the number of tokens retained. The damping control module is used to monitor the changes in the request sending rate of multiple tracked targets or multiple low-altitude robots during the dynamic determination process; when the increase in the request sending rate is detected to exceed the preset rate increase threshold, a nonlinear damping mechanism is activated to adjust the queuing priority of the requests. The counterfactual reasoning module is used to perform counterfactual reasoning based on the response delay and target tracking success rate generated by the system after the actual execution of dynamic determination and nonlinear damping mechanism: compare the real response with the virtual response that might have been generated if the current dynamic determination had not been executed, calculate the causal utility value of the current dynamic determination, and use the causal utility value to update the adjustment strategy of the inference granularity parameter.

[0054] All numerical calculations in the above algorithms or formulas are dimensionless, using only numerical substitution for computation. The values ​​of each preset parameter are derived from statistical analysis and experimental verification of data; the specific values ​​can be adjusted within a certain range according to the actual application scenario.

[0055] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0056] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0057] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0058] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A large-scale end-side inference method for low-altitude robots, characterized in that, Includes the following steps: The system collects data on the onboard computing resource utilization rate of the low-altitude robot, the available bandwidth and communication latency of the communication link, as well as the movement speed and occlusion ratio of the tracked target, and fuses the collected data into a system state vector. Based on the system state vector, the probability of occurrence of concealed states that cannot be directly measured is calculated using a probabilistic inference method. Concealed states include the target's true position offset, confidence measurement noise, and the probability of communication link interruption. Based on the posterior probability of the target's true position offset in the probability of the occurrence of the concealed state, the inference granularity parameters of the large model on the edge are dynamically determined. Based on the posterior probability of the communication link interruption probability in the probability of the occurrence of the concealed state, the computational task splitting point between the low-altitude robot and the cloud is dynamically determined. The inference granularity parameters include the model pruning ratio and the number of tokens retained. During the dynamic determination process, the request sending rate changes of multiple tracked targets or multiple low-altitude robots are monitored; when the increase in the request sending rate is detected to exceed the preset rate increase threshold, a nonlinear damping mechanism is activated to adjust the queuing priority of the requests. Based on the response delay and target tracking success rate generated by the system after the actual execution of dynamic determination and nonlinear damping mechanism, counterfactual reasoning is performed: the actual response is compared with the virtual response that might have been generated if the current dynamic determination had not been executed, the causal utility value of the current dynamic determination is calculated, and the adjustment strategy of the inference granularity parameter is updated using the causal utility value. The specific implementation method for adjusting the queuing priority of requests by enabling the nonlinear damping mechanism is as follows: Assign a dynamic priority value to each tracked target, with the initial value of the dynamic priority value being a preset value; Whenever the increase in the request sending rate for the same tracked target exceeds the preset rate increase threshold, the dynamic priority value of the tracked target is reduced by a preset step value. In the request queuing queue, the requests of each tracked target are sorted in descending order of dynamic priority value, with requests with lower dynamic priority values ​​placed at the back of the request queuing queue.

2. The end-side large model inference method for low-altitude robots according to claim 1, characterized in that, The specific steps involved in fusing the collected data into a system state vector include: The onboard computing resource utilization, available bandwidth of the communication link, communication latency of the communication link, movement speed of the tracked target, and occlusion ratio of the tracked target are linearly normalized so that the value of each component is mapped to the interval between zero and one. Then, they are arranged into a five-dimensional vector according to a preset order. This five-dimensional vector is the system state vector.

3. The end-side large model inference method for low-altitude robots according to claim 2, characterized in that, The steps for calculating the probability of occurrence of a concealed state that cannot be directly measured using probabilistic inference methods include the following calculation process: The first step is to use the speed of the tracked target to make a one-step linear prediction of the target's current position, thus obtaining the predicted position; By using the occlusion ratio of the tracked target, the absolute value of the difference between the observed position and the predicted position directly measured by the sensor is corrected. The absolute value of the difference is multiplied by one minus the difference in the occlusion ratio of the tracked target to obtain the observation error value. The second step is to set the prior probability distribution of the target's true position offset as a normal distribution, with the expected value of the normal distribution being zero and the variance being a preset value. The third step is to update the posterior probability of the target's true position offset based on the observation error value and the onboard computing resource utilization rate. The calculation formula is as follows: The posterior probability value = the function value of the probability density function of the prior probability distribution at the observation error value multiplied by the preset scaling factor, and then divided by the preset normalization constant. The fourth step involves using the same computational logic as the first to third steps to calculate the posterior probability of the confidence measurement noise and the posterior probability of the communication link interruption. When calculating the posterior probability of the confidence measurement noise, the variance of the target detection confidence output by the large end-side model at three consecutive time points is used as the observation input. When calculating the posterior probability of the communication link interruption, the available bandwidth and communication delay of the communication link are used to calculate the rate of change of available bandwidth and the rate of change of communication delay within two consecutive time windows, and the weighted sum of the rate of change of available bandwidth and the rate of change of communication delay is used as the observation input.

4. The end-side large model inference method for low-altitude robots according to claim 3, characterized in that, The specific steps for dynamically determining the inference granularity parameters of the large-scale model on the edge and the computational task splitting points between the low-altitude robot and the cloud are as follows: The posterior probability of the target's true position offset in the probability of the concealed state is compared with a first preset threshold: When the posterior probability of the target's true location offset is greater than the first preset threshold, increase the model pruning ratio and reduce the number of tokens retained. When the posterior probability of the target's true location offset is less than the first preset threshold, reduce the model pruning ratio and increase the number of tokens retained. When the posterior probability of the target's true location offset is equal to the first preset threshold, the current model pruning ratio and the current number of tokens retained remain unchanged. The increase or decrease is determined by multiplying the difference between the posterior probability of the target's true position offset and the first preset threshold by a preset step size coefficient. The posterior probability of the communication link interruption probability in the probability of the occurrence of the concealed state is compared with the second preset threshold: When the posterior probability of the communication link interruption probability is greater than the second preset threshold, the calculation task splitting point is adjusted so that all calculation tasks are executed on the end side of the low-altitude robot. When the posterior probability of the communication link interruption probability is less than the second preset threshold, the calculation task splitting point is adjusted so that the first part of the calculation task is executed on the end side of the low-altitude robot and the second part of the calculation task is executed in the cloud. When the posterior probability of the communication link interruption probability is equal to the second preset threshold, the current computation task split point remains unchanged.

5. The end-side large model inference method for low-altitude robots according to claim 4, characterized in that, The preset rate increase threshold is a pre-stored value that represents the maximum allowed increase in the number of requests sent per unit time. The specific method for detecting a rate increase exceeding the preset rate increase threshold is as follows: The average request sending rate within the first time window before the current time is recorded as the first rate reference value, and the average request sending rate within the second time window before the current time is recorded as the second rate reference value, wherein the duration of the second time window is shorter than the duration of the first time window; Calculate the difference between the second rate reference value and the first rate reference value. When the difference is greater than the preset rate growth threshold, it is determined that the increase in the request sending rate exceeds the preset rate growth threshold.

6. The end-side large model inference method for low-altitude robots according to claim 5, characterized in that, The steps for performing counterfactual reasoning and calculating causal utility values ​​include the following calculation process: Record the first real value of the response delay generated by the system after the actual execution of the dynamic determination steps and the second real value of the target tracking success rate; A virtual scene is constructed and it is assumed that the adjustment actions in the dynamic determination step are not executed in the virtual scene. Instead, the inference granularity parameters and calculation task split points of the previous moment are used. The subsequent input data with the same real environment is used for simulation calculation to obtain the virtual response delay value and the virtual target tracking success rate value. The difference between the first real value and the virtual response delay value is calculated as the delay change, and the difference between the second real value and the virtual target tracking success rate value is calculated as the success rate change. The delayed change is transformed into a positive indicator that contributes in the same direction as the success rate change, and then the two are weighted and summed to generate a causal utility value.

7. The end-side large model inference method for low-altitude robots according to claim 6, characterized in that, The specific method for adjusting the inference granularity parameters using causal utility values ​​is as follows: When the causal utility value is positive, increase the first preset threshold and decrease the second preset threshold; When the causal utility value is negative, decrease the first preset threshold and increase the second preset threshold; The increase or decrease is equal to the absolute value of the causal utility value multiplied by a preset adjustment factor, and then superimposed on the initial values ​​of the first preset threshold and the second preset threshold. When the causal utility value is zero, no update operation is performed, and the updated inference granularity parameter is limited to the corresponding preset value range.

8. An end-side large model inference system for low-altitude robots, used to implement the end-side large model inference method for low-altitude robots as described in any one of claims 1-7, characterized in that, include: The data acquisition and fusion module is used to collect the onboard computing resource utilization rate of the low-altitude robot, the available bandwidth and communication latency of the communication link, as well as the movement speed and occlusion ratio of the tracked target, and fuse the collected data into a system state vector. The probability inference module is used to calculate the probability of occurrence of concealed states that cannot be directly measured based on the system state vector using a probability inference method. Concealed states include the target's true position offset, confidence measurement noise, and the probability of communication link interruption. The dynamic determination module is used to dynamically determine the inference granularity parameters of the large model on the edge side based on the posterior probability of the target's true position offset in the probability of the occurrence of the concealed state, and to dynamically determine the computation task splitting point between the low-altitude robot and the cloud based on the posterior probability of the communication link interruption probability in the probability of the occurrence of the concealed state. The inference granularity parameters include the model pruning ratio and the number of tokens retained. The damping control module is used to monitor the changes in the request sending rate of multiple tracked targets or multiple low-altitude robots during the dynamic determination process; when the increase in the request sending rate is detected to exceed the preset rate increase threshold, a nonlinear damping mechanism is activated to adjust the queuing priority of the requests. The counterfactual reasoning module is used to perform counterfactual reasoning based on the response delay and target tracking success rate generated by the system after the actual execution of dynamic determination and nonlinear damping mechanism: compare the real response with the virtual response that might have been generated if the current dynamic determination had not been executed, calculate the causal utility value of the current dynamic determination, and use the causal utility value to update the adjustment strategy of the inference granularity parameter. The specific implementation method for adjusting the queuing priority of requests by enabling the nonlinear damping mechanism is as follows: Assign a dynamic priority value to each tracked target, with the initial value of the dynamic priority value being a preset value; Whenever the increase in the request sending rate for the same tracked target exceeds the preset rate increase threshold, the dynamic priority value of the tracked target is reduced by a preset step value. In the request queuing queue, the requests of each tracked target are sorted in descending order of dynamic priority value, with requests with lower dynamic priority values ​​placed at the back of the request queuing queue.

Images