An open dynamic scene unmanned system navigation method based on a world model

Abstract

The application is a kind of unmanned system navigation method, system, device, medium, program and task sensitive structured pruning method of world model based on open dynamic scene, the navigation method comprises: acquiring multi-modal perception data collected by unmanned aerial vehicle on-board sensor;Feature extraction and fusion are performed on the multi-modal perception data to obtain environment representation;The environment representation is input into the world model to generate potential state and predict future state and reward to construct imagined trajectory;Based on the imagined trajectory, the comprehensive risk evaluation value is calculated;According to the predicted reward value and the comprehensive risk evaluation value, an action strategy is selected;The selected action strategy is executed through the unmanned aerial vehicle control system to perform autonomous navigation. And taking state transition error, reward prediction error, action output deviation and collision risk error as joint evaluation index, the world model is task sensitive structured pruned, and the reasoning scale is adaptively adjusted, realizing real-time, safe and low resource consumption decision of unmanned aerial vehicle.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and autonomous navigation technology for unmanned aerial vehicles (UAVs), specifically to a navigation method, system, device, medium, program, and task-sensitive structured pruning method for the world model in an open dynamic scene unmanned system. Background Technology

[0002] In recent years, with the rapid development of unmanned systems, artificial intelligence, and embedded computing technologies, unmanned aerial vehicles (UAVs) have been widely used in security patrols, environmental monitoring, disaster relief, urban patrols, and complex area exploration. For UAVs to complete autonomous flight missions in real-world environments, they need capabilities such as environmental perception, path planning, dynamic obstacle avoidance, and real-time decision-making. Especially in urban streets, indoor spaces, forest areas, and disaster zones, the environment is typically characterized by complex obstacle distribution, significant lighting changes, frequent appearances of dynamic targets, and unstable communication conditions, which places higher demands on UAV autonomous navigation systems. Traditional methods relying on GPS, rule-based control, or fixed path planning are ill-suited to complex and ever-changing flight environments. Therefore, how to enable UAVs to achieve efficient, safe, and real-time intelligent decision-making in dynamic and complex scenarios has become an important research direction in the field of UAV autonomous navigation. Existing UAV navigation methods mainly include map-based methods and reinforcement learning-based methods. Map-based methods typically rely on SLAM, visual sensors, or LiDAR to localize and map the environment, then combine this with path planning algorithms to generate flight routes. While effective in static or structured environments, these methods struggle with dynamic obstacles, sudden occlusion, and rapid environmental changes, often resulting in map update delays, untimely path replanning, and obstacle avoidance failures. Reinforcement learning-based methods learn navigation strategies through agent-environment interaction, exhibiting strong autonomous decision-making capabilities. However, traditional model-free reinforcement learning usually requires a large number of interaction samples, has a long training period, and suffers from low convergence efficiency under conditions of sparse rewards and high-dimensional visual input. In recent years, world-model-based reinforcement learning methods have emerged, enabling agents to predict future states and perform imaginative policy optimization by learning the latent state transition patterns of the environment, providing a new technological path for dynamic obstacle avoidance in UAVs. However, existing world models typically include multiple modules such as perceptual encoders, state transition networks, reward prediction networks, and policy networks, resulting in complex models with large parameter scales, making them difficult to deploy directly on UAV platforms with limited computing power, storage, and energy consumption. Meanwhile, existing model pruning and quantization methods mostly rely on weight size or channel activation values ​​for compression, lacking characterization of the differences in contributions from state prediction, risk prediction, and action decision-making in UAV navigation tasks. This can easily weaken the UAV's ability to judge future risks while compressing the model. Therefore, there is a need for an intelligent UAV decision-making system that can simultaneously take into account dynamic environmental risk prediction, world model decision-making capabilities, and lightweight deployment requirements. Summary of the Invention

[0003] This application aims to address the problems of existing UAV autonomous navigation methods, such as lag in obstacle avoidance and failure in path planning in dynamic and complex scenarios, as well as the problems that existing world models are complex in structure and have large parameter scale, making them difficult to deploy directly on UAV platforms with limited computing power, and that traditional pruning methods easily weaken the UAV's ability to judge future risks.

[0004] To achieve the above objectives, the first aspect of this application provides a navigation method for an unmanned system in an open dynamic scene based on a world model, comprising: acquiring multimodal perception data collected by airborne sensors of an unmanned aerial vehicle (UAV); extracting and fusing features from the multimodal perception data to obtain an environmental representation O; inputting the environmental representation O into a world model to generate potential states and predict future states and rewards to construct an imagined trajectory; calculating a comprehensive risk assessment value based on the imagined trajectory; selecting an action strategy according to the predicted reward value and the comprehensive risk assessment value; and executing the selected action strategy through the UAV control system to perform autonomous navigation.

[0005] Optionally, the multimodal perception data includes: acquiring RGB images, depth images, and pose state information P; performing feature extraction and fusion on the multimodal perception data to obtain an environment representation O, including: constructing RGB visual features and depth spatial features respectively, fusing the RGB visual features, depth spatial features, and pose state information P to obtain a unified environment representation O that simultaneously includes scene texture information, obstacle spatial distance information, and the UAV's own motion state information.

[0006] Optionally, the world model includes a state representation model, a state transition model, and a reward prediction model. The environmental representation O is input into the world model to generate potential states and predict future states and rewards to construct an imagined trajectory. This includes: generating the current potential state based on the current environmental representation O and historical latent states using the state representation model; predicting the next potential state based on the current potential state and the actions performed by the UAV using the state transition model; predicting future reward values ​​based on the potential state using the reward prediction model; and continuously predicting future states for the next H steps using the world model, starting from the current potential state, to form a potential imagined trajectory.

[0007] Optionally, calculating the comprehensive risk assessment value based on the imagined trajectory includes: estimating the collision probability, the distance to the nearest obstacle, and the uncertainty of state prediction within a certain number of future time steps in the potential imagined trajectory; and calculating the comprehensive risk assessment value Rt based on the collision probability, the distance to the nearest obstacle, and the uncertainty of state prediction, wherein the formula for calculating Rt is:

[0008] In the formula, , , This represents the weighting coefficient of each risk factor. This represents a very small constant to prevent the denominator from being zero. express t The probability of collision at any given moment. express t The nearest obstacle distance at any given time. express t Uncertainty in predicting the state at any given moment; The action strategy is selected based on the future reward value and the comprehensive risk assessment value, including: Construct an optimization objective function for the selected action based on the future reward value and the comprehensive risk assessment value:

[0009] In the formula, express Future reward value at any given moment Indicates the discount factor. Indicates the risk penalty weight; The objective function for selecting actions takes into account both future reward values ​​and collision risk constraints when generating actions, and prioritizes action strategies with higher rewards and lower collision risks.

[0010] Optionally, the perception modality includes the current scene risk level G, the remaining power of the drone B, the processor load C, and the current inference latency T. When in low-risk lightweight inference mode, the potential imagined trajectory length is shortened and some perception modal inputs are reduced. When in medium-risk balanced inference mode, the default imagined trajectory length and the main perception modal inputs are maintained. When in high-risk safe inference mode, the potential imagined trajectory length is increased, and full fusion input of RGB images, depth images, and pose state information with all perception modalities is enabled.

[0011] To achieve the above objectives, a second aspect of this application provides a task-sensitive structured pruning method for world models, comprising: Calculate the task sensitivity importance score of each channel or structural unit in the world model. ,in, In the formula This represents the state transition error caused by removing the i-th channel or structural unit. This represents the reward prediction error caused by removing the i-th channel or structural unit; This indicates the deviation in motion output caused by removing the i-th channel or structural unit; This represents the collision risk error caused by removing the i-th channel or structural unit. , , , These are the weighting coefficients for each indicator; All channels or structural units are sorted according to their task sensitivity and importance scores; Prioritize pruning channels or structural units with importance scores below a preset threshold, and perform short-term fine-tuning on the pruned world model.

[0012] To achieve the above objectives, a third aspect of this application provides a navigation system for an open dynamic scene unmanned system based on a world model, comprising: a data acquisition module for acquiring multimodal perception data collected by airborne sensors of an unmanned aerial vehicle (UAV); a feature processing module for extracting and fusing features from the multimodal perception data to obtain an environmental representation O; a world model prediction module for inputting the environmental representation O into a world model to generate potential states and predict future states and rewards to construct an imagined trajectory; a risk decision module for calculating a comprehensive risk assessment value based on the imagined trajectory and selecting an action strategy based on the predicted reward value and the comprehensive risk assessment value; a control execution module for executing the selected action strategy through the UAV control system; and a model optimization module for performing structured pruning and fine-tuning of the world model based on a task sensitivity and importance score.

[0013] To achieve the above objectives, a fourth aspect of this application provides an electronic device including a processor and a memory, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the open dynamic scene unmanned system navigation method based on a world model as provided in the preceding embodiments and the task-sensitive structured pruning method based on a world model as described in claim 6.

[0014] To achieve the above objectives, a fifth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the open dynamic scene unmanned system navigation method based on a world model as provided in the preceding embodiments and the task-sensitive structured pruning method of the world model as described in claim 6.

[0015] To achieve the above objectives, a sixth aspect of this application provides a computer program product including software instructions that, when executed on an electronic device, cause the electronic device to perform the open dynamic scene unmanned system navigation method based on a world model as provided in the preceding embodiments or the task-sensitive structured pruning method based on a world model as described in claim 6.

[0016] This application discloses a navigation method for unmanned systems in open dynamic scenes based on a world model. The method includes acquiring multimodal perception data collected by onboard sensors of an unmanned aerial vehicle (UAV); extracting and fusing features from the multimodal perception data to obtain an environmental representation O; inputting the environmental representation O into a world model to generate potential states and predict future states and rewards to construct an imagined trajectory; calculating a comprehensive risk assessment value based on the imagined trajectory; selecting an action strategy based on the predicted reward value and the comprehensive risk assessment value; and executing the selected action strategy through the UAV control system for autonomous navigation. This application constructs a unified environmental representation through multimodal perception data fusion, uses a world model to predict future states and rewards in the potential space and constructs an imagined trajectory, calculates a comprehensive risk assessment value by combining collision probability, obstacle distance, and prediction uncertainty, and considers both reward and risk constraints when selecting actions, thereby improving the UAV's forward-looking judgment ability in dynamic environments and enhancing obstacle avoidance success rate and real-time decision-making capabilities. Through a task-sensitive structured pruning method, model compression is performed using state transition error, reward prediction error, action output deviation, and collision risk error as joint evaluation indicators, reducing model computational overhead and inference latency while maintaining navigation decision-making performance. Through a risk and resource adaptive reasoning mechanism, the reasoning scale is dynamically adjusted according to scenario risks and platform resources to achieve real-time, safe, and low-resource-consumption decision-making for drones. Attached Figure Description

[0017] Figure 1 This is a scene diagram provided for an embodiment of the navigation method for open dynamic scene unmanned systems based on a world model in this application; Figure 2 This is a schematic diagram of an embodiment of the navigation method for unmanned systems in open dynamic scenes based on a world model, as provided in this application. Figure 3 This is a flowchart of an embodiment of the navigation method for an open dynamic scene unmanned system based on a world model according to this application.

[0018] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0020] Key terms explained: Edge device / server / terminal: In this application, the world model reasoning and decision-making process can be executed on the UAV's onboard processor (edge ​​device) or ground station server (server), and the control commands ultimately act on the UAV flight control system (terminal).

[0021] Business object / control object: refers to the flight attitude, heading and speed of the UAV, which are directly controlled through action strategies.

[0022] Current UAV navigation methods primarily rely on SLAM mapping combined with path planning algorithms, or employ model-free reinforcement learning for decision-making. The former is prone to map update lag and obstacle avoidance failures in dynamic environments; the latter has long training cycles and low convergence efficiency under high-dimensional visual input. In recent years, reinforcement learning methods based on world models have achieved imaginative policy optimization by learning the latent state transition rules of the environment. However, existing world models are complex in structure and have a large parameter scale, making them difficult to deploy on UAV platforms with limited computing power. At the same time, traditional model pruning methods often compress based on weight size or channel activation values, lacking characterization of the differences in contributions from state prediction, risk assessment, and action decision-making in navigation tasks. This can easily weaken the UAV's ability to judge future risks during the compression process.

[0023] This paper proposes a research approach for unmanned systems in open dynamic scenarios based on a world model. The principle is as follows: In the dynamic obstacle avoidance and autonomous navigation tasks of unmanned aerial vehicles (UAVs), current observation information cannot fully reflect future environmental trends, especially in the face of dynamic obstacles, sudden occlusion, and complex spatial structures. Relying solely on current perception results for action decisions can easily lead to obstacle avoidance lag or path planning failure. A world model, however, can learn the environmental state transition patterns in a potential state space and predict future states, reward changes, and potential risks based on historical observations and action sequences. Therefore, before the UAV executes actions, generating potential imagined trajectories using a world model allows for advance assessment of the potential future outcomes of different actions, thereby improving the UAV's ability to make forward-looking judgments about dynamic environments.

[0024] Furthermore, this invention utilizes the following principle: in the world model, different network channels and structural units contribute differently to the UAV navigation task. Some channels primarily affect image texture reconstruction, some primarily affect state transition prediction, while others contribute more to reward prediction, action output, and collision risk assessment. Therefore, pruning the model solely based on weight norms or channel activation values ​​may inadvertently delete channels crucial to UAV safety decisions, resulting in a smaller model but a decreased obstacle avoidance capability. Based on this principle, this invention uses state transition error... Reward prediction error Action output deviation and collision risk error As a joint channel importance evaluation indicator, task-sensitive structured pruning is performed on the world model to reduce the model's computational overhead while maintaining its predictive ability for future states and navigation risks.

[0025] Based on the above principles, when a UAV is in a dynamic and complex scenario, this system first uses multimodal perception information to construct the potential state of the environment. Then, it predicts the future state, reward, and collision risk through a world model and makes risk perception action decisions within the potential imagined trajectory. At the same time, it performs structured compression of the world model based on the task sensitivity and importance score, and adaptively adjusts the inference scale, imagined trajectory length, and perception modality input method in combination with the current scenario risk level and the UAV's equipment resource status. This enables the UAV to make real-time, safe, and low-resource-consumption decisions in complex and dynamic scenarios.

[0026] refer to Figure 2 and Figure 3 The first embodiment of this application provides a navigation method for unmanned systems in open dynamic scenes based on a world model, in order to solve the technical problems of obstacle avoidance lag, path planning failure, and difficulty in lightweight deployment of models in dynamic and complex scenes mentioned in the background art. The method can be executed by a processor, which can be set in a terminal or a server. The execution process of the method is as follows: Step S101: Acquire multimodal perception data collected by the UAV's onboard sensors.

[0027] In one embodiment of this application, the multimodal sensing data includes RGB images, depth images, and pose state information, denoted as follows: , And P. Among them This represents an RGB image from the drone's forward-facing perspective. This represents the depth image corresponding to the field of view of the RGB image; the pose state information P represents the motion state of the UAV, such as its position, velocity, attitude angle, and flight direction.

[0028] Step S102: Perform feature extraction and fusion on the multimodal sensing data to obtain the environmental representation O.

[0029] In one embodiment of this application, RGB visual features and depth spatial features are constructed separately. The RGB visual features, depth spatial features, and pose state information P are then fused to obtain a unified environment representation O that simultaneously includes scene texture information, obstacle spatial distance information, and the UAV's own motion state information. Specifically, feature extraction networks are constructed for both the RGB image and the depth image to obtain the RGB visual features. and depth spatial features As shown in the following formula:

[0030] in This represents the RGB image feature extraction function. This represents the depth image feature extraction function. Then... , and pose state features By fusion, a unified environmental perception representation O is obtained:

[0031] in This represents the multimodal feature fusion function. This environmental perception representation simultaneously includes scene texture information, obstacle spatial distance information, and the UAV's own motion state information, which is used for subsequent latent state modeling of the world model.

[0032] This environmental perception representation includes scene texture information, obstacle spatial distance information, and the UAV's own motion state information, which are used for potential state modeling in subsequent world models.

[0033] Step S103: Input the environmental representation O into the world model to generate potential states and predict future states and rewards to construct an imagined trajectory.

[0034] In one embodiment of this application, the world model includes a state representation model, a state transition model, and a reward prediction model. The state representation model generates the current potential state based on the current environmental representation O and historical latent states; the state transition model predicts the next potential state based on the current potential state and the actions performed by the UAV; the reward prediction model predicts the future reward value based on the potential state; starting from the current potential state, the world model continuously predicts the states for the next H steps, forming a potential imagined trajectory.

[0035] Specifically, the environmental perception representation O obtained in step 1 is input into the world model, mapping high-dimensional observations to a low-dimensional latent state space to reduce computational complexity. Let... Represents the potential state at time t. This represents the action performed by the drone at time t. Let t represent the reward obtained at time t. Then the world model mainly includes the state representation model, the state transition model, and the reward prediction model.

[0036] The state representation model is used to generate the current latent state based on the current observations and historical hidden states:

[0037] State transition models are used to predict the next potential state based on the current potential state and actions:

[0038] Reward prediction models are used to predict future rewards based on potential states:

[0039] In this way, the world model can simulate the future movement of a drone in a potential space without directly accessing the real environment, and can predict the future environmental state, reward changes and action results.

[0040] Step S104: Calculate the comprehensive risk assessment value based on the imagined trajectory.

[0041] Step S105: Select an action strategy based on the predicted reward value and the comprehensive risk assessment value.

[0042] In one embodiment of this application, an optimization objective function for selecting actions is constructed based on future reward values ​​and comprehensive risk assessment values:

[0043] express Future reward value at any given moment Indicates the discount factor. This represents the risk penalty weight; where the optimization objective function for selecting actions simultaneously considers future reward values ​​and collision risk constraints when generating actions, prioritizing action strategies with higher rewards and lower collision risks.

[0044] Step S106: The selected action strategy is executed by the UAV control system to perform autonomous navigation.

[0045] In one embodiment of this application, the strategy decision module outputs action commands (such as moving forward, backward, left, right, and hovering) based on the potential state and risk assessment results, and sends the actions to the UAV control system for execution. After the UAV executes the actions, it obtains new observation information and reward feedback from the environment and stores them in the experience cache for subsequent world model updates and strategy optimization.

[0046] Furthermore, the perception modalities include the current scene risk level G, the drone's remaining battery power B, the processor load C, and the current inference latency T. In low-risk lightweight inference mode, the potential imagined trajectory length is shortened and some perception modal inputs are reduced. In medium-risk balanced inference mode, the default imagined trajectory length and primary perception modal inputs are maintained. In high-risk safe inference mode, the potential imagined trajectory length is increased, and full fusion of RGB images, depth images, and pose state information with all perception modalities is enabled. This adaptive inference mechanism allows the drone to reduce unnecessary computational overhead while ensuring safety.

[0047] In one embodiment of this application, during training or inference, the current potential state is used. Starting from this point, the world model is used to continuously predict the future state in the next H steps, generating a potential imagined trajectory:

[0048] In this potential imagined trajectory, a collision risk prediction model is introduced to estimate the collision probability, the distance to the nearest obstacle, and the uncertainty of state prediction within several future time steps. Let... Indicates the probability of future collisions. Indicates the distance to the nearest obstacle. Indicating uncertainty in state prediction, the comprehensive risk assessment value... Represented as:

[0049] in , , This represents the weighting coefficient of each risk factor. This represents a minimal constant to prevent the denominator from being zero. When generating actions, the strategy decision-making module considers not only the predicted reward but also the future collision risk; its optimization objective is expressed as:

[0050] in Indicates the discount factor. This indicates the risk penalty weight. In this way, the system can assess the potential future risks of different actions before execution and prioritize action strategies with higher rewards and lower collision risks, thereby achieving proactive obstacle avoidance decisions for the drone.

[0051] Continue to refer to Figure 2 The second embodiment of this application provides a task-sensitive structured pruning method for a world model to solve the same technical problem as the method embodiment. This method can be executed by a processor, which can be located on a terminal or a server. The execution process of this method is as follows: Step S201: Calculate the task sensitivity importance score Ii for each channel or structural unit in the world model.

[0052] In one embodiment of this application, for the i-th channel or structural unit in the world model, its task sensitivity importance score is defined as: Calculate the task sensitivity importance score of each channel or structural unit in the world model. ,in, In the formula This represents the state transition error caused by removing the i-th channel or structural unit. This represents the reward prediction error caused by removing the i-th channel or structural unit; This indicates the deviation in motion output caused by removing the i-th channel or structural unit; This represents the collision risk error caused by removing the i-th channel or structural unit. , , , These are the weighting coefficients for each indicator.

[0053] Step S202: Sort all channels or structural units according to their task sensitivity and importance scores.

[0054] In one embodiment of this application, the system sequentially masks or removes candidate channels or structural units in the world model, and calculates the corresponding... , , and and according to All candidate channels are sorted. For channels or structural units with lower importance scores, the system prioritizes structured pruning; for channels or structural units with higher importance scores, the system retains them.

[0055] Step S203: Prioritize pruning channels or structural units with importance scores below a preset threshold, and perform short-term fine-tuning on the pruned world model.

[0056] In one embodiment of this application, the pruning objects include, but are not limited to, convolutional channels in the perceptual encoder, hidden state dimensions in the latent state transition network, neurons in the reward prediction network, and structural units in the policy decision network. After pruning, the system performs short-term fine-tuning on the pruned world model to restore its state prediction capability, reward prediction capability, risk prediction capability, and action decision-making capability. Through the above-described task-sensitive structured pruning method, the decision-making performance of the world model in UAV dynamic obstacle avoidance tasks can be maintained as much as possible while reducing the number of model parameters, storage consumption, and inference latency.

[0057] Specifically, addressing the limitations of computing power, storage, and energy consumption on UAV platforms, this invention performs task-sensitive structured pruning on the world model. Unlike traditional methods that prune solely based on weight norms or channel activation values, this invention incorporates the state prediction capability, reward prediction capability, action decision-making capability, and risk assessment capability of UAV navigation tasks into the channel importance assessment.

[0058] For the i-th channel or structural unit in the world model, define its task sensitivity importance score. for:

[0059] in, This represents the state transition error caused by removing the i-th channel or structural unit. This indicates the error in reward prediction. This indicates the deviation in action output. Indicates collision risk error. , , , This represents the weighting coefficient of each indicator.

[0060] Among them, state transition error Used to measure the change in the world model's prediction of the next potential state before and after pruning; reward prediction error. Used to measure the change in predicted reward value before and after pruning; action output bias. Used to measure the change in the output action of the policy network before and after pruning; collision risk error. Used to measure the change in risk prediction results before and after pruning.

[0061] After obtaining the importance score for each channel, all channels or structural units are classified according to... The model is sorted, with channels of lower importance scores being pruned first, while channels with a greater impact on state prediction, reward prediction, action output, and collision risk prediction are retained. After pruning, the model undergoes short-term fine-tuning to re-adapt the compressed world model to the UAV navigation task, thereby reducing the number of model parameters and inference latency while maintaining navigation decision performance.

[0062] In one embodiment of this application, the perception modality includes the current scene risk level G, the remaining power of the drone B, the processor load C, and the current inference latency T. When in low-risk lightweight inference mode, the potential imagined trajectory length is shortened and some perception modality inputs are reduced. When in medium-risk balanced inference mode, the default imagined trajectory length and the main perception modality input are maintained. When in high-risk safe inference mode, the potential imagined trajectory length is increased, and the full fusion input of RGB image, depth image, and pose state information with all perception modalities is enabled.

[0063] refer to Figure 1 Specifically, the requirements for model inference capabilities vary depending on the scenario during actual drone flight. When the drone is in an open, low-risk environment, the system can use a shorter imagined trajectory and a smaller inference scale to reduce computational overhead. When the drone is in a scenario with dense dynamic obstacles or sudden occlusion, the system needs to increase the length of the imagined trajectory of the world model and the input of the perception modality to improve risk prediction capabilities and decision-making safety.

[0064] Let G represent the current scenario risk level, B represent the remaining battery power of the drone, C represent the processor load, and T represent the current inference latency. Then, the system selects the inference mode based on the following constraints:

[0065] Here, Mode represents the current inference mode, including low-risk lightweight inference mode, medium-risk balanced inference mode, and high-risk safe inference mode. In low-risk lightweight inference mode, the system shortens the length of the potential imagination trajectory and reduces some perceptual inputs; in medium-risk balanced inference mode, the system maintains the default inference scale and the main perceptual modality; in high-risk safe inference mode, the system increases the length of the potential imagination trajectory and enables full fusion input of RGB image, depth image, and pose state information.

[0066] In summary, this invention realizes a UAV world model decision-making system for dynamic and complex scenarios through multimodal perception state construction, potential world model prediction, risk perception and imagination decision-making, task-sensitive structured pruning, and risk and resource adaptive reasoning. This system can improve the obstacle avoidance success rate, real-time decision-making capability, and platform deployment adaptability of UAVs under dynamic obstacles, sudden occlusion, and complex spatial structures, while reducing model storage consumption, computational overhead, and inference latency.

[0067] The third embodiment of this application provides a navigation system for an open dynamic scene unmanned system based on a world model to solve the same technical problem as the method embodiment. The system may include the following modules: a data acquisition module, a feature processing module, a world model prediction module, a risk decision-making module, a control execution module, and a model optimization module.

[0068] The system comprises the following modules: a data acquisition module for acquiring multimodal perception data collected by the UAV's onboard sensors; a feature processing module for extracting and fusing features from the multimodal perception data to obtain an environmental representation O; a world model prediction module for inputting the environmental representation O into the world model to generate potential states and predict future states and rewards to construct an imagined trajectory; a risk decision module for calculating a comprehensive risk assessment value based on the imagined trajectory and selecting an action strategy based on the predicted reward value and the comprehensive risk assessment value; a control execution module for executing the selected action strategy through the UAV control system; and a model optimization module for performing structured pruning and fine-tuning of the world model based on the task sensitivity and importance score.

[0069] It is understood that the above-described device embodiments correspond to the method embodiments. Therefore, the device embodiments have all the beneficial effects of the method embodiments, and will not be repeated here.

[0070] The fourth embodiment of this application provides an electronic device, including a processor and a memory. The memory stores a computer program, and the processor is configured to execute the computer program to implement the open dynamic scene unmanned system navigation method based on the world model and the task-sensitive structured pruning method based on the world model provided in the previous embodiments.

[0071] It is understood that the above-described electronic device embodiments correspond to the method embodiments. Therefore, the electronic device embodiments have all the beneficial effects of the method embodiments, and will not be repeated here.

[0072] The fifth embodiment of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the open dynamic scene unmanned system navigation method based on a world model and the task-sensitive structured pruning method based on the world model provided in the previous embodiments.

[0073] It is understood that the above storage medium embodiments correspond to the method embodiments. Therefore, the storage medium embodiments have all the beneficial effects of the method embodiments, and will not be repeated here.

[0074] The sixth embodiment of the application provides a computer program product, including software instructions, which, when executed on an electronic device, cause the electronic device to perform the open dynamic scene unmanned system navigation method based on the world model provided in the previous embodiment or the task-sensitive structured pruning method based on the world model provided in the previous embodiment.

[0075] It is understood that the above-described program product embodiments correspond to the method embodiments. Therefore, the program product embodiments have all the beneficial effects of the method embodiments, and will not be repeated here.

[0076] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A navigation method for unmanned systems in open dynamic scenes based on a world model, characterized in that, include: Acquire multimodal perception data collected by UAV onboard sensors; Feature extraction and fusion are performed on the multimodal sensing data to obtain the environmental representation O; The environmental representation O is input into the world model to generate potential states and predict future states and rewards to construct an imagined trajectory; Calculate the comprehensive risk assessment value based on the imagined trajectory; Select an action strategy based on the predicted reward value and the comprehensive risk assessment value; And to execute selected action strategies through the drone control system for autonomous navigation.

2. The navigation method for unmanned systems in open dynamic scenes based on a world model as described in claim 1, characterized in that, The multimodal sensing data includes: acquiring RGB images, depth images, and pose state information P; Feature extraction and fusion are performed on the multimodal sensing data to obtain an environmental representation O, including: RGB visual features and depth spatial features are constructed separately. The RGB visual features, depth spatial features and pose state information P are fused to obtain a unified environment representation O that simultaneously includes scene texture information, obstacle spatial distance information and UAV's own motion state information.

3. The navigation method for unmanned systems in open dynamic scenes based on a world model as described in claim 1, characterized in that, The world model includes state representation models, state transition models, and reward prediction models; The environmental representation O is input into the world model to generate potential states and predict future states and rewards to construct an imagined trajectory, including: The current latent state is generated by the state representation model based on the current environment representation O and the historical hidden states. The state transition model is used to predict the potential state at the next moment based on the current potential state and the actions performed by the UAV. Predict future reward values ​​based on potential states using a reward prediction model; Starting from the current potential state, the world model is used to continuously predict the future H-step states, forming a potential imagined trajectory.

4. The navigation method for unmanned systems in open dynamic scenes based on a world model as described in claim 3, characterized in that, The comprehensive risk assessment value is calculated based on the imagined trajectory, including: In the potential imagined trajectory, the collision probability, the distance to the nearest obstacle, and the uncertainty of state prediction are estimated within a certain number of future time steps; A comprehensive risk assessment value is calculated based on the collision probability, the distance to the nearest obstacle, and the uncertainty of the state prediction. ,in The calculation formula is: In the formula, , , This represents the weighting coefficient of each risk factor. This represents a very small constant to prevent the denominator from being zero. express t The probability of collision at any given moment. express t The nearest obstacle distance at any given time. express t Uncertainty in predicting the state at any given moment; The action strategy is selected based on the future reward value and the comprehensive risk assessment value, including: Construct an optimization objective function for the selected action based on the future reward value and the comprehensive risk assessment value: in, express Future reward value at any given moment Indicates the discount factor. Indicates the risk penalty weight; The objective function for selecting actions takes into account both future reward values ​​and collision risk constraints when generating actions, and prioritizes action strategies with higher rewards and lower collision risks.

5. The navigation method for unmanned systems in open dynamic scenes based on a world model as described in claim 1, characterized in that, The perception modality includes the current scene risk level G, the remaining battery power of the drone B, the processor load C, and the current inference latency T; The method further includes: When in a low-risk, lightweight reasoning mode, shorten the length of potential imagination trajectories and reduce perceptual modal input; When in a medium-risk equilibrium reasoning mode, maintain the default imaginary trajectory length and the main perceptual modality input; When in high-risk safe reasoning mode, increase the length of potential imagined trajectory and enable full fusion input of RGB images, depth images and pose state information with all perceptual modalities.

6. A task-sensitive structured pruning method for world models, characterized in that, include: Calculate the task sensitivity importance score of each channel or structural unit in the world model. ,in, In the formula This represents the state transition error caused by removing the i-th channel or structural unit. This represents the reward prediction error caused by removing the i-th channel or structural unit; This indicates the deviation in motion output caused by removing the i-th channel or structural unit; This represents the collision risk error caused by removing the i-th channel or structural unit. , , , These are the weighting coefficients for each indicator; All channels or structural units are sorted according to their task sensitivity and importance scores; Prioritize pruning channels or structural units with importance scores below a preset threshold, and perform short-term fine-tuning on the pruned world model.

7. A navigation system for unmanned systems in open dynamic scenes based on a world model, characterized in that, include: The data acquisition module is used to acquire multimodal perception data collected by the UAV's onboard sensors; The feature processing module is used to extract and fuse features from the multimodal sensing data to obtain an environmental representation O. The world model prediction module is used to input the environmental representation O into the world model, generate potential states, and predict future states and rewards to construct an imagined trajectory. The risk decision-making module is used to calculate a comprehensive risk assessment value based on the imagined trajectory, and select an action strategy based on the predicted reward value and the comprehensive risk assessment value. The control execution module is used to execute the selected action strategy through the drone control system. as well as The model optimization module is used to perform structured pruning and fine-tuning of the world model based on the task sensitivity and importance score.

8. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the open dynamic scene unmanned system navigation method based on the world model as described in any one of claims 1 to 5 and the task-sensitive structured pruning method based on the world model as described in claim 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the open dynamic scene unmanned system navigation method based on a world model as described in any one of claims 1 to 5 and the task-sensitive structured pruning method based on a world model as described in claim 6.

10. A computer program product comprising software instructions, characterized in that, When the software instructions are executed on the electronic device, the electronic device performs the open dynamic scene unmanned system navigation method based on the world model as described in any one of claims 1 to 5, or the task-sensitive structured pruning method based on the world model as described in claim 6.

Images