Physical knowledge injection-based unmanned aerial vehicle implicit semantic thinking chain mechanism construction method

Abstract

The invention provides a physical knowledge injection-based unmanned aerial vehicle implicit semantic thinking chain mechanism construction method, which comprises the following steps that: firstly, an unmanned aerial vehicle carries a monocular camera to collect an image frame, inputs the image frame into an edge calculation unit for processing, and obtains a physical feature vector representing unmanned aerial vehicle single-view physical geometric information; an original task instruction is aggregated and mapped into task semantic embedding by a task instruction embedding module, and a physical feature vector is converted into physical knowledge embedding matched with an implicit reasoning generator by a feature mapping module; and then multi-modal joint embedding fusion operation is executed, an implicit reasoning token is output and injected into the large language model, and a final unmanned aerial vehicle task execution instruction is output. According to the method, the problem of physical constraint deficiency in large language model decision is solved, the kinematics and geometric topology feasibility of the instruction is enhanced, the problem of high decision delay caused by explicit reasoning text redundancy is relieved through an implicit reasoning mechanism of semantic alignment, and the task instruction understanding ability of an unmanned aerial vehicle system in a complex environment is improved.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence, robot control and autonomous navigation, and in particular relates to a method for constructing an implicit semantic thinking chain mechanism for unmanned aerial vehicles (UAVs) with physical knowledge injection. Background Technology

[0002] In recent years, with the rapid development of drone and mobile robot technologies in fields such as logistics, reconnaissance, and environmental monitoring, higher demands have been placed on environmental perception, high-level mission understanding, and real-time decision-making. In particular, when drones perform missions in complex, unstructured, and dynamic environments, their decision-making not only needs to accurately understand abstract human instructions, but also needs to strictly adhere to the geometric and motion constraints of the physical world to ensure mission success and flight safety.

[0003] Traditional autonomous unmanned aerial vehicle (UAV) systems primarily rely on control strategies based on geometric mapping and pre-defined rules. While these methods demonstrate stability in tasks such as path planning and obstacle avoidance, their decision-making capabilities lack generalization and high-level abstract reasoning abilities, making it difficult to integrate fine-grained environmental information (such as the precise motion trends of dynamic objects) into multi-step logical reasoning. Furthermore, these methods struggle to handle high-level task commands expressed in natural language, which are often fuzzy and involve complex logic.

[0004] To overcome the reasoning limitations of traditional methods, researchers have begun to introduce Large Language Models (LLMs) into high-level decision-making systems for robots, particularly by enhancing their logical reasoning capabilities through thought chain mechanisms. However, existing LLM-driven autonomous systems face significant challenges when applied to edge computing scenarios involving unmanned aerial vehicles (UAVs), primarily in two interrelated aspects:

[0005] On the one hand, there is a lack of rigid constraints from physical knowledge. Large language models primarily derive their knowledge from textual data, inherently lacking an understanding of real-world 3D geometry, topology, depth information, and kinematic feasibility. Existing mainstream multimodal methods typically only inject coarse-grained 2D image features, making it difficult to support the fine-grained 3D spatial constraints required for UAV navigation. For example, relying solely on 2D image features makes it difficult to accurately determine whether an obstacle represented by a set of Gaussian points is on a safe flight path. This lack of rigid physical constraints in the decision-making mechanism can easily lead to large language models generating physically infeasible or unsafe execution commands.

[0006] On the other hand, there are bottlenecks in inference efficiency and edge real-time performance. Traditional explicit thought chain mechanisms require large language models to autoregressively generate lengthy, intermediate inference text, which introduces huge computational latency. This millisecond or even second-level latency directly affects the drone's real-time response capability to dynamic environments, and is a fatal efficiency obstacle for drone missions that need to perform high-frequency, high-real-time path adjustments and action outputs on resource-constrained edge computing units.

[0007] In summary, the key challenge that current autonomous systems driven by large language models urgently need to address is: how to achieve low-cost and high-efficiency extraction and injection of high-fidelity 3D geometric and physical knowledge under the edge computing architecture of UAVs, while overcoming the inference latency of large language models through innovative inference mechanisms, so as to output task execution instructions that are both semantically correct and physically feasible.

[0008] To address this, this invention proposes a method for constructing an implicit semantic thinking chain mechanism for UAVs with physical knowledge injection. By converting depth and residual flow features online into a 3D Gaussian point set as a source of physical knowledge, and using a lightweight implicit generator and semantic alignment loss to convert this knowledge into an acceleration token for injection into a large language model, this method effectively solves the two core bottlenecks of large language models in real-time UAV decision-making, and possesses strong real-time performance, adaptability, and engineering application potential. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of existing technologies by providing a method for constructing an implicit semantic thinking chain mechanism for unmanned aerial vehicles (UAVs) with physical knowledge injection. By introducing a real-time 3D geometric perception module, the problem of missing physical constraints in large language model decision-making is solved, significantly enhancing the kinematic and geometric topological feasibility of commands. At the same time, by adopting a semantically aligned implicit reasoning mechanism, the problem of high decision-making delay caused by explicit reasoning text redundancy is effectively alleviated, improving the UAV system's ability to understand high-level mission commands and the physical reliability of real-time decision-making in complex dynamic environments.

[0010] The present invention achieves the above-mentioned technical objectives through the following technical solutions.

[0011] A method for constructing an implicit semantic thought chain mechanism for drones infused with physical knowledge includes the following process:

[0012] Step 1: The UAV, equipped with a monocular camera, acquires image frames in real time and inputs them into the multi-task online feedforward feature extractor of the edge computing unit. Based on an image encoder, a depth estimation network, and a residual flow network, the multi-task online feedforward feature extractor generates depth feature maps and residual flow features from the UAV's single-view perspective online. Utilizing a sparse representation-based 3D Gaussian sputtering mechanism, it reconstructs a 3D Gaussian point set for the local scene from the current viewpoint. Subsequently, through global average pooling and a point cloud encoder, the depth feature map and the 3D Gaussian point set are nonlinearly aggregated into a physical feature vector representing the physical geometry information of the UAV's single-view perspective. ;

[0013] Step 2: Aggregate and map the raw task commands received by the UAV into fixed-length task semantic embeddings using the task command embedding module. Design a feature mapping module based on a multilayer perceptron to map physical feature vectors. As input, it is mapped and aligned to the preset input embedding dimension of the large language model through multi-layer nonlinear projection operations to obtain the physical knowledge embedding. ;

[0014] Step 3: Embed task semantics using the multimodal knowledge fusion and implicit thought chain generation module. Embedded with physics knowledge Channel-level stitching is performed, followed by linear compression projection for fusion, to obtain the joint embedding tensor. ; and will jointly embed tensors The input is fed into a semantically aligned implicit inference generator, whose core structure consists of a lightweight Transformer encoder, to generate implicit inference tokens. ;

[0015] Step 4: Transfer the implicit reasoning token As a differentiable prefix, it is injected into the front end of the input layer embedding sequence of a pre-trained large language model to form a joint input; in implicit inference tokens Guided by an attention weight transfer mechanism, the autoregressive generation of explicit reasoning text is skipped, accelerating the reasoning of thought chains and outputting UAV mission execution instructions that meet physical constraints in terms of kinematic and geometric topological feasibility. This is to guide drones to complete designated tasks.

[0016] Furthermore, the specific process of step 1 is as follows:

[0017] The multi-task online feedforward feature extractor first extracts basic image features from the input image frames using a lightweight image encoder based on the MobileNetV3 architecture. These features are then fed in parallel to the depth estimation network and the residual flow network for feedforward inference. The depth estimation network employs a ResNet-18-based encoder-decoder structure, utilizing ResNet-18 as the encoder to extract powerful geometric features. These features are then passed to the decoder for upsampling via skip connections. The depth estimation network outputs a single-view depth feature map. The feature map quantifies the distance from each pixel in the scene to the drone and serves as a key constraint for 3D geometric modeling. The residual flow network adopts the PWC-Net architecture, which takes two consecutive frames as input, performs pyramid feature extraction, feature warping, and cost volume calculation, and outputs a full-scene optical flow field including both rigid and non-rigid motion. From the full-scene optical flow field The non-rigid motion components are explicitly separated as features of the residual flow. ;

[0018] Then and The input is fed into a 3D Gaussian sputtering mechanism based on sparse representation, utilizing... The pixel-level depth information is used as a supervision signal for geometric initialization to determine the initial position of the Gaussian point, while utilizing... The encoded motion residual information is used to predict the rotation, scale, and color attributes of each Gaussian point, thereby reconstructing a 3D Gaussian point set of the local scene for the current viewpoint within the framework of differential renderability. To characterize the spatiotemporal geometric information of the current environment of the UAV;

[0019] Next to Perform global average pooling to obtain the deep feature vector. Simultaneously, a PointNet point cloud encoder based on local feature aggregation and hierarchical structure is employed to process the Gaussian point set. The physical properties of each Gaussian point are encoded and aggregated to obtain the Gaussian point feature vector. Ultimately and By concatenating the data along the channel dimension, we obtain a physical feature vector representing the single-view physical geometry information of the UAV. .

[0020] Furthermore, the specific process of step 2 is as follows:

[0021] The edge computing unit inputs the raw mission commands received by the UAV into the mission command embedding module. The mission command embedding module first performs word segmentation and word embedding processing on the commands, and then uses a lightweight Transformer encoder to aggregate and map the variable-length embedding sequence of mission commands into fixed-length mission semantic embeddings. Then, the feature mapping module is designed. The feature mapping module adopts a feature mapping module with... The multilayer perceptron structure consists of multiple layers. The multilayer perceptron first... As initial input Then, the initial input pass The hidden layer is iteratively processed, and the... Layer operations employ residual connections, layer normalization, and the GELU activation function to ensure information flow and nonlinear mapping capability during the projection process. The calculation formula is as follows:

[0022]

[0023] in, For the first The hidden state of the layer's output Indicates the first The hidden state of the layer's output The range of values ​​for is: ; , They are the first The weight matrix and bias vector of the layer, , , For the fixed dimensions of the hidden layers of a multilayer perceptron, This represents the GELU activation function. Representation layer normalization; finally, in the first... Layer, the multilayer perceptron performs the final linear projection, changing the dimension from Mapping to target dimension The calculation formula is: , , They represent the first The weight matrix and bias vector of the layer, Indicates the first The layer outputs hidden states; the final feature mapping module maps high-dimensional physical feature vectors. Transform into physical knowledge embeddings aligned with the input embedding dimension of the implicit inference generator. .

[0024] Furthermore, the specific process of step 3 is as follows:

[0025] Edge computing units utilize multimodal knowledge fusion and implicit thought chain generation modules to perform multimodal joint embedding fusion operations, embedding physical knowledge. Injected into task semantic embedding In this process, an input tensor for implicit reasoning is formed; specifically, a mechanism of linear compression projection after feature channel-level concatenation is adopted: and Concatenate along the feature channel dimension to obtain the joint embedding vector. :

[0026]

[0027] in, Indicates a splicing operation;

[0028] Then, Through a single-layer linear projection Using weight matrix and Compress the dimensions and align them back to the target dimensions. The fused joint embedding tensor is obtained. :

[0029]

[0030] in, , ;

[0031] Next, the joint embedding tensor As input, it is fed into a semantically aligned implicit inference generator. The core structure of the implicit inference generator consists of a lightweight Transformer encoder, which internally includes an input projection layer, multiple self-attention blocks, and an output sequence expansion layer. The input projection layer is responsible for receiving a single fixed-length vector. And map it to an implicit token with a preset length. Internal representation tensor ;

[0032] Subsequently, the internal representation tensor After encoding through multiple layers of self-attention blocks, the latent reasoning relationships within the fused features are captured using a self-attention mechanism; finally, the output sequence expansion layer outputs a series of fixed-length implicit reasoning tokens. :

[0033]

[0034] in, This represents an implicit reasoning generator.

[0035] Furthermore, the aforementioned parameters Training and optimization are performed independently of the final large language model, while the core weights of the large language model remain frozen. The optimization objective is determined by the total loss, and the total loss for: , For predefined hyperparameters, For semantic alignment loss, Modeling loss for language.

[0036] Furthermore, the specific process of step 4 is as follows:

[0037] First, the multimodal knowledge fusion and implicit thought chain generation module obtains the length. Dimensions Implicit reasoning tokens And feed it into a pre-trained large language model In the input computation stream, to guide the large language model to quickly generate the final execution instructions. The injection mechanism employs an input layer embedding and concatenation strategy, using implicit reasoning knowledge as a differentiable prefix of the input sequence of the large language model. The specific process is as follows:

[0038] First, the original mission instructions for the drone... Input to the word embedding layer of the large language model In this process, an embedded sequence of original task instructions is generated. , ,in, The word segmentation length of the instruction; then, the implicit inference token. spliced ​​together in the sequence dimension The front end forms the final joint input embedding sequence. :

[0039]

[0040] Subsequently, the joint input embedding sequence As input, it is fed into the large language model. In the Transformer stack; ultimately, the large language model Implicit inference tokens Under strong guidance, the standard autoregressive decoding is executed to output the final execution command for the UAV mission:

[0041]

[0042] in, This indicates an autoregressive decoding operation.

[0043] The present invention has the following beneficial effects:

[0044] (1) By introducing a physical knowledge injection mechanism based on real-time 3D Gaussian point set, 3D geometric topology information is transformed into implicit reasoning tokens, which effectively eliminates the problem of missing physical constraints in the decision-making of large language models and ensures the kinematic and geometric topology feasibility of instructions.

[0045] (2) Relying on a lightweight implicit reasoning generator and a differentiable prefix injection mechanism, it realizes accelerated thinking chain reasoning, effectively avoids the huge computational delay caused by the autoregressive generation of lengthy texts on edge computing units of large language models, significantly enhances the real-time response speed of UAV system in complex dynamic environments, significantly enhances the timeliness, accuracy and robustness of UAV mission execution commands, and provides an efficient and scalable technical path for achieving high-security autonomous navigation under complex commands. Attached Figure Description

[0046] Figure 1 Flowchart of the method for constructing the implicit semantic thinking chain mechanism of UAVs that injects physical knowledge as described in this invention. Detailed Implementation

[0047] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the embodiments described.

[0048] The method for constructing the implicit semantic thinking chain mechanism of UAVs with physical knowledge injection described in this invention is as follows: Figure 1 As shown, the specific process includes the following:

[0049] Step 1: The drone is equipped with a monocular camera to collect real-time data on the current moment. Image frames This serves as the raw input for environmental perception; and the acquired image frames... Multi-task online feedforward feature extractor that periodically inputs to edge computing units (hereinafter referred to as extractor) In ), extractor The aim is to efficiently extract depth geometry and motion residual information from single-view images for constructing 3D scene representations. The specific steps are as follows:

[0050] Extractor First, a lightweight image encoder based on the MobileNetV3 architecture is used to process the input image frames. Extracting basic image features , This image encoder utilizes inverted residual blocks and depthwise separable convolutions to achieve efficient feature extraction, wherein... Represents the space of real numbers. , These represent the height and width of the feature map, respectively. Representing image features The number of channels. Next, As input, the data is fed in parallel to the depth estimation network and the residual flow network for feedforward inference. The depth estimation network employs an Encoder-Decoder structure based on ResNet-18 (a convolutional neural network architecture). It utilizes ResNet-18 as the encoder to extract powerful geometric features and passes these features to the decoder for upsampling via skip connections. The depth estimation network outputs a single-view depth feature map. , This feature map quantifies the distance from each pixel in the scene to the drone and serves as a key constraint for 3D geometric modeling. Representation of deep feature map The number of channels; the residual flow network adopts the PWC-Net (a deep learning model for optical flow estimation) architecture, and the residual flow network will process two consecutive frames of images ( , Using this as input, pyramid feature extraction, feature distortion, and cost volume calculation are performed to efficiently output the full-scene optical flow field, including both rigid and non-rigid motion. From the full-scene optical flow field The non-rigid motion components are explicitly separated as features of the residual flow. The residual flow characteristics It is used to model non-rigid motion residuals in image sequences, and its role is to explicitly encode the local motion of dynamic objects, thereby providing temporal and dynamic information for 3D geometric representation.

[0051] Then, and The input is fed into a sparse representation-based 3D Gaussian sputtering mechanism, which performs multi-task optimization to achieve online reconstruction of 3D point sets. Specifically, it utilizes... The pixel-level depth information is used as a supervision signal for geometric initialization to determine the Gaussian points. The initial position, while utilizing The encoded motion residual information is used to predict the rotation, scale, and color attributes of each Gaussian point, thereby reconstructing a 3D Gaussian point set of the local scene for the current viewpoint within the framework of differential renderability. This is to accurately characterize the spatiotemporal geometric information of the UAV's current environment. The Gaussian point set... Each Gaussian point in Include , , , Several key physical properties, For spatial location, , For color, determined by a first-order spherical harmonic function express, For opacity, and 3D covariance matrix Using a rotation matrix that describes the spatial orientation of the Gaussian point and the scale vector describing the scale of a Gaussian point along its three principal axes Perform parameterization: , This represents the matrix transpose operation.

[0052] Next, in order to extract the deep feature map With Gaussian point set The unified encoding is used to injectable knowledge vectors for deep feature maps. Perform global average pooling to obtain the deep feature vector. Simultaneously, a PointNet point cloud encoder based on local feature aggregation and hierarchical structure is employed to process the Gaussian point set. Each Gaussian point The physical properties are encoded and aggregated to obtain Gaussian point feature vectors. Finally, the deep feature vectors and Gaussian point eigenvectors By concatenating the data along the channel dimension, we obtain a physical feature vector representing the single-view physical geometry information of the UAV. This serves as the input for the physics knowledge in this round.

[0053] Step 2: The edge computing unit uses the physical feature vector representing the single-view physical geometry information of the UAV, output from Step 1. and the original mission instructions received by the drone , , the original task instructions The input is fed into the task instruction embedding module, which first processes the input into the task instruction embedding module. The process involves word segmentation and word embedding. Then, a lightweight Transformer encoder aggregates and maps the variable-length embedding sequence of task instructions into fixed-length task semantic embeddings. , ,in, It is an implicit reasoning generator The input embedding space dimension of the large language model used is predetermined during the system training phase and referenced as an external parameter.

[0054] Then, to achieve dimensional alignment and nonlinear semantic projection of heterogeneous modal features, a feature mapping module is designed. This module uses a A multilayer perceptron structure is described, which aims to transform high-dimensional physical feature vectors through nonlinear transformation. Project and embed into the implicit inference generator. In the semantic space of the large language model used, it simultaneously preserves The physical constraint information in the process. The specific process is as follows: The multilayer perceptron first... As initial input : Its dimensions are Subsequently, the initial input pass The hidden layer is iteratively processed, and the... Layer operations employ residual connections, layer normalization, and the GELU activation function to ensure information flow and nonlinear mapping capability during the projection process. The calculation formula is as follows:

[0055]

[0056] in, For the first The hidden state of the layer's output Indicates the first The hidden state of the layer's output The range of values ​​for is: ; , They are the first The weight matrix and bias vector of the layer, , , For the fixed dimensions of the hidden layers of a multilayer perceptron, This represents the GELU activation function. Representation layer normalization; finally, in the first... Layer, the multilayer perceptron performs the final linear projection, changing the dimension from Mapping to target dimension The calculation formula is: , , They represent the first The weight matrix and bias vector of the layer, Indicates the first The layer outputs hidden states; finally, the feature mapping module maps high-dimensional physical feature vectors. Transformed into a physical knowledge embedding that is strictly aligned with the input embedding dimension of the implicit inference generator. , This physical knowledge is embedded It carries spatiotemporal geometric constraint information in the UAV environment for subsequent task semantic embedding. To carry out joint integration.

[0057] Step 3: First, the edge computing unit calculates the dimension based on the output of Step 2. Task semantic embedding Embedded with physics knowledge Next, a multimodal joint embedding fusion operation is performed using the multimodal knowledge fusion and implicit thought chain generation module to efficiently integrate high-dimensional physical constraint information. Injected into task semantic embedding This forms the input tensor for implicit inference. Specifically, a mechanism of linear compression projection after feature channel-level concatenation is employed: [The mechanism involves...] and Concatenate along the feature channel dimension to obtain the joint embedding vector. :

[0058]

[0059] in, Indicates a splicing operation;

[0060] Then, Through a single-layer linear projection Using weight matrix and Compress the dimensions and align them back. The fused joint embedding tensor is obtained. :

[0061]

[0062] in, , ;

[0063] Next, the joint embedding tensor ( As input, it is fed into a semantically aligned implicit reasoning generator. middle; The core structure consists of a lightweight Transformer encoder, which internally includes an input projection layer, multiple self-attention blocks, and an output sequence expansion layer. The input projection layer is responsible for receiving a single fixed-length vector. And map it to an implicit token with a preset length. Internal representation tensor : ;

[0064] This process is a mapping from vectors to sequences, providing the initial state for the subsequent Transformer encoder's serialization; subsequently, the internal representation tensor... After encoding through multiple layers of self-attention blocks, the latent reasoning relationships within the fused features are captured using a self-attention mechanism; finally, the output sequence expansion layer outputs a series of fixed-length implicit reasoning tokens. The generation process is represented as follows:

[0065]

[0066] Meanwhile, during the training phase, parameters Training and optimization are performed independently of the final large language model, while the core weights of the large language model remain frozen. This separate training mechanism aims to maximize the preservation of the general reasoning capabilities of the pre-trained large language model, while optimizing... To learn the efficient mapping of cross-modal knowledge to the implicit reasoning space. The optimization objective is determined by the total loss, and the total loss for: , For predefined hyperparameters, only parameters Receive by semantic alignment loss and language modeling loss The gradient generated by the combined action ensures Minimize the final execution instructions While outputting errors, it can form a structured implicit reasoning space; Employing metric learning principles, based on the original task instructions. The intrinsic semantic categories (e.g., identified as "long-distance path planning" or "near-distance obstacle avoidance") are used to construct a contrastive learning dataset, and the dataset generated by tasks of the same category is minimized. Distance between sequences, while maximizing the difference between different classes The distance between sequences, thus forcing Learn a highly structured and distinguishable implicit representation space to ensure that similar physical tasks and inference requirements are addressed in implicit inference tokens. They are aggregated in the embedded space, while heterogeneous tasks are effectively separated.

[0067] The implicit inference token Implicit encoding of the original task instructions within the non-text space To the final execution instruction The logical reasoning chain, and due to the embedding of physical knowledge Injection, implicit inference token The system rigorously encodes physical constraints such as the drone's 3D geometry, environmental depth constraints, and kinematic feasibility. Finally, implicit inference tokens are used. It is sent to step 4 for injection and accelerated reasoning.

[0068] Step 4: First, the multimodal knowledge fusion and implicit thought chain generation module obtains the length of... Dimensions Implicit reasoning tokens And feed it into a pre-trained large language model In the input computation stream, to guide the large language model to quickly generate the final execution instructions. The injection mechanism (i.e., injecting implicit reasoning tokens into the pre-trained large language model) employs an input layer embedding concatenation strategy, aiming to use implicit reasoning knowledge as a differentiable prefix of the input sequence of the large language model. The specific process is as follows:

[0069] First, the original mission instructions for the drone... Input to the word embedding layer of the large language model In this process, an embedded sequence of original task instructions is generated. , ,in, This is the word segmentation length of the instruction. Next, the implicit inference token... spliced ​​together in the sequence dimension The front end forms the final joint input embedding sequence. :

[0070]

[0071] Subsequently, the joint input embedding sequence As input, it is fed into the large language model. Within the Transformer stack, the implicit thought chain's accelerated reasoning mechanism is specifically reflected in the large language model's... In the processing: during the self-attention mechanism calculation of each Transformer layer in the large language model, the instruction token (i.e. The attention weights (in part) will be focused on implicit inference tokens. Sequence-wise, this allows large language models to directly infer tokens from implicit reasoning. It extracts the necessary reasoning steps and decision logic without having to gradually build a thought process through explicit text generation. Implicit reasoning tokens It acts as a pre-computed, high-dimensional reasoning summary, greatly compressing traditional autoregressive reasoning paths. Due to implicit reasoning tokens... length With a length much shorter than the explicit thought chain text, this mechanism accelerates thought chain reasoning and significantly reduces the latency required for real-time decision-making by drones.

[0072] Ultimately, large language models Implicit inference tokens Under strong guidance, the standard autoregressive decoding is executed to output the final execution command for the UAV mission. :

[0073]

[0074] in, This indicates an autoregressive decoding operation;

[0075] The final execution instruction It is a concise text sequence that conforms to physical constraints, and its kinematic and geometric topological feasibility is determined by implicit inference tokens. It indirectly inherits the spatial geometric constraints and motion residual constraints applied to the 3D Gaussian point set of the environment in step 1, thereby directly guiding the UAV to complete the designated task.

[0076] As described above, although the invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A method for constructing an implicit semantic thinking chain mechanism for unmanned aerial vehicles (UAVs) infused with physical knowledge, characterized in that, The process includes the following: Step 1: The UAV, equipped with a monocular camera, acquires image frames in real time and inputs them into the multi-task online feedforward feature extractor of the edge computing unit. Based on an image encoder, a depth estimation network, and a residual flow network, the multi-task online feedforward feature extractor generates depth feature maps and residual flow features from the UAV's single-view perspective online. Utilizing a sparse representation-based 3D Gaussian sputtering mechanism, it reconstructs a 3D Gaussian point set for the local scene from the current viewpoint. Subsequently, through global average pooling and a point cloud encoder, the depth feature map and the 3D Gaussian point set are nonlinearly aggregated into a physical feature vector representing the physical geometry information of the UAV's single-view perspective. ; Step 2: Aggregate and map the raw task commands received by the UAV into fixed-length task semantic embeddings using the task command embedding module. Design a feature mapping module based on a multilayer perceptron to map physical feature vectors. As input, it is mapped and aligned to the preset input embedding dimension of the large language model through multi-layer nonlinear projection operations to obtain the physical knowledge embedding. ; Step 3: Embed task semantics using the multimodal knowledge fusion and implicit thought chain generation module. Embedded with physics knowledge Channel-level stitching is performed, followed by linear compression projection for fusion, to obtain the joint embedding tensor. ; and will jointly embed tensors The input is fed into a semantically aligned implicit inference generator, whose core structure consists of a lightweight Transformer encoder, to generate implicit inference tokens. ; Step 4: Transfer the implicit reasoning token As a differentiable prefix, it is injected into the front end of the input layer embedding sequence of a pre-trained large language model to form a joint input; in implicit inference tokens Guided by an attention weight transfer mechanism, the autoregressive generation of explicit reasoning text is skipped, accelerating the reasoning of thought chains and outputting UAV mission execution instructions that meet physical constraints in terms of kinematic and geometric topological feasibility. This is to guide drones to complete designated tasks.

2. The method for constructing an implicit semantic thinking chain mechanism for UAVs with injected physical knowledge according to claim 1, characterized in that, The specific process of step 1 is as follows: The multi-task online feedforward feature extractor first extracts basic image features from the input image frames using a lightweight image encoder based on the MobileNetV3 architecture. These features are then fed in parallel to the depth estimation network and the residual flow network for feedforward inference. The depth estimation network employs a ResNet-18-based encoder-decoder structure, utilizing ResNet-18 as the encoder to extract powerful geometric features. These features are then passed to the decoder for upsampling via skip connections. The depth estimation network outputs a single-view depth feature map. The feature map quantifies the distance from each pixel in the scene to the drone and serves as a key constraint for 3D geometric modeling. The residual flow network adopts the PWC-Net architecture, which takes two consecutive frames as input, performs pyramid feature extraction, feature warping, and cost volume calculation, and outputs a full-scene optical flow field including both rigid and non-rigid motion. From the full-scene optical flow field The non-rigid motion components are explicitly separated as features of the residual flow. ; Then and The input is fed into a 3D Gaussian sputtering mechanism based on sparse representation, utilizing... The pixel-level depth information is used as a supervision signal for geometric initialization to determine the initial position of the Gaussian point, while utilizing... The encoded motion residual information is used to predict the rotation, scale, and color attributes of each Gaussian point, thereby reconstructing a 3D Gaussian point set of the local scene for the current viewpoint within the framework of differential renderability. To characterize the spatiotemporal geometric information of the current environment of the UAV; Next to Perform global average pooling to obtain the deep feature vector. Simultaneously, a PointNet point cloud encoder based on local feature aggregation and hierarchical structure is employed to process the Gaussian point set. The physical properties of each Gaussian point are encoded and aggregated to obtain the Gaussian point feature vector. Ultimately and By concatenating the data along the channel dimension, we obtain a physical feature vector representing the single-view physical geometry information of the UAV. .

3. The method for constructing an implicit semantic thinking chain mechanism for UAVs with injected physical knowledge according to claim 1, characterized in that, The specific process of step 2 is as follows: The edge computing unit inputs the raw mission commands received by the UAV into the mission command embedding module. The mission command embedding module first performs word segmentation and word embedding processing on the commands, and then uses a lightweight Transformer encoder to aggregate and map the variable-length embedding sequence of mission commands into fixed-length mission semantic embeddings. Then, the feature mapping module is designed. The feature mapping module adopts a feature mapping module with... The multilayer perceptron structure consists of multiple layers. The multilayer perceptron first... As initial input Then, the initial input pass The hidden layer is iteratively processed, and the... Layer operations employ residual connections, layer normalization, and the GELU activation function to ensure information flow and nonlinear mapping capability during the projection process. The calculation formula is as follows: in, For the first The hidden state of the layer's output Indicates the first The hidden state of the layer's output The range of values ​​for is: ; , They are the first The weight matrix and bias vector of the layer, , , For the fixed dimensions of the hidden layers of a multilayer perceptron, This represents the GELU activation function. Representation layer normalization; finally, in the first... Layer, the multilayer perceptron performs the final linear projection, changing the dimension from Mapping to target dimension The calculation formula is: , , They represent the first The weight matrix and bias vector of the layer, Indicates the first The layer outputs hidden states; the final feature mapping module maps high-dimensional physical feature vectors. Transform into physical knowledge embeddings aligned with the input embedding dimension of the implicit inference generator. .

4. The method for constructing an implicit semantic thinking chain mechanism for UAVs with injected physical knowledge according to claim 1, characterized in that, The specific process of step 3 is as follows: Edge computing units utilize multimodal knowledge fusion and implicit thought chain generation modules to perform multimodal joint embedding fusion operations, embedding physical knowledge. Injected into task semantic embedding In this process, an input tensor for implicit reasoning is formed; specifically, a mechanism of linear compression projection after feature channel-level concatenation is adopted: and Concatenate along the feature channel dimension to obtain the joint embedding vector. : in, Indicates a splicing operation; Then, Through a single-layer linear projection Using weight matrix and Compress the dimensions and align them back to the target dimensions. The fused joint embedding tensor is obtained. : in, , ; Next, the joint embedding tensor As input, it is fed into a semantically aligned implicit inference generator. The core structure of the implicit inference generator consists of a lightweight Transformer encoder, which internally includes an input projection layer, multiple self-attention blocks, and an output sequence expansion layer. The input projection layer is responsible for receiving a single fixed-length vector. And map it to an implicit token with a preset length. Internal representation tensor ; Subsequently, the internal representation tensor After encoding through multiple layers of self-attention blocks, the latent reasoning relationships within the fused features are captured using a self-attention mechanism; finally, the output sequence expansion layer outputs a series of fixed-length implicit reasoning tokens. : in, This represents an implicit reasoning generator.

5. The method for constructing an implicit semantic thinking chain mechanism for UAVs with injected physical knowledge according to claim 4, characterized in that, The parameters Training and optimization are performed independently of the final large language model, while the core weights of the large language model remain frozen. The optimization objective is determined by the total loss, and the total loss for: , For predefined hyperparameters, For semantic alignment loss, Modeling loss for language.

6. The method for constructing an implicit semantic thinking chain mechanism for UAVs with injected physical knowledge according to claim 1, characterized in that, The specific process of step 4 is as follows: First, the multimodal knowledge fusion and implicit thought chain generation module obtains the length. Dimensions Implicit reasoning tokens And feed it into a pre-trained large language model In the input computation stream, to guide the large language model to quickly generate the final execution instructions. The injection mechanism employs an input layer embedding and concatenation strategy, using implicit reasoning knowledge as a differentiable prefix of the input sequence of the large language model. The specific process is as follows: First, the original mission instructions for the drone... Input to the word embedding layer of the large language model In this process, an embedded sequence of original task instructions is generated. , ,in, The word segmentation length of the instruction; then, the implicit inference token. spliced ​​together in the sequence dimension The front end forms the final joint input embedding sequence. : Subsequently, the joint input embedding sequence As input, it is fed into the large language model. Within the Transformer stack; ultimately, the large language model Implicit inference tokens Under strong guidance, the standard autoregressive decoding is executed to output the final execution command for the UAV mission: in, This indicates an autoregressive decoding operation.

Images