A method and system for training and recognizing large-scale AI models of low-altitude remote sensing images

Abstract

This application relates to the field of artificial intelligence technology and discloses a method and system for training and recognizing large AI models in low-altitude remote sensing images. The method includes: constructing an implicit 3D scene representation model based on multi-temporal remote sensing image sequences, and pre-training it with temporal consistency loss to learn high-dimensional semantic features that are robust to illumination and seasonal changes and independent of observation direction; using the pre-trained model to target and synthesize contextual views and close-up views for identifying targets with low confidence. The system includes a data acquisition module, a model construction module, a pre-training module, a fine-tuning module, and a target recognition module. This invention improves the recognition performance of large AI models in low-altitude remote sensing images for small targets, occluded targets, and targets with complex backgrounds.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and system for training and recognizing large-scale AI models of low-altitude remote sensing images. Background Technology

[0002] In recent years, large-scale AI models based on deep learning have made groundbreaking progress in the field of general computer vision and have been gradually applied to target recognition tasks in low-altitude remote sensing imagery, such as drone inspection, precision agriculture monitoring, and disaster assessment. In these applications, AI models are used to automatically identify and locate specific features or equipment in images.

[0003] However, when general-purpose visual models are directly applied to low-altitude remote sensing scenes, their performance often encounters bottlenecks. This is mainly due to the inherent complexity of low-altitude remote sensing imagery. Targets in these images are typically small in size, densely packed, and often partially obscured by vegetation, buildings, or other objects. Furthermore, due to dynamic changes in lighting, weather, and seasons, the same ground feature may exhibit drastically different appearances in images taken at different times, posing a significant challenge to the model's stable recognition.

[0004] Currently, the mainstream technical approach to solving the above problems typically involves fine-tuning a pre-trained general-purpose visual model using labeled low-altitude remote sensing imagery through supervised learning. While this method allows the model to adapt to the characteristics of remote sensing imagery data, its improvement effect is limited. On the one hand, the standard fine-tuning process struggles to effectively learn the deep semantic relationships between the target and its complex surrounding environment, leading to false positives or false negatives when the background is obfuscated. On the other hand, conventional data augmentation techniques, such as random rotation, cropping, and color dithering, are mostly pixel-level transformations and cannot realistically simulate appearance changes caused by the physical laws of the three-dimensional world, such as perspective, lighting, and seasonal changes. Therefore, the model's robustness to dynamic changes in the real world is insufficient, making it difficult to effectively identify targets that are difficult to distinguish due to poor imaging conditions.

[0005] Therefore, this invention proposes a method and system for AI large-scale model recognition and training of low-altitude remote sensing images to address the shortcomings of existing technologies. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a method and system for training and recognizing large AI models of low-altitude remote sensing images. This method solves the problems of low recognition accuracy and poor robustness of existing large AI models when recognizing low-altitude remote sensing images, due to the small size of the target, its susceptibility to occlusion, and drastic changes in appearance such as lighting and seasonal variations.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for training a large AI model for low-altitude remote sensing imagery, the method comprising the following steps:

[0008] S1. Obtain a low-altitude remote sensing image sequence covering at least two different time points of the same geographical area, wherein the low-altitude remote sensing image sequence includes multiple images and camera pose parameters corresponding to each image;

[0009] S2. Based on the acquired low-altitude remote sensing image sequence and camera pose parameters, an implicit three-dimensional scene representation model is constructed. The implicit three-dimensional scene representation model includes sub-models corresponding to at least two different time points, and a mapping relationship is established from three-dimensional spatial points to rendering attributes and high-dimensional semantic features.

[0010] S3. Apply temporal consistency loss to the implicit 3D scene representation model for pre-training to obtain a pre-trained implicit 3D scene representation model, wherein the temporal consistency loss is used to establish temporal anchor points for the high-dimensional semantic features of land features of a preset static category between sub-models at at least two different time points.

[0011] S4. Synthesize the context view and close-up view of the target image using a pre-trained implicit 3D scene representation model, and apply hierarchical feature distillation loss based on the context view and close-up view to fine-tune the AI ​​large model to obtain the fine-tuned AI large model.

[0012] S5. Based on the finely tuned AI model, target recognition is performed on the input low-altitude remote sensing images.

[0013] Preferably, in step S2, the step of constructing an implicit 3D scene representation model based on the acquired low-altitude remote sensing image sequence and camera pose parameters, wherein the implicit 3D scene representation model includes sub-models corresponding to at least two different time points, and establishing the mapping relationship between 3D spatial points and rendering attributes and high-dimensional semantic features includes:

[0014] The three-dimensional spatial points are taken as input and processed by the first neural network module to output volume density and a first feature vector. The high-dimensional semantic features are determined by the first feature vector. Since the input of the first neural network module does not include the observation direction, the high-dimensional semantic features are independent of the observation direction and are used to characterize the inherent semantic attributes of the three-dimensional spatial points.

[0015] The first feature vector and the observation direction used to render the three-dimensional spatial point are taken as common inputs and processed by the second neural network module to output a color value, so that the color value is related to the observation direction and is used to characterize the appearance effect related to the viewpoint.

[0016] The volume density output by the first neural network module and the color value output by the second neural network module together constitute the rendering attributes.

[0017] Preferably, in step S3, the implicit 3D scene representation model is pre-trained by applying a temporal consistency loss to obtain a pre-trained implicit 3D scene representation model. The step of using the temporal consistency loss to establish temporal anchor points for the high-dimensional semantic features of land features of a preset static category between at least two different time points includes:

[0018] Calculate the first time point The coordinate system of the sub-model to the second time point Rigid transformation matrix of the coordinate system of the sub-model , used to align the coordinate systems of sub-models in three-dimensional space;

[0019] A set of points belonging to a preset static category is obtained by sampling in three-dimensional space. The preset static category includes land features whose shape and location remain stable over a time span;

[0020] For the point set any three-dimensional point in Execution to obtain the first time point High-dimensional semantic features in the sub-model And obtain via the rigid transformation matrix After transformation at the second time point High-dimensional semantic features at corresponding positions in the sub-model ;

[0021] Based on the acquired high-dimensional semantic features belonging to sub-models at different time points and high-dimensional semantic features The temporal consistency loss is constructed by calculating the cosine similarity between two sets of high-dimensional semantic features. .

[0022] Preferably, in step S3, the implicit 3D scene representation model is pre-trained by applying a temporal consistency loss to obtain a pre-trained implicit 3D scene representation model. The step of using the temporal consistency loss to establish temporal anchor points for the high-dimensional semantic features of land features of a preset static category between at least two different time points further includes:

[0023] Define a photometric reconstruction loss The photometric reconstruction loss is used to minimize the pixel color difference between the pixel predicted color value synthesized by volume rendering technology based on rendering attributes and the pixel color value of the real image in the low-altitude remote sensing image sequence.

[0024] The timing consistency loss With photometric reconstruction loss By performing weighted summation, a pre-trained combined loss function is obtained. The calculation method is as follows:

[0025] ;

[0026] In the formula, and These are preset weighting coefficients;

[0027] The pre-trained combined loss function is minimized by employing a stochastic gradient descent optimization algorithm. The implicit 3D scene representation model is pre-trained.

[0028] Preferably, in step S4, the steps of synthesizing a context view and a close-up view of the target image using a pre-trained implicit 3D scene representation model, and applying hierarchical feature distillation loss based on the context view and the close-up view to fine-tune the AI ​​large model to obtain the fine-tuned AI large model include:

[0029] The context view and close-up view are respectively input into the AI ​​large model to extract the corresponding features;

[0030] Based on the extracted corresponding features, scale self-consistency constraints are constructed. and contextual constraints Wherein, the scale self-consistency constraint The contextual constraint is used to force the target image features extracted from the close-up view to be consistent with the target image features extracted from the corresponding target image region in the context view. Used to establish a predictive mapping relationship between environmental context features extracted from the context view and target image features;

[0031] The scale self-consistency constraint Contextual constraints Perform a weighted summation to form the hierarchical characteristic distillation loss. :

[0032] ;

[0033] In the formula, and Preset weighting coefficients used to balance the contributions of the two constraints;

[0034] Distillation loss of the hierarchical features Compared with supervised learning loss By combining these, we obtain the fine-tuned combined loss function. :

[0035] ;

[0036] In the formula, These are preset weighting coefficients used to balance supervised learning and feature distillation;

[0037] By minimizing the fine-tuned combination loss function The AI ​​model is then fine-tuned to complete the fine-tuning process.

[0038] Preferably, the step of constructing scale self-consistency constraints based on the extracted corresponding features includes:

[0039] The close-up view Feature extraction network input to large AI model To obtain close-up view features of the target image. ;

[0040] From the feature extraction network of the AI ​​large model For context view Output feature map In the context view, the corresponding region of the target image is extracted using region feature extraction operations. Contextual region features;

[0041] The scale self-consistency constraint is constructed by calculating the distance between the close-up view features and the context region features. .

[0042] Preferably, the step of constructing contextual relationship constraints based on the extracted corresponding features includes:

[0043] The true target features of the target image are obtained from the features extracted by the AI ​​big model for the context view. and the environmental context features of the area surrounding the target image. ;

[0044] Through auxiliary network The environmental context features As input, to generate the predicted target features ;

[0045] By calculating the true target features With the predicted target features The differences between them are used to construct the contextual relationship constraints. :

[0046] ;

[0047] In the formula, These are the real target features extracted from the context view; These are environmental context features extracted from the context view; It is an auxiliary network; These are target features predicted from environmental characteristics; This represents the square of the L2 norm.

[0048] Preferably, before step S4 is performed, the method further includes:

[0049] The AI ​​big model is used to identify labeled images in a low-altitude remote sensing image sequence to obtain the confidence level of the AI ​​big model in identifying labeled targets contained in the labeled images.

[0050] Determine whether the identification confidence level is lower than a preset confidence threshold;

[0051] If the identification confidence level is lower than the preset confidence level threshold, the labeled target is determined to be a difficult-to-identify sample, and step S4 is only performed on the difficult-to-identify sample.

[0052] Preferably, the method further includes:

[0053] Sampling is performed on regions in three-dimensional space that are not of a preset static category to obtain the three-dimensional spatial points to be detected;

[0054] For any three-dimensional spatial point obtained, the high-dimensional semantic features in the sub-model at the first time point are obtained, and the high-dimensional semantic features at the corresponding position in the sub-model at the second time point after coordinate system alignment are obtained.

[0055] The semantic change score is obtained by calculating the difference between two sets of high-dimensional semantic features belonging to sub-models at different time points.

[0056] When the semantic change score is greater than a preset change threshold, it is determined that a physical dynamic change has occurred in the region where the three-dimensional spatial point is located.

[0057] This invention also provides a low-altitude remote sensing image AI large model recognition and training system, the system comprising:

[0058] The data acquisition module is used to acquire a sequence of low-altitude remote sensing images covering the same geographical area at at least two different time points. The low-altitude remote sensing image sequence includes multiple images and camera pose parameters corresponding to each image.

[0059] The model building module is used to construct an implicit three-dimensional scene representation model based on the acquired low-altitude remote sensing image sequence and camera pose parameters. The implicit three-dimensional scene representation model includes sub-models corresponding to at least two different time points, and establishes a mapping relationship from three-dimensional spatial points to rendering attributes and high-dimensional semantic features.

[0060] The pre-training module is used to pre-train the implicit 3D scene representation model by applying temporal consistency loss to obtain a pre-trained implicit 3D scene representation model, wherein the temporal consistency loss is used to establish temporal anchor points for the high-dimensional semantic features of ground objects of a preset static category between the sub-models at at least two different time points.

[0061] The fine-tuning module is used to synthesize the context view and close-up view of the target image using a pre-trained implicit 3D scene representation model, and to apply hierarchical feature distillation loss based on the context view and close-up view to fine-tune the AI ​​large model, thereby obtaining the fine-tuned AI large model.

[0062] The target recognition module is used to identify targets in input low-altitude remote sensing images based on a finely tuned AI model.

[0063] This invention provides a method and system for training and recognizing large-scale AI models of low-altitude remote sensing images. It has the following beneficial effects:

[0064] 1. This invention applies a temporal consistency loss to the implicit 3D scene representation model during the pre-training stage, forcing the model to generate stable and consistent high-dimensional semantic features for preset static land cover categories at different time points (even with changes in lighting and seasons). This method effectively removes interference from non-essential factors, enabling the model to learn more essential, time-independent, and deep features of land covers, thus exhibiting stronger robustness and higher recognition accuracy when facing complex and ever-changing temporal remote sensing images.

[0065] 2. This invention utilizes a pre-trained model to synthesize contextual and close-up views of a target image, and applies hierarchical feature distillation loss for fine-tuning. This method forces the model to simultaneously learn the feature representation of the target at different scales (scale self-consistency constraint) and the relationship between the target and its surrounding environment (contextual relationship constraint). Through the effective fusion of multi-scale information and contextual information, it significantly improves the recognition performance of occluded, complex, or excessively small targets.

[0066] 3. This invention incorporates a judgment step before the fine-tuning process. The subsequent fine-tuning using synthetic data is only triggered when the AI ​​model's confidence level in recognizing the target image falls below a preset threshold. This on-demand, targeted fine-tuning strategy avoids unnecessary and costly fine-tuning of all samples, concentrating computational resources on addressing weaknesses in model recognition and achieving both economy and efficiency in the training process.

[0067] 4. This invention utilizes a model pre-trained with temporal consistency loss to detect changes by calculating the differences in high-dimensional semantic features of non-static category regions at different time points. Since the model has learned to distinguish between essential physical changes and non-essential apparent changes, significant differences in semantic features can directly and reliably indicate the actual physical changes occurring on land features, rather than apparent changes such as illumination or shadows. This provides a new technical means for applications such as land use monitoring and the detection of illegal buildings. Attached Figure Description

[0068] Figure 1 This is a flowchart illustrating the AI ​​large-scale model recognition and training method for low-altitude remote sensing images according to the present invention.

[0069] Figure 2 This is a schematic diagram of the implicit 3D scene representation model of the present invention;

[0070] Figure 3 This is a schematic diagram of the AI ​​large model hierarchical feature distillation fine-tuning process of the present invention;

[0071] Figure 4 This is a structural block diagram of the low-altitude remote sensing image AI large model recognition and training system of the present invention.

[0072] The module consists of: 10. Data acquisition module; 20. Model building module; 30. Pre-training module; 40. Fine-tuning module; and 50. Target recognition module. Detailed Implementation

[0073] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0074] Reference Figure 1 This invention provides a method for training a large AI model for low-altitude remote sensing imagery. This method constructs a temporally consistent 3D scene representation and uses this representation to fine-tune the large AI model. Specifically, the method may include the following steps:

[0075] S1. Obtain a low-altitude remote sensing image sequence covering at least two different time points of the same geographical area, wherein the low-altitude remote sensing image sequence includes multiple images and camera pose parameters corresponding to each image;

[0076] S2. Based on the acquired low-altitude remote sensing image sequence and camera pose parameters, an implicit three-dimensional scene representation model is constructed. The implicit three-dimensional scene representation model includes sub-models corresponding to at least two different time points, and a mapping relationship is established from three-dimensional spatial points to rendering attributes and high-dimensional semantic features.

[0077] S3. Apply temporal consistency loss to the implicit 3D scene representation model for pre-training to obtain a pre-trained implicit 3D scene representation model, wherein the temporal consistency loss is used to establish temporal anchor points for the high-dimensional semantic features of ground objects of a preset static category between the sub-models at at least two different time points.

[0078] S4. Synthesize the context view and close-up view of the target image using a pre-trained implicit 3D scene representation model, and apply hierarchical feature distillation loss based on the context view and close-up view to fine-tune the AI ​​large model to obtain the fine-tuned AI large model.

[0079] S5. Based on the finely tuned AI model, target recognition is performed on the input low-altitude remote sensing images.

[0080] This invention also provides a low-altitude remote sensing image AI large-scale model recognition and training system. (Refer to...) Figure 4 The system is configured to perform the aforementioned method and its structure may include: a data acquisition module 10, a model building module 20, a pre-training module 30, a fine-tuning module 40, and a target recognition module 50.

[0081] The data acquisition module 10 is used to acquire low-altitude remote sensing image sequences at least two different time points covering the same geographical area and the corresponding camera pose parameters.

[0082] The model building module 20 is connected to the output of the data acquisition module 10 and is used to build an implicit three-dimensional scene representation model based on the image sequence and camera pose parameters.

[0083] The pre-training module 30 is connected to the output of the model building module 20 and is used to pre-train the implicit 3D scene representation model by applying temporal consistency loss and other methods to obtain the pre-trained implicit 3D scene representation model.

[0084] The fine-tuning module 40 is connected to the output of the pre-training module 30 and is used to synthesize a view using the pre-trained model and apply hierarchical feature distillation loss based on the synthesized view to fine-tune the large AI model.

[0085] The target recognition module 50 is connected to the output of the fine-tuning module 40 and is used to perform target recognition tasks on the input low-altitude remote sensing images based on the fine-tuned AI model.

[0086] Reference Figure 1 Step S1 is the initial step of the method described in this embodiment of the invention. This step involves acquiring basic data for subsequent model construction and training.

[0087] Specifically, this step involves acquiring low-altitude remote sensing image sequences covering the same geographical area at at least two different time points. In one specific implementation, these image sequences can be obtained by using a drone (UAV) equipped with a camera to conduct multiple aerial photography operations on the same target area on different dates or in different seasons.

[0088] Each low-altitude remote sensing image sequence at each time point contains two core types of data:

[0089] Multiple image frames: These are a series of high-resolution digital images taken continuously from different positions and angles when a drone flies along a predetermined route.

[0090] Camera pose parameters: A set of parameters that precisely corresponds to each image frame. This set typically includes the camera's intrinsic and extrinsic parameter matrices. The intrinsic parameter matrix describes the camera's internal optical characteristics, such as focal length and principal point coordinates. The extrinsic parameter matrix describes the camera's position and orientation (i.e., rotation and translation information) in the global three-dimensional coordinate system when the image frame was captured. These pose parameters can be obtained through onboard inertial measurement unit (IMU) and global positioning system (GPS) data, and then precisely calculated and optimized using the subsequent Structure-from-Motion (SfM) algorithm.

[0091] Acquiring data from at least two different time points is a key prerequisite for the technical solution of this invention. This means that at least two independent aerial photography missions need to be performed on the same geographical area, with a significant time interval between the two missions, such as several months or years. This design is not simply data redundancy, but rather provides the necessary data foundation for constructing the temporal consistency loss in the subsequent step S3, enabling the model to learn which elements in the scene are dynamic elements that change over time and which are static elements that remain unchanged by comparing images from different time points.

[0092] Reference Figure 2 , Figure 2 This is a functional structure diagram of an implicit 3D scene representation model according to an embodiment of the present invention. The core of step S2 is to construct a function model for each time point image sequence that can map continuous 3D spatial coordinates to their physical and semantic attributes. This model is implemented by a neural network.

[0093] In one specific implementation, the implicit 3D scene representation model achieves its function through the concatenation and combination of two neural network modules, with the specific structure as follows:

[0094] The first neural network module (semantic and geometric encoding module): This module maps the coordinates of a point in three-dimensional space to the volume density and orientation-independent high-dimensional semantic features of that point. This module can be implemented using a multilayer perceptron (MLP). Its mathematical expression can be defined as a function... :

[0095] ;

[0096] In the formula, This represents the coordinates of a point in three-dimensional space, which serves as the input to this module. This represents the first neural network module, whose parameters are: ; The volume density represents the output, and is a scalar value. Volume density describes the volume at that point. The differential probability of light being blocked is used in the subsequent volume rendering process; The first feature vector representing the output is the high-dimensional semantic feature. This is a high-dimensional vector that encodes the semantic information of the land cover category to which the spatial point belongs.

[0097] This structural design incorporates high-dimensional semantic features. The generation process is completely separate from the observation direction. That is, for a fixed point in three-dimensional space, its essential semantic attributes (e.g., whether the point belongs to a building or vegetation) are constant and should not change with the observer's perspective. Therefore, the input to this module is only spatial coordinates. This ensures the semantic features of the output. It is independent of the observation direction.

[0098] The second neural network module (color decoding module): This module's function is to decode the color of a point under the specific observation direction based on the semantic features output by the first module. This module can also be implemented using a multilayer perceptron. Its mathematical expression can be defined as a function... :

[0099] ;

[0100] In the formula, The high-dimensional semantic features output by the first neural network module; The unit observation direction vector used to render this 3D spatial point describes the direction of the observation line of sight. This represents the second neural network module, whose parameters are: ; The color value representing the output is usually a three-dimensional vector, corresponding to the intensity of the three color channels RGB (red, green, and blue).

[0101] High-dimensional semantic features and observation direction Used together as input to generate color values Its technical purpose is to model orientation-related appearance effects (such as specular highlights and reflections) in a scene. The same point (with the same semantic features) When viewed from different directions, its color may vary due to physical phenomena such as light reflection. This design allows the model to accurately reconstruct a realistic light appearance.

[0102] Finally, the volume density output by the first module The color value output by the second module These collectively constitute the rendering attributes of the three-dimensional spatial point, which are used to synthesize a two-dimensional image using volumetric rendering techniques. The high-dimensional semantic features output by the first module... This serves as the semantic representation of that point, used for subsequent temporal consistency analysis and change detection. For each time point in the dataset, a complete implicit 3D scene representation sub-model is independently constructed and trained.

[0103] Reference Figure 1 Step S3 is the model pre-training stage of the method described in this embodiment of the invention. The goal of this stage is to optimize the implicit 3D scene representation model constructed in step S2, so that it can not only accurately reconstruct the appearance of the scene, but also learn time-invariant, essential semantic features of ground objects. This process is guided by a combined loss function.

[0104] Due to the different time points (e.g., the first time point) Second time point The camera pose parameters of an image sequence are typically calculated using independent Structure for Motion (SfM) processes, resulting in each sub-model having an independent coordinate system at each time point. To compare the properties of points at the same physical location in two sub-models, their coordinate systems must first be aligned.

[0105] This alignment process involves calculating a value from the first time point. coordinate system to the second time point Rigid transformation matrix of the coordinate system This is achieved by finding common feature points (i.e., corresponding points in 3D space) in image data at two time points, and then using these pairs of corresponding points to solve for a transformation relationship that minimizes the distance between them. This rigid transformation matrix... It consists of a rotation matrix and a translation vector, and is capable of... Precisely transform any point in the coordinate system to In the coordinate system.

[0106] The purpose of temporal consistency loss is to constrain the model so that it can generate stable and consistent high-dimensional semantic features for static features that have not changed physically in the scene at different time points, thereby ignoring apparent changes such as illumination and seasons.

[0107] The construction of this loss first requires sampling a set of points belonging to a preset static category in three-dimensional space. The preset static categories include land features whose shape and location remain stable over a time span, such as buildings, roads, and bridges. The regions for these categories can be determined based on prior geographic information data or a rough land use classification map.

[0108] For point sets any three-dimensional point in Using the calculated rigid transformation matrix From the time points respectively and High-dimensional semantic features are extracted from the sub-model. Then, a temporal consistency loss is constructed by calculating the cosine similarity between these two sets of features. The specific formula is as follows:

[0109] ;

[0110] In the formula, It is a three-dimensional spatial point sampled from the static category region; It is a point At the point of time The corresponding high-dimensional semantic features in the sub-model; It is a point After coordinate system alignment, at time point The high-dimensional semantic features at the corresponding positions in the sub-model; Represents the vector dot product. This represents the L2 norm of a vector.

[0111] By minimizing This can cause the semantic feature vectors output by the two sub-models for the same static physical point to tend to be consistent in direction, thereby achieving temporal stability of semantic representation.

[0112] To ensure that the model can accurately reconstruct the visual appearance of the scene while maintaining consistency in the learning timeline, this embodiment of the invention also introduces a photometric reconstruction loss.

[0113] Photometric reconstruction loss This is used to minimize the pixel color difference between the image rendered by the model and the actual remote sensing image. The calculation process is as follows: For any pixel in the training image, using volume-rendering technology, the integral is performed along the camera ray passing through that pixel, and the volume density predicted by the model is used. and color value Calculate the predicted color value of the pixel. Then, the predicted color value is compared with the actual pixel color value. The L2 norm distance between them. Its formula is expressed as:

[0114] ;

[0115] In the formula, Represents a camera ray. It is the set of all rays in the current training batch.

[0116] Finally, the temporal consistency loss and photometric reconstruction loss are weighted and summed to obtain a pre-trained combined loss function. :

[0117] ;

[0118] in, and These are preset weighting coefficients used to balance the contributions of the two losses.

[0119] By employing optimization algorithms such as stochastic gradient descent, the pre-trained combined loss function is minimized. This allows for the pre-training of the implicit 3D scene representation model. The pre-trained model possesses both high-fidelity scene appearance reconstruction capabilities and the ability to understand the temporal consistency of static features.

[0120] Reference Figure 1 and Figure 3 Step S4 is the AI ​​large model fine-tuning stage of the method described in the embodiments of the present invention. Figure 3 This is a schematic diagram of the hierarchical feature distillation fine-tuning process of an AI large model according to an embodiment of the present invention. The core of this stage is to use the implicit 3D scene representation model pre-trained in step S3 to generate targeted training data and apply specific constraints to optimize the AI ​​large model, so as to improve its ability to process difficult-to-identify targets.

[0121] The fine-tuning process in this embodiment of the invention is not performed indiscriminately on all training samples, but rather employs a targeted triggering mechanism to improve training efficiency. Specifically, before performing this step, the method further includes the following pre-operations:

[0122] First, the current large AI model (i.e., the model to be fine-tuned) is used to identify the labeled images in the low-altitude remote sensing image sequence. For each labeled target in the image, the large AI model will output a corresponding recognition confidence score.

[0123] Next, the confidence level of the identification is compared with a preset confidence threshold.

[0124] If the recognition confidence of a target is below a certain threshold, the target is classified as a difficult-to-recognize sample. Subsequent view synthesis and hierarchical feature distillation fine-tuning steps will only be performed on images of these targets classified as difficult-to-recognize samples. This mechanism concentrates computational resources on optimizing the weak links of the model, avoiding unnecessary retraining on samples that the model can already accurately recognize, thereby improving the efficiency of the overall training process.

[0125] For each difficult-to-identify target identified by the mechanism, this step will use a pre-trained implicit 3D scene representation model to synthesize two new training views:

[0126] Context view Define the pose of a virtual camera, whose field of view can completely cover the target itself and its surrounding related environmental areas. Using volume rendering technology, an image is generated from this virtual camera pose using an implicit model; this is the context view.

[0127] Close-up view Define the pose of another virtual camera that is very close to the target, so that the target dominates the field of view and can display more details. The corresponding image is then generated using volume rendering technology, resulting in a close-up view.

[0128] Since both types of views are generated from the same implicit model constrained by geometric and temporal consistency, they are perfectly aligned in 3D space, providing a high-quality data foundation for subsequent construction of cross-scale feature constraints.

[0129] Reference Figure 3 Get the context view and close-up view Then, these features are fed into a large AI model to extract features, and a hierarchical feature distillation loss is constructed. This is used to fine-tune the model. The loss consists of two parts:

[0130] Scale self-consistency constraint The goal of this constraint is to enable large AI models to extract consistent features from the same target at different observation scales. Its construction process is as follows: [The text then abruptly shifts to a seemingly unrelated topic:] ...close-up view... Input a large AI model to obtain high-resolution close-up view features of the target; simultaneously, use a context view. Input a large AI model and extract the context region features corresponding to the target region from its output feature map. Construct the constraint by calculating the distance between these two feature sets. The formula is expressed as:

[0131] ;

[0132] In the formula, Feature extraction networks representing large AI models; It is a close-up view feature; It is the feature map of the context view; It is the location area of ​​the target in the context view (e.g., the bounding box); It is an operation that accurately extracts region features from a feature map; Represents the L1 norm.

[0133] Contextual constraints The goal of this constraint is to enable the large AI model to learn to utilize environmental information surrounding the target to aid in identification. Its construction process is as follows: starting with the large AI model targeting the context view... The extracted features include target features from the target image and environmental context features from the surrounding area. Then, an auxiliary network is used to establish a predictive mapping from environmental context features to target features to generate predicted target features. This constraint is constructed by calculating the difference between the predicted target features and the true target features. The formula is expressed as:

[0134] ;

[0135] In the formula, These are the real target features extracted from the context view; These are environmental context features extracted from the context view; It is an auxiliary network; These are target features predicted from environmental characteristics; This represents the square of the L2 norm.

[0136] By weighting and combining the two constraints mentioned above, we obtain the hierarchical feature distillation loss. :

[0137] ;

[0138] In the formula, and These are preset weighting coefficients used to balance the contributions of the two constraints.

[0139] Finally, the hierarchical feature distillation loss is compared with a standard supervised learning loss. (For example, classification cross-entropy loss or bounding box regression loss calculated based on manual annotation) are combined to form the final fine-tuned combined loss function. :

[0140] ;

[0141] In the formula, These are preset weight coefficients used to balance supervised learning and feature distillation. By minimizing this fine-tuning combined loss function, the parameters of the large AI model are updated, thus completing the fine-tuning process.

[0142] Reference Figure 1 Step S5 is the final application stage of the method described in this embodiment of the invention, that is, using the trained model to perform the actual recognition task.

[0143] In this step, the large AI model obtained after thorough fine-tuning based on hierarchical feature distillation loss in step S4 is deployed as the final target recognition model. The parameters of this model have been fully optimized through the aforementioned pre-training and fine-tuning process.

[0144] The specific execution process is as follows: one or more new low-altitude remote sensing images to be identified are input into the fine-tuned AI model. After receiving the input images, the model performs a complete forward propagation calculation and outputs the identification results of one or more targets in the images.

[0145] The recognition results, depending on the specific task settings, may include the category information (e.g., vehicles, buildings) of each recognized target, its location information in the image (e.g., bounding boxes defined by coordinates or pixel-level segmentation masks), and the corresponding recognition confidence score.

[0146] Because the AI ​​model has learned how to integrate multi-scale information and environmental context information through scale self-consistency constraints and contextual relationship constraints during the fine-tuning process, it has higher recognition accuracy and robustness for small targets, partially occluded targets, or targets in complex backgrounds when performing the target recognition task in this step compared to before fine-tuning.

[0147] The implicit 3D scene representation model pre-trained in step S3 of this embodiment of the invention can be used not only for fine-tuning of subsequent large AI models, but also directly applied to high-precision dynamic change detection tasks. This is an application of the trained model, and its purpose is to detect changes, not to train the model. This is fundamentally different from the training process in step S3 that uses static points for temporal constraints.

[0148] The specific implementation steps of this dynamic change detection method are as follows:

[0149] First, regions in 3D space that are not classified as static are sampled to obtain a set of 3D spatial points to be detected. These regions are areas where physical changes may occur, such as parking lots, farmland, construction sites, or vegetation-covered areas.

[0150] Next, for any three-dimensional point in this point set... Perform the following operations:

[0151] From the first point of time In the sub-model, obtain the high-dimensional semantic features of that point. .

[0152] Using the calculated rigid transformation matrix Point Perform a coordinate transformation to obtain its value at the second time point. Corresponding position in coordinate system .

[0153] From the second time point In the sub-model, obtain the high-dimensional semantic features of the corresponding position. .

[0154] Then, the difference value between the two sets of high-dimensional semantic features obtained above and belonging to the sub-models at different time points is calculated. In one implementation, this difference value can be defined as a semantic change score. This is obtained by calculating the cosine distance between two semantic feature vectors:

[0155] ;

[0156] In the formula, the definitions of each symbol are the same as those in the formula for timing consistency loss.

[0157] Finally, based on the calculated semantic change score This determines whether the area where the point is located has undergone dynamic changes. A change threshold can be set, when... When the value exceeds this threshold, a decision point is made. Significant physical changes occurred at that location.

[0158] The reason this method can reliably indicate real physical changes is that, through pre-training with the temporal consistency loss in step S3, the model has learned to output stable and consistent semantic features for static features (such as buildings) under different lighting and seasonal conditions. This means the model can distinguish between apparent changes caused by non-essential factors such as lighting and essential changes caused by alterations in the objects themselves (such as the appearance or disappearance of vehicles, or the demolition or construction of buildings). Therefore, when there is a significant difference in semantic features between two time points (i.e.,...) When the value is high, it directly reflects that the semantic attribution of the point has changed, thus indicating that a real physical change has occurred at that location, effectively filtering out the interference of false change information caused by factors such as lighting and shadows.

[0159] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for identifying and training an AI large model for low-altitude remote sensing images, characterized in that, The method comprises the following steps: S1, acquiring a low-altitude remote sensing image sequence covering the same geographical area at at least two different time points, the low-altitude remote sensing image sequence comprising a plurality of images and a camera pose parameter corresponding to each image; S2, constructing an implicit three-dimensional scene representation model based on the acquired low-altitude remote sensing image sequence and camera pose parameter, the implicit three-dimensional scene representation model comprising sub-models corresponding to at least two different time points respectively, and establishing a mapping relationship between a three-dimensional space point and a rendering attribute and a high-dimensional semantic feature; S3, pre-training is carried out on the implicit three-dimensional scene representation model by applying a time sequence consistency loss, to obtain a pre-trained implicit three-dimensional scene representation model, wherein the time sequence consistency loss is used to establish a time sequence anchor point for the high-dimensional semantic features of the pre-set static category of ground objects between the sub-models at at least two different time points; wherein a rigid transformation matrix of the coordinate system of the sub-model at the first time point is calculated to the coordinate system of the sub-model at the second time point ; and the coordinate systems of the sub-models are aligned in the three-dimensional space . Sampling a point set belonging to a preset static category in three-dimensional space The preset static category includes a ground object with stable morphology and position within a time span For the point set any three-dimensional point in Execution to obtain the first time point High-dimensional semantic features in the sub-model And obtain via the rigid transformation matrix After transformation at the second time point High-dimensional semantic features at corresponding positions in the sub-model ; based on the acquired high-dimensional semantic features of the sub-models belonging to different time points and high-dimensional semantic features , the time sequence consistency loss is constructed by calculating the cosine similarity between the two groups of high-dimensional semantic features ; defining a photometric reconstruction loss for minimizing pixel color differences between pixel predicted color values synthesized by a volume rendering technique based on rendering properties and pixel color values of real images in a low-altitude remote sensing image sequence; The timing consistency loss The photometric reconstruction loss The pre-training combined loss function is obtained by weighted summation The calculation method is: ； In the formula, and are preset weight coefficients; minimizing the pre-training combined loss function by employing a stochastic gradient descent optimization algorithm completing pre-training of the implicit three-dimensional scene representation model S4, synthesizing a contextual view and a close-up view of a target image using a pre-trained implicit three-dimensional scene representation model, and applying a hierarchical feature distillation loss based on the contextual view and the close-up view to fine-tune an AI large model to obtain a fine-tuned AI large model; S5, performing target recognition on an input low-altitude remote sensing image based on the fine-tuned AI large model.

2. The low-altitude remote sensing image AI large model identification training method according to claim 1, characterized in that, In step S2, based on the acquired low-altitude remote sensing image sequence and camera pose parameter, an implicit three-dimensional scene representation model is constructed, the implicit three-dimensional scene representation model comprising sub-models corresponding to at least two different time points respectively, and a mapping relationship between a three-dimensional space point and a rendering attribute and a high-dimensional semantic feature is established, and the step comprises: taking a three-dimensional space point as input, processing it through a first neural network module to output a body density and a first feature vector, wherein the high-dimensional semantic feature is determined by the first feature vector, and since the input of the first neural network module does not include an observation direction, the high-dimensional semantic feature is independent of the observation direction, and is used to represent the inherent semantic attribute of the three-dimensional space point; taking the first feature vector and an observation direction used for rendering the three-dimensional space point as common input, processing it through a second neural network module to output a color value, so that the color value is related to the observation direction, and is used to represent the appearance effect related to the viewing angle; the body density output by the first neural network module and the color value output by the second neural network module jointly constitute the rendering attribute.

3. The low-altitude remote sensing image AI large model identification training method according to claim 1, characterized in that, In step S4, the contextual view and the close-up view of the target image are synthesized using a pre-trained implicit three-dimensional scene representation model, and a hierarchical feature distillation loss is applied based on the contextual view and the close-up view to fine-tune an AI large model to obtain a fine-tuned AI large model, and the step comprises: inputting the contextual view and the close-up view into the AI large model respectively to extract corresponding features; constructing a scale self-consistency constraint based on the corresponding features extracted and a context relationship constraint wherein the scale self-consistency constraint for forcing the target image features extracted from the close-up view to be consistent with the target image features extracted from the corresponding target image region in the context view, the context relationship constraint for establishing a predictive mapping relationship between the environmental context features extracted from the context view and the target image features; applying the scale consistency constraint contextual relationship constraint performing a weighted sum to form the hierarchical feature distillation loss : ； wherein and are preset weight coefficients for balancing the contributions of the two constraints; The hierarchical feature distillation loss is combined with a supervised learning loss to obtain a fine-tuning combined loss function : ； In the formula, is a preset weight coefficient for balancing supervised learning and feature distillation; by minimizing the fine-tuning ensemble loss function performing fine-tuning of the AI large model, thereby completing the fine-tuning process.

4. The low-altitude remote sensing image AI large model identification training method according to claim 3, characterized in that, the step of constructing a scale self-consistency constraint based on the extracted corresponding features comprises: the close-up view feature extraction network input to an AI large model to obtain a close-up view feature of the target image ; from the al large model for a contextual view output feature map In some embodiments, a contextual region feature of a target image in the contextual view is extracted using a region feature extraction operation. ​ constructing the scale consistency constraint by computing a distance between the close-up view feature and the context region feature .

5. The low-altitude remote sensing image AI large model identification training method according to claim 3, characterized in that, the step of constructing a contextual relationship constraint based on the extracted corresponding features comprises: from the features extracted by the AI large model for the context view, respectively obtaining real target features of the target image , and environment context features of the surrounding area of the target image ; By an auxiliary network The environmental context features As input to generate predicted target features ; by computing a difference between the real target feature and the predicted target feature to build the contextual relationship constraint : ； wherein are real target features extracted from the contextual view; are environmental contextual features extracted from the contextual view; is an auxiliary network; are target features predicted by the environmental features; represents the square of the L2 norm. 6.The low-altitude remote sensing image AI large model identification training method according to claim 1, characterized in that, Before step S4 is executed, the method further comprises: using the AI large model to identify a labeled image in the low-altitude remote sensing image sequence to obtain an identification confidence of the AI large model for a labeled target contained in the labeled image; determining whether the identification confidence is lower than a preset confidence threshold; if the identification confidence is lower than the preset confidence threshold, the labeled target is determined as a difficult-to-identify sample, and step S4 is only performed for the difficult-to-identify sample.

7. The low-altitude remote sensing image AI large model identification training method according to claim 1, characterized in that, The method further comprises: Sampling a region of a non-pre-set static category in a three-dimensional space to obtain a three-dimensional space point to be detected; For any three-dimensional space point obtained, a high-dimensional semantic feature in a sub-model at a first time point is obtained, and a high-dimensional semantic feature in a corresponding position of the sub-model after coordinate system alignment at a second time point is obtained; A semantic change score is obtained by calculating the difference between the two groups of high-dimensional semantic features obtained from sub-models at different time points; When the semantic change score is greater than a pre-set change threshold, it is determined that the region where the three-dimensional space point is located has undergone a physical dynamic change.

8. A low-altitude remote sensing image AI large model identification training system applied to the method of any one of claims 1-7, characterized in that, The system comprises: A data acquisition module is configured to acquire a low-altitude remote sensing image sequence covering the same geographical region at different time points, the low-altitude remote sensing image sequence comprising a plurality of images and corresponding camera pose parameters for each image; A model construction module is configured to construct an implicit three-dimensional scene representation model based on the acquired low-altitude remote sensing image sequence and camera pose parameters, the implicit three-dimensional scene representation model comprising sub-models corresponding to different time points, and a mapping relationship between a three-dimensional space point and a rendering attribute and a high-dimensional semantic feature; A pre-training module is configured to pre-train the implicit three-dimensional scene representation model by applying a temporal consistency loss to obtain a pre-trained implicit three-dimensional scene representation model, wherein the temporal consistency loss is used to establish a temporal anchor point for the high-dimensional semantic feature of a pre-set static category of a geographical object between the sub-models at different time points; A fine-tuning module is configured to synthesize a context view and a close-up view of a target image using the pre-trained implicit three-dimensional scene representation model, and to fine-tune an AI large model based on the context view and the close-up view by applying a hierarchical feature distillation loss to obtain a fine-tuned AI large model; An identification module is configured to perform target identification on an input low-altitude remote sensing image based on the fine-tuned AI large model.

Images