Method and device for interpreting low-altitude data

Abstract

The invention provides a low-altitude data interpretation method and device, and the method comprises the steps: carrying out the calculation based on the data quality features of the low-altitude data of each modal, and obtaining the modal confidence; fusing the multi-modal low-altitude data based on the modal confidence coefficient and the information entropy of each modal low-altitude data to obtain a fused low-altitude data feature vector; and inputting the fused low-altitude data feature vector into an interpretation model to obtain an interpretation result. According to the method provided by the invention, the multi-modal low-altitude data is fused through the modal confidence coefficient and the information entropy of the low-altitude data of each modal to obtain the dynamically-evaluated and self-adaptive fused low-altitude data feature vector, and then the large language model is used for interpretation, so that the advantages of different data sources can be fully mined, high-quality fused data is formed, and the reliability of the multi-modal low-altitude data is improved. And accurate interpretation is realized by virtue of the powerful capability of a large model, so that the automation level, the accuracy and the reliability of intelligent interpretation of the low-altitude data are integrally improved.

Description

Technical Field

[0001] This invention relates to the field of low-altitude data processing technology, and in particular to a method and apparatus for interpreting low-altitude data. Background Technology

[0002] With the rapid development of low-altitude detection technologies such as drones and low-altitude satellites, low-altitude data has become a core source of information about the Earth's surface. Low-altitude data interpretation, as a crucial step in data value transformation, requires accurate identification, classification, and attribute extraction of ground targets to support subsequent decision-making. Current low-altitude data interpretation primarily employs fixed-weight fusion of multimodal low-altitude data, followed by interpretation of the fused multimodal low-altitude data.

[0003] However, current interpretation methods for multimodal data fusion lack intelligent adaptability and fail to consider the quality differences between different modal data. This results in the fused data not being able to fully leverage the advantages of each modality, leading to lower accuracy in the interpretation results of multimodal low-altitude data. Summary of the Invention

[0004] This invention provides a method and apparatus for interpreting low-altitude data, which addresses the shortcomings of existing technologies where multimodal data fusion lacks intelligent adaptability and fails to consider quality differences, resulting in low accuracy of low-altitude data interpretation results.

[0005] This invention provides a method for interpreting low-altitude data, comprising: Acquire multimodal low-altitude data; Based on the data quality characteristics of each modality of low-altitude data, the modal confidence of each modality of low-altitude data is calculated; Based on the modal confidence and information entropy of each modal low-altitude data, the multimodal low-altitude data is fused to obtain a fused low-altitude data feature vector; The fused low-altitude data feature vector is input into the interpretation model to obtain the interpretation result of the multimodal low-altitude data; the interpretation model is constructed based on a large-scale language model.

[0006] According to a method for interpreting low-altitude data provided by the present invention, the multimodal low-altitude data includes at least two of the following: optical image low-altitude data, point cloud low-altitude data, and spectral low-altitude data. The modal confidence scores of the low-altitude data are calculated based on the data quality characteristics of each modality, including: Based on at least one of the image sharpness and image contrast of the optical image low-altitude data, the modal confidence of the optical image low-altitude data is calculated. Based on at least one of the point cloud density and point cloud completeness of the point cloud low-altitude data, the modal confidence of the point cloud low-altitude data is calculated. Based on at least one of the smoothness of the spectral curve and the completeness of the spectral characteristic bands of the low-altitude spectral data, the modal confidence of the low-altitude spectral data is calculated.

[0007] According to a method for interpreting low-altitude data provided by the present invention, the step of determining the information entropy of each modality of low-altitude data includes: Based on the grayscale distribution entropy of the low-altitude optical image data, the information entropy of the low-altitude optical image data is calculated. Based on the spatial distribution entropy of the point cloud low-altitude data, the information entropy of the point cloud low-altitude data is calculated. The information entropy of the low-altitude spectral data is calculated based on the band information entropy of the low-altitude spectral data.

[0008] According to a method for interpreting low-altitude data provided by the present invention, the step of inputting the fused low-altitude data feature vector into an interpretation model to obtain the interpretation result of the multimodal low-altitude data includes: The fused low-altitude data feature vector is input into the interpretation model to obtain the model inference result output by the interpretation model; the model inference result includes the model interpretation category, the target space coordinates of the model interpretation category, and the target confidence score; Based on the rule verification library, the model's interpretation category and the target space coordinates are matched to obtain the rule matching score of the model's inference result; Based on the target confidence score of the model inference result and the rule matching score, the double verification score is calculated. The interpretation result is obtained by filtering from the model inference results based on the double verification score.

[0009] According to a method for interpreting low-altitude data provided by the present invention, the acquisition of multimodal low-altitude data includes: Collect raw multimodal low-altitude data; The original multimodal low-altitude data is denoised to obtain denoised low-altitude data; The noise-reduced low-altitude data is aligned to a spatial coordinate system to obtain aligned low-altitude data; The aligned low-altitude data is converted into a preset standardized format to obtain the converted preset standardized format.

[0010] According to a method for interpreting low-altitude data provided by the present invention, the method further includes: Obtain the current knowledge graph; The low-altitude knowledge elements were extracted from the interpretation results; Based on the aforementioned low-altitude knowledge elements, the current knowledge graph is updated to obtain an updated knowledge graph.

[0011] According to a method for interpreting low-altitude data provided by the present invention, the step of obtaining the interpretation model includes: Acquire the large-scale language model and sample multimodal low-altitude data; The large language model is pre-trained and fine-tuned based on the sample multimodal low-altitude data to obtain the interpretation model; The interpretation model can be used for any one of the following: land cover classification, target recognition, and anomaly detection.

[0012] The present invention also provides a low-altitude data interpretation apparatus, comprising: Acquisition unit, acquires multimodal low-altitude data; The modal confidence determination unit calculates the modal confidence of each modal low-altitude data based on the data quality characteristics of each modal low-altitude data. The multimodal data fusion unit fuses the multimodal low-altitude data based on the modal confidence and information entropy of each modality to obtain a fused low-altitude data feature vector. The interpretation unit inputs the fused low-altitude data feature vector into the interpretation model to obtain the interpretation result of the multimodal low-altitude data; the interpretation model is constructed based on a large-scale language model.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the low-altitude data interpretation method as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the low-altitude data interpretation method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the low-altitude data interpretation method as described above.

[0016] The low-altitude data interpretation method and apparatus provided by this invention fuses multimodal low-altitude data by using modal confidence and information entropy of each modality of low-altitude data to obtain dynamically evaluated and adaptive fused low-altitude data feature vectors. Then, a large-scale language model is used for interpretation, which can fully explore the advantages of different data sources, form high-quality fused data, and achieve accurate interpretation by leveraging the powerful capabilities of the large-scale model. Overall, it improves the automation level, accuracy and reliability of intelligent interpretation of low-altitude data. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts illustrating the low-altitude data interpretation method provided by the present invention; Figure 2 This is the second flowchart illustrating the low-altitude data interpretation method provided by the present invention; Figure 3 This is a schematic diagram of the low-altitude data interpretation device provided by the present invention; Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0020] To address the aforementioned problems, this invention provides a method for interpreting low-altitude data, enabling adaptive fusion of multimodal low-altitude data, thereby improving the accuracy of interpretation based on the fused multimodal low-altitude data. Figure 1 This is one of the flowcharts illustrating the low-altitude data interpretation method provided by the present invention, such as... Figure 1 As shown, the method includes: Step 110: Obtain multimodal low-altitude data.

[0021] Here, multimodal low-altitude data refers to a collection of data acquired from a low-altitude perspective, originating from different types of sensors, used to describe the same geographical area or target. Its core function is to provide complementary information about the target or scene, thereby achieving a more comprehensive and accurate understanding than a single data source.

[0022] It should be noted that multimodal low-altitude data includes, but is not limited to: optical imagery data that provides rich texture, color, and two-dimensional spatial information, such as visible light aerial photographs; laser point cloud data that provides high-precision three-dimensional structure and elevation information; and spectral data that provides information on the material and physicochemical properties of ground features.

[0023] Specifically, this data can be acquired through platforms such as drones, low-altitude aircraft, or low-altitude remote sensing satellites equipped with corresponding sensors. For example, a drone can first take aerial photographs of the target area to obtain high-resolution optical images of the area in TIFF format. Simultaneously, an airborne lidar can scan the same area to obtain laser point cloud data in LAS format. If analysis of vegetation or soil composition is required, spectral data in ENVI format can also be acquired using a hyperspectral imager. After acquiring the raw data, it is necessary to preprocess the data to provide high-quality input for subsequent fusion processing, thereby obtaining rich, diverse, and high-quality multimodal low-altitude data.

[0024] Step 120: Based on the data quality characteristics of each modal low-altitude data, calculate the modal confidence of each modal low-altitude data.

[0025] Here, data quality characteristics refer to a series of quantitative indicators that can objectively measure the quality of each modality of data, and their role is to provide a basis for assessing the reliability of the data. Additionally, modality confidence here refers to a comprehensive quantitative score calculated based on data quality characteristics, used to represent the reliability or contribution value of the current modality data in the multimodal fusion process. Understandably, a higher modality confidence score indicates better data quality for that modality, and it should be given higher weight in subsequent fusion decisions.

[0026] Specifically, for data of different modalities, different data quality features can be used for evaluation. For example, for optical images, image sharpness and contrast can be calculated as quality features; the higher the sharpness and contrast, the higher the modal confidence. For laser point cloud data, point cloud density and data integrity can be used as quality features; the denser the point cloud and the fewer the missing data regions, the higher the modal confidence. For spectral data, the smoothness of the spectral curve and the integrity of key characteristic bands can be analyzed; the smoother the curve and the more complete the characteristic band information, the higher the modal confidence.

[0027] Then, the data quality characteristics of each modality of low-altitude data can be calculated using the corresponding algorithm, and then normalized to a preset interval, such as [0, 1], as the modal confidence of the low-altitude data of that modality.

[0028] It should be noted that by conducting independent quality assessments on each modality of data, a quantitative measurement of data reliability is achieved. This enables dynamic and differentiated treatment of data of different quality, avoiding negative interference from poor-quality data on the overall analysis, thereby improving the intelligence and effectiveness of subsequent data fusion.

[0029] Step 130: Based on the modal confidence and information entropy of each modal low-altitude data, the multimodal low-altitude data is fused to obtain a fused low-altitude data feature vector.

[0030] Here, information entropy is an indicator used to measure the uncertainty or information content of data. It can be understood that the more explicit the effective information and the less redundant information contained in the data, the lower its information entropy value usually is. Therefore, information entropy and modality confidence can complement each other to jointly determine the fusion weight of each modality. In other words, low-altitude data for each modality should not only be of high quality but also contain a large amount of information.

[0031] Furthermore, the fused low-altitude data feature vector here refers to a unified, more informative data representation, such as a high-dimensional feature vector, formed by weighting and combining data or features from multiple modalities according to calculated dynamic weights. Its purpose is to serve as input to subsequent interpretation models, enabling them to make more accurate judgments based on this comprehensive and complementary information.

[0032] Specifically, first, the information entropy of each modality is calculated. For example, the information entropy of an optical image can be calculated using its grayscale distribution entropy; the information entropy of a laser point cloud can be calculated using its spatial distribution entropy; and the information entropy of spectral data can be calculated using its band information entropy. Then, combining the modality confidence obtained in the previous step with the currently calculated information entropy, a dynamic fusion weight is calculated for each modality. A preferred method for calculating the weight is that the weight is directly proportional to the modality confidence and inversely proportional to the information entropy. The fusion weight here can be calculated based on the following formula, as shown below: ; In the formula, This represents the fusion weight of the i-th modality of low-altitude data; This represents the modal confidence level of the i-th modal low-altitude data. The information entropy of the i-th modal low-altitude data is represented; n=3, j is the modal index, j=1, 2, 3, which correspond to optical image low-altitude data, point cloud low-altitude data, and spectral low-altitude data, respectively; This represents the modal confidence of the j-th modal low-altitude data; Let represent the information entropy of the j-th modal low-altitude data.

[0033] Finally, the feature vectors extracted from each modality are multiplied by their corresponding fusion weights and then summed to generate the final fused low-altitude data feature vector. This fused low-altitude data feature vector can be calculated using the following formula, as shown below: ; In the formula, This represents the feature vector of the fused low-altitude data; n represents the total number of modes, and i represents the mode number; This represents the fusion weight of the i-th modality of low-altitude data; This represents the feature vector extracted from the low-altitude data of the i-th mode; It should be noted that by fusing the modal confidence and information entropy of low-altitude data from each modality, multimodal low-altitude data is obtained, resulting in a fused low-altitude data feature vector. This constructs a dynamic adaptive fusion mechanism that can simultaneously consider data quality and information content. It can intelligently assign higher weights to high-quality and information-rich modalities and suppress the contributions of low-quality or information-redundant modalities, thereby generating fused data with complementary information and integrated advantages, which greatly improves the data quality and information density input into the interpretation model.

[0034] Step 140: Input the fused low-altitude data feature vector into the interpretation model to obtain the interpretation result of the multimodal low-altitude data; the interpretation model is constructed based on a large-scale language model.

[0035] Here, the interpretation model is a deep learning model built upon a large-scale language model architecture and fine-tuned with knowledge from the low-altitude data domain. Its role is to leverage the powerful feature learning, logical reasoning, and generalization capabilities of the large-scale model to perform deep analysis and understanding of the input fused data, thereby completing complex interpretation tasks such as feature identification, target classification, or anomaly detection. Furthermore, the interpretation result here is the final judgment output by the interpretation model after analyzing the input data. This typically includes a category determination for the target area, such as "farmland," "buildings," or "water bodies," and may be accompanied by a confidence score to indicate the model's degree of certainty in its judgment.

[0036] Specifically, first, the fused low-altitude data feature vector generated in the previous step is formatted, such as by adjusting its dimensions or normalizing it, to meet the input requirements of the interpretation model. Then, the processed feature vector is input into a pre-loaded and fine-tuned multimodal large-scale language model. It should be noted that this model internally uses complex structures such as its attention mechanism to perform layer-by-layer operations and inferences on the features, capturing deep correlations between different features. Finally, the model outputs a probability distribution for each possible category. The category with the highest probability can be selected as the final interpretation result, and this highest probability value is used as the confidence level of the interpretation result. For example, the model might output: interpretation category is "building", confidence level is 0.95.

[0037] It should be noted that, compared with traditional methods, the powerful learning and generalization capabilities of large-scale language models significantly improve the accuracy and robustness of low-altitude data interpretation, enabling a better understanding of complex scenes, reducing reliance on massive amounts of specific scene-annotated data, and more flexibly adapting to different interpretation tasks and environments, ultimately outputting high-precision interpretation results.

[0038] The method provided in this invention fuses multimodal low-altitude data by using modal confidence and information entropy of low-altitude data of each modality to obtain dynamically evaluated and adaptive fused low-altitude data feature vectors. Then, it uses a large language model for interpretation, which can fully explore the advantages of different data sources, form high-quality fused data, and achieve accurate interpretation by leveraging the powerful capabilities of the large model. Overall, it improves the automation level, accuracy and reliability of intelligent interpretation of low-altitude data.

[0039] Based on any of the above embodiments, multimodal low-altitude data includes at least two of the following: optical image low-altitude data, point cloud low-altitude data, and spectral low-altitude data. Step 120 includes: Based on at least one of the image sharpness and image contrast of the optical image low-altitude data, the modal confidence of the optical image low-altitude data is calculated. Based on at least one of the point cloud density and point cloud completeness of the point cloud low-altitude data, the modal confidence of the point cloud low-altitude data is calculated. Based on at least one of the smoothness of the spectral curve and the completeness of the spectral characteristic bands of the low-altitude spectral data, the modal confidence of the low-altitude spectral data is calculated.

[0040] Here, low-altitude optical imagery data, such as high-resolution visible light photographs taken by drones, provides rich textures, colors, and two-dimensional spatial relationships of ground features, playing a role in the initial identification of ground feature types and macroscopic understanding of scenes. Low-altitude point cloud data, such as point clouds generated by lidar scanning, provides high-precision three-dimensional coordinate information, accurately depicting the geometric structure, height, and undulations of ground features, which is crucial for distinguishing features with similar two-dimensional textures but different heights. Furthermore, low-altitude spectral data, such as data acquired by hyperspectral or multispectral imagers, reflects the differences in reflectivity of ground features across different electromagnetic bands, revealing their physical materials and chemical composition characteristics, and is used for the precise differentiation of different types of vegetation, soil, or building materials.

[0041] It should be noted that by combining at least two types of data from optical image low-altitude data, point cloud low-altitude data, and spectral low-altitude data, a comprehensive perception of the target area can be achieved from multiple dimensions such as visual appearance, three-dimensional structure, and intrinsic properties, forming a comprehensive understanding that far exceeds that of a single data source.

[0042] Specifically, for low-altitude optical imagery data, modal confidence can be calculated based on at least one of image sharpness and image contrast. Image sharpness measures the sharpness of object edges and details in an image, directly affecting the interpretation model's ability to accurately identify small targets or fine structures. Image contrast reflects the brightness differences between different regions of an image; high contrast helps separate targets from the background.

[0043] In one embodiment, image processing algorithms such as the Laplacian operator or energy gradient function can be used to quantify the impact sharpness of low-altitude optical image data; a higher operator response value indicates higher impact sharpness. Furthermore, image contrast can be measured by calculating the standard deviation or information entropy of the image grayscale histogram; a larger standard deviation or higher information entropy indicates higher image contrast. Further, the calculated sharpness and contrast values ​​can be normalized and then fused into a comprehensive quality score using a weighted average; this score serves as the modal confidence score of the low-altitude optical image data.

[0044] For low-altitude point cloud data, modal confidence can be calculated based on at least one of point cloud density and point cloud completeness. Point cloud density refers to the number of points contained within a unit area or volume, and can be used to reflect the level of detail that a 3D model can achieve. Point cloud completeness measures whether there are holes caused by occlusion, laser absorption by object surface materials, etc., during data acquisition, and can be used to reflect whether the target structure can be completely reconstructed.

[0045] In one embodiment, point cloud density can be evaluated by dividing the target region corresponding to the low-altitude point cloud data into a three-dimensional voxel grid and statistically analyzing the point distribution within each non-empty voxel. Conversely, point cloud integrity can be quantified by projecting the low-altitude point cloud data onto a two-dimensional plane and analyzing the area ratio of void regions. Similarly, normalizing and weighted fusion of the quantification results of these two indicators yields the modal confidence of the low-altitude point cloud data.

[0046] For low-altitude spectral data, modal confidence can be calculated based on at least one of spectral curve smoothness and spectral feature band completeness. Spectral curve smoothness reflects the degree of noise interference in the spectral data; a smoother curve indicates higher signal quality, while a sharply fluctuating curve may contain a large amount of random noise. Spectral feature band completeness focuses on whether key bands crucial for specific land cover classification exhibit data saturation, missing data, or anomalies.

[0047] In one embodiment, the smoothness of the spectral curve can be evaluated by calculating the norm of the second derivative of the spectral curve of each pixel; the smaller the norm value, the smoother the curve. The completeness of the spectral characteristic bands can be determined by checking whether the preset key band data values ​​fall within the valid range based on prior knowledge. Furthermore, the modal confidence of the low-altitude spectral data can be calculated by comprehensively processing the spectral curve smoothness and the completeness of the spectral characteristic bands.

[0048] The method provided by this invention quantifies the data quality characteristics of each modality of low-altitude data more objectively and accurately, so that the subsequent fusion process is no longer a blind data superposition, but an intelligent weighting based on the reliability of each modality's data. This ensures that high-quality, high-information modalities play a dominant role in the fusion decision, thereby providing a more solid and reliable data foundation for the accurate judgment of the final interpretation model and significantly improving the robustness and accuracy of the entire interpretation scheme.

[0049] Based on any of the above embodiments, the steps for determining the information entropy of low-altitude data for each modality include: Based on the grayscale distribution entropy of the low-altitude optical image data, the information entropy of the low-altitude optical image data is calculated. Based on the spatial distribution entropy of the point cloud low-altitude data, the information entropy of the point cloud low-altitude data is calculated. The information entropy of the low-altitude spectral data is calculated based on the band information entropy of the low-altitude spectral data.

[0050] Specifically, for low-altitude optical image data, its information entropy is obtained by calculating the gray-level distribution entropy. Here, gray-level distribution entropy refers to an indicator that measures the richness of image information based on the distribution of image pixel gray values. It can be used to quantify the complexity of image texture and the certainty of information. Generally, the more uniform the gray-level distribution and the more gray levels an image covers, the greater its gray-level distribution entropy, indicating that the image may contain more complex details and texture information, and the higher the uncertainty. In one embodiment, the acquired color optical image is first converted into a grayscale image. Then, the number of pixels appearing at each gray level in the grayscale image is counted, and the probability of each gray level appearing is calculated. Finally, the gray-level distribution entropy of the low-altitude optical image data is calculated according to the Shannon entropy formula in information theory.

[0051] For low-altitude point cloud data, its information entropy is obtained by calculating the spatial distribution entropy. Here, spatial distribution entropy is an indicator used to describe the uniformity or clustering of point cloud data in three-dimensional space, and can be used to measure the complexity and regularity of the three-dimensional structure described by the point cloud. Understandably, if the point cloud is very uniformly distributed in space, such as describing a flat ground, its spatial distribution entropy is low, indicating a simple structure and high determinism; conversely, if the point cloud is very irregularly distributed, such as describing a dense tree canopy, its spatial distribution entropy will be high, indicating a complex structure and high uncertainty.

[0052] In one embodiment, the three-dimensional space containing the point cloud data can first be divided into equally sized three-dimensional grids, i.e., voxels. Then, the number of points falling within each voxel is counted. Based on the proportion of points within each voxel to the total number of points, the spatial distribution entropy of the point cloud data can also be calculated using the Shannon entropy formula.

[0053] For low-altitude spectral data, its information entropy is obtained by calculating band information entropy. Here, band information entropy is an indicator that measures the amount of information in the spectral dimension of hyperspectral or multispectral data, and can be used to assess the redundancy or uniqueness of the information provided by different spectral bands. Understandably, if the information between different bands is highly correlated, the overall information redundancy is high, and the band information entropy is relatively low; if each band can provide differentiated and complementary information, the band information entropy will be higher.

[0054] In one embodiment, the grayscale distribution entropy of each spectral band image can be calculated separately, and then the information entropy values ​​calculated for all individual bands can be averaged or weighted and summed to obtain a band information entropy that can comprehensively reflect the overall information content of the spectral data.

[0055] The method provided in this invention, by defining a method for calculating information entropy for different modal data, can more accurately quantify the information complexity of each data source. This allows the method to combine data quality, as reflected by modality confidence, with data uncertainty, as reflected by information entropy, during data fusion, achieving a more scientific and intelligent dynamic weight allocation. For example, for high-quality but structurally complex data, its fusion weights will be appropriately adjusted to balance its reliability and uncertainty. This ensures that the final generated fused low-altitude data feature vector not only incorporates the advantages of each modality but also avoids the negative impacts of excessive complexity or chaos in individual modal information, thus providing optimized input for subsequent interpretation models and further improving the accuracy and reliability of the interpretation results.

[0056] To further improve the accuracy of the interpretation results, based on any of the above embodiments, step 140 includes: The fused low-altitude data feature vector is input into the interpretation model to obtain the model inference result output by the interpretation model; the model inference result includes the model interpretation category, the target space coordinates of the model interpretation category, and the target confidence score; Based on the rule verification library, the model's interpretation category and the target space coordinates are matched to obtain the rule matching score of the model's inference result; Based on the target confidence score of the model inference result and the rule matching score, the double verification score is calculated. The interpretation result is obtained by filtering from the model inference results based on the double verification score.

[0057] Here, model inference results refer to the preliminary analytical conclusions directly output by the interpretation model without any post-processing or verification. It should be noted that model inference results can include: model interpretation category, i.e., the category judgment made by the interpretation model for the features or targets represented by the input data, such as "building," "road," or "woodland"; target spatial coordinates, i.e., the specific location and range of the feature of this category identified by the interpretation model in three-dimensional or two-dimensional space, which can be represented as bounding box coordinates, polygon vertex coordinate sequences, or pixel-level segmentation masks; and target confidence, i.e., the deterministic assessment of the interpretation model's own interpretation category judgment, typically a probability value between 0 and 1.

[0058] Specifically, in step one, the fused low-altitude data feature vector is input into the interpretation model. This feature vector is then converted into an input format acceptable to the larger model, including feature dimension adjustment (e.g., unifying the dimension to 2048) and data normalization (e.g., mapping to the [0,1] interval) to ensure the input features meet the requirements of the larger model. Then, the interpretation model's inference engine is activated, performing layer-by-layer operations on the input fused low-altitude data feature vector. This includes feature association via attention mechanisms, capturing spatial relationships between different regions, and feature mapping in fully connected layers. The fused features are converted into category probabilities, and finally, the probability distribution of each interpretation category is output, yielding the model inference result. It is understandable that the model inference result output through this step fully utilizes the analytical capabilities of a large language model, but it may still contain errors that contradict common sense in the real world.

[0059] Step two involves matching the model's interpretation category with the target spatial coordinates based on the rule validation library, yielding a rule matching score for the model's inference results. Here, the rule validation library is a pre-built knowledge set containing common sense, physical laws, and specific project constraints within the field of geographic information science. The rule validation library can be used to examine the reasonableness of the model's inference results. Furthermore, the rule matching score here can be used to measure the degree to which the model's inference results conform to the rules in the rule validation library; a higher score indicates that the results are more consistent with common sense and logic.

[0060] In one embodiment, matching the model interpretation category and target space coordinates based on a rule verification library can be achieved by using the model interpretation category and target space coordinates from the model inference result as input, and then searching for all applicable rules in the rule library for verification one by one. For each rule passed, the score increases; for each rule violated, the corresponding score is deducted, ultimately resulting in a standardized rule matching score. The rule matching score R∈[0,1], where R=1 for a complete match, R is determined by the degree of matching for a partial match, and R=0 for a complete mismatch.

[0061] The rules in the rule verification library include, but are not limited to: spatial association rules for land features, such as high-density building areas should not exist around cultivated land and water body boundaries should coincide with low terrain elevation areas; and attribute constraint rules for land features, such as the spectral reflectance of industrial land should meet a specific range and road width should be within a preset range. The rule library supports dynamic addition or modification.

[0062] It should be noted that by introducing external prior knowledge to test the original output of the interpretation model, "illusions" that are similar to the patterns learned by the model in terms of data features but are illogical or impossible to exist in the real world are effectively filtered out, thus adding an extra layer of protection to the reliability of the interpretation results.

[0063] Step 3: Based on the target confidence score and rule matching score of the model inference results, calculate the double-verification score. Here, the double-verification score is a comprehensive final evaluation score that integrates the target confidence score, which reflects the certainty of the interpretation model's own judgment, and the rule matching score, which reflects the logical conformity of external knowledge. It can be considered a more comprehensive and robust decision-making basis than a single score.

[0064] In one embodiment, the double check score can be calculated using a weighted summation method. For example, the double check score can be calculated using the following formula, as shown below: ; In the formula, This indicates the double check score; Output the weight coefficients for the confidence score of the model. Let be the weight coefficient for the rule matching score, and ,0< <1, Adjusted according to the interpretation task, such as the land feature classification task. =0.6、 =0.4; Anomaly detection task =0.5、 =0.5, To interpret the target confidence score output by the model, Score for rule matching.

[0065] Step four involves filtering the model inference results based on the double-checked verification score to obtain the interpretation results. For example, a threshold for the double-checked verification score can be set, such as 0.85. The double-checked verification scores of all model inference results are compared with this threshold. Only inference results with scores greater than or equal to the threshold are accepted and output as the final interpretation results. Results with scores below the threshold are considered low-confidence results and can be discarded or marked for further manual review. It should be noted that the double-checked verification score threshold can be adjusted according to the accuracy requirements of the application scenario. Thus, the interpretation results are a set of highly reliable final conclusions filtered through the double-checked verification mechanism.

[0066] The method provided in this invention introduces a dual verification mechanism, combining internal model confidence and external rule verification. It not only relies on the powerful data-driven analysis capabilities of large-scale language models, but also incorporates human common sense and domain knowledge as constraints, achieving dual verification of the interpretation results. This greatly improves the reliability and authenticity of the final output results, effectively suppresses false alarms and false negatives, and makes the entire low-altitude data interpretation method highly automated and intelligent while improving the reliability of the interpretation results.

[0067] Based on any of the above embodiments, step 110 includes: Collect raw multimodal low-altitude data; The original multimodal low-altitude data is denoised to obtain denoised low-altitude data; The noise-reduced low-altitude data is aligned to a spatial coordinate system to obtain aligned low-altitude data; The aligned low-altitude data is converted into a preset standardized format to obtain the converted preset standardized format.

[0068] Specifically, the first step is to collect raw multimodal low-altitude data. This can be achieved, for example, by integrating multiple sensors onto low-altitude flight platforms such as drones. For instance, a visible-light aerial survey camera can capture raw optical imagery low-altitude data, an airborne lidar system can record raw point cloud low-altitude data, and a hyperspectral imager can collect raw spectral low-altitude data for each image point. These collected raw optical imagery low-altitude data, raw point cloud low-altitude data, and raw spectral low-altitude data are then used as the raw multimodal low-altitude data. It can be understood that this raw multimodal low-altitude data can be considered as initial data directly captured by various sensors without any processing.

[0069] Next, the original multimodal low-altitude data is denoised to obtain denoised low-altitude data. Specifically, different denoising techniques can be used for different data modalities. For example, for optical image modal data, median filtering or Gaussian filtering can be used to smooth noise. For point cloud modal data, statistical outlier removal algorithms can be used to eliminate isolated noise points caused by measurement errors. For spectral modal data, Savitzky-Golay filtering algorithms can be used to smooth the spectral curve of each pixel and remove random noise interference.

[0070] Furthermore, the denoised low-altitude data is aligned to a spatial coordinate system to obtain aligned low-altitude data. It should be noted that due to differences in installation location and operating principle among different sensors, the data collected by each sensor has an independent spatial coordinate system. Therefore, all modalities of denoised low-altitude data can be unified under a common spatial coordinate system. Aligned low-altitude data means that all data have been precisely registered in space, i.e., any point in different data corresponds to the same geographical location in the real world. In one embodiment, one modality of data can be selected as a reference, such as high-precision optical imagery. By finding and matching corresponding feature points in each modality of data, such as road intersections and building corners, a transformation matrix from other coordinate systems to the reference coordinate system is calculated. This matrix is ​​then applied to perform geometric correction and reprojection on the other modalities of data, thereby completing spatial alignment.

[0071] It should be noted that by aligning the spatial coordinate system of the noise-reduced low-altitude data, it is ensured that the data from different sensors to be fused describe the attributes of the same spatial location, thus giving the subsequent joint analysis physical meaning and spatial basis.

[0072] Finally, the aligned low-altitude data is converted into a preset standardized format, thus unifying the aligned data from different sources and with varying formats into a standardized structure that facilitates subsequent algorithmic processing. This standardization process includes both data format and data value standardization. For data formats, optical images are in TIFF format, laser point clouds are in LAS format, and spectral data are in ENVI format. Furthermore, for data values, all modal data can be resampled to the same spatial resolution, the pixel values ​​of optical images can be normalized to the [0, 1] interval, the point cloud data can be organized into a unified voxel grid, and all processed data can be encapsulated in a container file such as HDF5 that supports multiple datasets.

[0073] The method provided in this invention provides high-quality, highly consistent, and uniformly formatted multimodal data input for subsequent intelligent interpretation tasks through a series of systematic operations including acquisition, denoising, alignment, and standardization.

[0074] Based on any of the above embodiments, the method further includes: Obtain the current knowledge graph; The low-altitude knowledge elements were extracted from the interpretation results; Based on the aforementioned low-altitude knowledge elements, the current knowledge graph is updated to obtain an updated knowledge graph.

[0075] Here, the current knowledge graph can store and organize various types of knowledge about the low-altitude environment in the form of a graph. Its core components are entities, relations, and attributes. Entities represent specific objects in the real world, such as a particular building or a specific river; relations describe the connections between entities, such as "adjacency" and "containment"; attributes describe the specific characteristics of entities, such as height and material. It should be noted that the constructed knowledge graph can provide basic data for rule validation, or provide rich background knowledge for interpretation models. More importantly, it can serve as a dynamic knowledge base that can be continuously updated and enriched.

[0076] Specifically, during the acquisition process, the knowledge graph can be stored in a specialized graph database or in a file system in a specific file format. Then, low-level knowledge elements can be extracted from the interpretation results. Here, low-level knowledge elements are structured knowledge fragments extracted from the interpretation results that follow a predetermined paradigm, such as "entity-relationship-entity" or "entity-attribute-value" triples.

[0077] For example, low-altitude knowledge elements extracted from the interpretation results can include ground features, such as "farmland A" and "building B"; ground feature attributes, such as farmland area and building number of floors; spatial relationships, such as "farmland A is located east of road E"; and attribute relationships, such as "the spectral reflectance of farmland A is..." ".

[0078] Then, the current knowledge graph can be updated using low-altitude knowledge elements to obtain an updated knowledge graph. Specifically, the extracted low-altitude knowledge elements are first matched with existing entities in the current knowledge graph. If identical entities exist, the attribute information and relationships of that entity are updated; otherwise, a new entity node is created, and relationships are established with other relevant entities to obtain the updated knowledge graph. For example, if a building that has never appeared on the map is discovered, a new entity node and its related attributes and relationship edges can be created in the current knowledge graph.

[0079] It should be noted that when updating the current knowledge graph, confidence level or timestamp mechanisms can be introduced to handle conflicts. For example, information with higher confidence can be retained, or both old and new information can be retained and timestamped for traceability. For instance, an incremental update mechanism can be used to add extracted knowledge elements to the nodes and edges of the knowledge graph, update the graph's attribute database and relation database, and record the update time and source of the knowledge, such as "Date: 202X, UAV low-altitude data interpretation," ensuring the timeliness and traceability of the knowledge graph.

[0080] The method provided in this invention introduces a knowledge graph update mechanism, ensuring that each interpretation task is not merely about completing the current analysis, but also about expanding and iterating upon its own knowledge base. It is understood that by continuously extracting knowledge from the interpretation results and updating the knowledge graph, increasingly rich domain knowledge can be accumulated. This accumulated knowledge, in turn, can enhance the context awareness and rule verification accuracy of future interpretation tasks.

[0081] Based on any of the above embodiments, the steps for obtaining the interpretation model include: Acquire the large-scale language model and sample multimodal low-altitude data; The large language model is pre-trained and fine-tuned based on the sample multimodal low-altitude data to obtain the interpretation model; The interpretation model can be used for any one of the following: land cover classification, target recognition, and anomaly detection.

[0082] Here, the multimodal low-altitude data sample is a high-quality, manually labeled dataset used for model training. This dataset includes not only low-altitude data in various modalities such as optical imagery, point clouds, and spectra, but also corresponding ground truth labels, such as land cover categories and target locations. Its purpose is to guide large-scale language models in learning the mapping relationship between low-altitude data and specific interpretation conclusions.

[0083] Specifically, firstly, a high-performance base version of a large-scale language model that supports multimodal input can be selected and obtained from open-source communities or commercial platforms. Simultaneously, domain experts can be organized to perform refined manual annotation on the large amount of raw multimodal low-altitude data collected. For example, polygonal boundaries of different ground features can be delineated on optical imagery and assigned category labels, and specific targets can be outlined in point cloud data and their 3D coordinates recorded, thereby constructing a large-scale, accurately annotated sample multimodal low-altitude dataset.

[0084] Next, a large language model is pre-trained and fine-tuned based on multimodal low-altitude data to obtain the interpretation model. For example, multimodal low-altitude data can be input into the large language model in batches. The large language model infers from the input data and produces a prediction result. This prediction result is compared with the ground truth labels in the sample data, and the error between the two is calculated using a preset loss function, such as the cross-entropy loss function. Finally, the backpropagation algorithm can be used to finely adjust the parameter weights within the large language model based on this error, so that the error will be smaller when the large language model makes similar predictions in the future. This "input-prediction-comparison-adjustment" process is repeated until the model's performance on an independent validation dataset reaches the preset accuracy standard. The final model with optimized parameter weights is the final interpretation model.

[0085] It should be noted that the knowledge transfer and capability adaptation from general artificial intelligence to domain-specific artificial intelligence uses sample data to teach the model how to understand the complex internal relationships of low-altitude data, thereby enabling the model to accurately complete specialized interpretation tasks and improve the generalization of the interpretation model in different scenarios.

[0086] Furthermore, the interpretation model here can be used for any of the following: land cover classification, object recognition, and anomaly detection. Land cover classification aims to assign a category label to each pixel or unit in the input region, such as distinguishing between farmland, woodland, water bodies, roads, and built-up areas, with the final output being a comprehensive land cover classification map. Object recognition focuses on locating and identifying specific, discrete target objects in a scene, such as finding all the photovoltaic panels on a rooftop or identifying all the individual trees in a field, with the output being the specific location and category of the target. Anomaly detection aims to discover areas or phenomena in the scene that do not conform to normal patterns, such as finding illegal buildings above long-distance pipelines or small-scale signs of pests and diseases in a forest area, with the output being the location of the anomaly and an alert.

[0087] It should be noted that the sample labels and model output layer structure used during model fine-tuning will vary depending on the specific task to be performed. For example, land cover classification tasks require pixel-level annotations, while object recognition tasks require object-level bounding box annotations. By selectively using different types of sample data for fine-tuning, the same basic large-scale language model can be shaped into an interpretation model capable of efficiently completing various specialized tasks.

[0088] It is understood that the model training method provided by this invention can be applied to solve a variety of different practical problems by simply adjusting the training target, thus meeting the diverse application needs in the field of low-altitude remote sensing and greatly enhancing the technical value and market potential of this invention.

[0089] Based on any of the above embodiments Figure 2 This is the second flowchart illustrating the low-altitude data interpretation method provided by the present invention, as shown below. Figure 2 As shown, the method includes: First, low-altitude data acquisition and preprocessing are performed to obtain multimodal low-altitude data. Then, the multimodal data is fused. Specifically, the modal confidence score of each modal can be calculated based on its data quality characteristics. Then, the fusion weights of each modal can be calculated from its modal confidence score and information entropy. Finally, the multimodal low-altitude data can be fused based on these weights to obtain a fused low-altitude data feature vector.

[0090] Furthermore, large-scale model-based reasoning and interpretation are employed. This involves using an interpretation model built upon a large-scale language model to reason and interpret the fused low-altitude data feature vectors, thereby obtaining the model's reasoning results.

[0091] Then, the model inference results are validated. For example, based on the rule validation library, the model interpretation category and target space coordinates in the model inference results are matched to obtain the rule matching score of the model inference results; based on the target confidence and rule matching score of the model inference results, the double validation score is calculated; based on the double validation score, the interpretation results are selected from the model inference results.

[0092] Finally, after obtaining the interpretation results, they can be output and the knowledge graph can be updated based on them. When outputting the interpretation results, the validated results can be converted into industry-standard formats, such as SHP format for Geographic Information Systems (GIS), GeoTIFF format for remote sensing interpretation, and JSON format for data exchange, meeting the format requirements of different application scenarios. Then, a visualization interface for the interpretation results is constructed, supporting 2D map display (e.g., marking interpreted features with different colors); 3D model display (e.g., constructing 3D models of features using laser point clouds); and attribute information querying (e.g., clicking on feature markers to view attributes and confidence levels). In addition, standardized application interfaces can be provided, including API interfaces, SDK interfaces, and database interfaces, supporting data integration with third-party systems to achieve real-time sharing and application of interpretation results.

[0093] Based on any of the above embodiments Figure 3 This is a schematic diagram of the low-altitude data interpretation device provided by the present invention, as shown below. Figure 3 As shown, the device includes: Acquisition unit 310 acquires multimodal low-altitude data; Modal confidence determination unit 320 calculates the modal confidence of each modal low-altitude data based on the data quality characteristics of each modal low-altitude data. The multimodal data fusion unit 330 fuses the multimodal low-altitude data based on the modal confidence and information entropy of each modality to obtain a fused low-altitude data feature vector. The interpretation unit 340 inputs the fused low-altitude data feature vector into the interpretation model to obtain the interpretation result of the multimodal low-altitude data; the interpretation model is constructed based on a large-scale language model.

[0094] The apparatus provided in this invention fuses multimodal low-altitude data by using modal confidence and information entropy of low-altitude data of each modality to obtain dynamically evaluated and adaptive fused low-altitude data feature vectors. Then, it uses a large language model for interpretation, which can fully explore the advantages of different data sources, form high-quality fused data, and achieve accurate interpretation by leveraging the powerful capabilities of the large model. Overall, it improves the automation level, accuracy and reliability of intelligent interpretation of low-altitude data.

[0095] Based on any of the above embodiments, multimodal low-altitude data includes at least two of the following: optical image low-altitude data, point cloud low-altitude data, and spectral low-altitude data. The modal confidence determination unit is specifically used for: Based on at least one of the image sharpness and image contrast of the optical image low-altitude data, the modal confidence of the optical image low-altitude data is calculated. Based on at least one of the point cloud density and point cloud completeness of the point cloud low-altitude data, the modal confidence of the point cloud low-altitude data is calculated. Based on at least one of the smoothness of the spectral curve and the completeness of the spectral characteristic bands of the low-altitude spectral data, the modal confidence of the low-altitude spectral data is calculated.

[0096] Based on any of the above embodiments, the multimodal data fusion unit is specifically used for: Based on the grayscale distribution entropy of the low-altitude optical image data, the information entropy of the low-altitude optical image data is calculated. Based on the spatial distribution entropy of the point cloud low-altitude data, the information entropy of the point cloud low-altitude data is calculated. The information entropy of the low-altitude spectral data is calculated based on the band information entropy of the low-altitude spectral data.

[0097] Based on any of the above embodiments, the decoding unit is specifically used for: The fused low-altitude data feature vector is input into the interpretation model to obtain the model inference result output by the interpretation model; the model inference result includes the model interpretation category, the target space coordinates of the model interpretation category, and the target confidence score; Based on the rule verification library, the model's interpretation category and the target space coordinates are matched to obtain the rule matching score of the model's inference result; Based on the target confidence score of the model inference result and the rule matching score, the double verification score is calculated. The interpretation result is obtained by filtering from the model inference results based on the double verification score.

[0098] Based on any of the above embodiments, the acquisition unit is specifically used for: Collect raw multimodal low-altitude data; The original multimodal low-altitude data is denoised to obtain denoised low-altitude data; The noise-reduced low-altitude data is aligned to a spatial coordinate system to obtain aligned low-altitude data; The aligned low-altitude data is converted into a preset standardized format to obtain the converted preset standardized format.

[0099] Based on any of the above embodiments, the device further includes a knowledge graph updating unit, which is specifically used for: Obtain the current knowledge graph; The low-altitude knowledge elements were extracted from the interpretation results; Based on the aforementioned low-altitude knowledge elements, the current knowledge graph is updated to obtain an updated knowledge graph.

[0100] Based on any of the above embodiments, the device further includes a model training unit, which is specifically used for: Acquire the large-scale language model and sample multimodal low-altitude data; The large language model is pre-trained and fine-tuned based on the sample multimodal low-altitude data to obtain the interpretation model; The interpretation model can be used for any one of the following: land cover classification, target recognition, and anomaly detection.

[0101] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a low-altitude data interpretation method, which includes: acquiring multimodal low-altitude data; calculating the modality confidence of each modality of low-altitude data based on the data quality characteristics of each modality of low-altitude data; fusing the multimodal low-altitude data based on the modality confidence and information entropy of each modality of low-altitude data to obtain a fused low-altitude data feature vector; and inputting the fused low-altitude data feature vector into an interpretation model to obtain the interpretation result of the multimodal low-altitude data; the interpretation model is constructed based on a large-scale language model.

[0102] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0103] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the low-altitude data interpretation method provided by the above methods. The method includes: acquiring multimodal low-altitude data; calculating the modality confidence of each modality of low-altitude data based on the data quality characteristics of each modality; fusing the multimodal low-altitude data based on the modality confidence and information entropy of each modality of low-altitude data to obtain a fused low-altitude data feature vector; and inputting the fused low-altitude data feature vector into an interpretation model to obtain the interpretation result of the multimodal low-altitude data. The interpretation model is constructed based on a large-scale language model.

[0104] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for interpreting low-altitude data provided by the methods described above. This method includes: acquiring multimodal low-altitude data; calculating the modality confidence of each modality based on its data quality characteristics; fusing the multimodal low-altitude data based on its modality confidence and information entropy to obtain a fused low-altitude data feature vector; and inputting the fused low-altitude data feature vector into an interpretation model to obtain the interpretation result of the multimodal low-altitude data. The interpretation model is constructed based on a large-scale language model.

[0105] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0106] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for interpreting low-altitude data, characterized in that, include: Acquire multimodal low-altitude data; Based on the data quality characteristics of each modality of low-altitude data, the modal confidence of each modality of low-altitude data is calculated; Based on the modal confidence and information entropy of each modal low-altitude data, the multimodal low-altitude data is fused to obtain a fused low-altitude data feature vector; The fused low-altitude data feature vector is input into the interpretation model to obtain the interpretation result of the multimodal low-altitude data; the interpretation model is constructed based on a large-scale language model.

2. The method for interpreting low-altitude data according to claim 1, characterized in that, The multimodal low-altitude data includes at least two of the following: optical image low-altitude data, point cloud low-altitude data, and spectral low-altitude data. The modal confidence scores of the low-altitude data are calculated based on the data quality characteristics of each modality, including: Based on at least one of the image sharpness and image contrast of the optical image low-altitude data, the modal confidence of the optical image low-altitude data is calculated. Based on at least one of the point cloud density and point cloud completeness of the point cloud low-altitude data, the modal confidence of the point cloud low-altitude data is calculated. Based on at least one of the smoothness of the spectral curve and the completeness of the spectral characteristic bands of the low-altitude spectral data, the modal confidence of the low-altitude spectral data is calculated.

3. The method for interpreting low-altitude data according to claim 2, characterized in that, The steps for determining the information entropy of each modality of low-altitude data include: Based on the grayscale distribution entropy of the low-altitude optical image data, the information entropy of the low-altitude optical image data is calculated. Based on the spatial distribution entropy of the point cloud low-altitude data, the information entropy of the point cloud low-altitude data is calculated. The information entropy of the low-altitude spectral data is calculated based on the band information entropy of the low-altitude spectral data.

4. The method for interpreting low-altitude data according to any one of claims 1 to 3, characterized in that, The step of inputting the fused low-altitude data feature vector into the interpretation model to obtain the interpretation result of the multimodal low-altitude data includes: The fused low-altitude data feature vector is input into the interpretation model to obtain the model inference result output by the interpretation model; the model inference result includes the model interpretation category, the target space coordinates of the model interpretation category, and the target confidence score; Based on the rule verification library, the model's interpretation category and the target space coordinates are matched to obtain the rule matching score of the model's inference result; Based on the target confidence score of the model inference result and the rule matching score, the double verification score is calculated. The interpretation result is obtained by filtering from the model inference results based on the double verification score.

5. The method for interpreting low-altitude data according to any one of claims 1 to 3, characterized in that, The acquisition of multimodal low-altitude data includes: Collect raw multimodal low-altitude data; The original multimodal low-altitude data is denoised to obtain denoised low-altitude data; The noise-reduced low-altitude data is aligned to a spatial coordinate system to obtain aligned low-altitude data; The aligned low-altitude data is converted into a preset standardized format to obtain the converted preset standardized format.

6. The method for interpreting low-altitude data according to any one of claims 1 to 3, characterized in that, The method further includes: Obtain the current knowledge graph; The low-altitude knowledge elements were extracted from the interpretation results; Based on the aforementioned low-altitude knowledge elements, the current knowledge graph is updated to obtain an updated knowledge graph.

7. The method for interpreting low-altitude data according to any one of claims 1 to 3, characterized in that, The steps for obtaining the interpretation model include: Acquire the large-scale language model and sample multimodal low-altitude data; The large language model is pre-trained and fine-tuned based on the sample multimodal low-altitude data to obtain the interpretation model; The interpretation model can be used for any one of the following: land cover classification, target recognition, and anomaly detection.

8. A device for interpreting low-altitude data, characterized in that, include: Acquisition unit, acquires multimodal low-altitude data; The modal confidence determination unit calculates the modal confidence of each modal low-altitude data based on the data quality characteristics of each modal low-altitude data. The multimodal data fusion unit fuses the multimodal low-altitude data based on the modal confidence and information entropy of each modality to obtain a fused low-altitude data feature vector. The interpretation unit inputs the fused low-altitude data feature vector into the interpretation model to obtain the interpretation result of the multimodal low-altitude data; the interpretation model is constructed based on a large-scale language model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the low-altitude data interpretation method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the low-altitude data interpretation method as described in any one of claims 1 to 7.

Images