A method and system for processing unmanned aerial vehicle surveying and mapping information of a prefabricated model

Abstract

The application discloses a kind of unmanned aerial vehicle surveying and mapping information processing method and system of prefabricated model, belong to unmanned aerial vehicle surveying and mapping technical field, for solving many deficiencies of existing unmanned aerial vehicle surveying and mapping technology in data processing efficiency, model adaptability, data fusion precision and positioning stability etc., urgent to be improved by technical innovation The technical problem of technical problem. Method includes: based on the unmanned aerial vehicle historical data collected in historical period, construct multimodal prefabricated model;According to real-time environmental parameters, the confidence weight of each element in multimodal prefabricated model is dynamically adjusted, to obtain dynamic prefabricated model;Construct space-time coding change detection model;Real-time data of unmanned aerial vehicle of target area and multimodal historical data loaded in dynamic prefabricated model are respectively extracted for features, to obtain real-time multimodal feature and historical multimodal feature sequence;Real-time multimodal feature and historical multimodal feature sequence are input into space-time coding change detection model, to obtain the surveying and mapping change information of target area.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle surveying and mapping, and particularly relates to a pre-prepared model unmanned aerial vehicle surveying and mapping information processing method and system. BACKGROUND

[0002] In the technical field of unmanned aerial vehicle surveying and mapping, although the existing technology has made certain progress, there are still several significant defects and industry pain points, which restrict the further improvement of surveying and mapping efficiency and accuracy.

[0003] The traditional unmanned aerial vehicle surveying and mapping method mainly relies on real-time full-quantity data acquisition. This way often causes redundant calculation on the repeatedly surveyed areas when dealing with complex terrain or large-area areas, resulting in waste of computing resources and low surveying and mapping efficiency. In addition, the traditional static pre-prepared model is not capable of coping with dynamic changes in the natural environment, such as vegetation growth and seasonal replacement, because these models cannot timely reflect these dynamic changes, thereby causing a high false alarm rate of change detection and affecting the accuracy and reliability of the surveying and mapping results.

[0004] In terms of data fusion, the existing multi-source data (including visible light, LiDAR, multispectral, etc.) fusion technology is not perfect, and it is difficult to achieve accurate identification of subtle changes. For example, the slight changes of key infrastructure such as ditches and pipelines are often ignored or misjudged. This insufficient fusion limits the application potential of unmanned aerial vehicle surveying and mapping in fine management. In addition, fixed markers (such as infrastructure such as towers) as positioning reference points in surveying and mapping, their positioning accuracy is easily affected by natural factors such as weather and light, resulting in unstable positioning results, which further affects the accuracy and reliability of the surveying and mapping data.

[0005] In summary, the existing unmanned aerial vehicle surveying and mapping technology has many deficiencies in data processing efficiency, model adaptability, data fusion accuracy, and positioning stability, and needs to be improved and promoted through technical innovation. SUMMARY

[0006] The present application provides a pre-prepared model unmanned aerial vehicle surveying and mapping information processing method and system, which is used to solve the following technical problems: the existing unmanned aerial vehicle surveying and mapping technology has many deficiencies in data processing efficiency, model adaptability, data fusion accuracy, and positioning stability, and needs to be improved and promoted through technical innovation.

[0007] The present application embodiment adopts the following technical scheme:

[0008] On the one hand, the present application embodiment provides a pre-prepared model unmanned aerial vehicle surveying and mapping information processing method, which comprises: constructing a multi-modal pre-prepared model based on the unmanned aerial vehicle historical data collected in the historical period.

[0009] According to real-time environmental parameters, the confidence weights of each element in the multi-modal pre-model are dynamically adjusted to obtain a dynamic pre-model;

[0010] Based on the U-Net network, the Transformer network and the cross attention mechanism, a spatio-temporal coding change detection model is constructed.

[0011] The real-time data of the target area and the multi-modal historical data loaded in the dynamic pre-model are respectively subjected to feature extraction to obtain real-time multi-modal features and historical multi-modal feature sequences.

[0012] The real-time multi-modal features and historical multi-modal feature sequences are input into the spatio-temporal coding change detection model to obtain the surveying and mapping change information of the target area.

[0013] In a feasible implementation, a multi-modal pre-model is constructed based on historical unmanned aerial vehicle data collected in a historical period, specifically including:

[0014] Unmanned aerial vehicle historical data collected in a plurality of historical periods of a target area are obtained; wherein the unmanned aerial vehicle historical data at least includes RGB image data, laser radar point cloud data and multi-spectral image data;

[0015] The RGB image data and the multi-spectral image data are input into a DeepLabV3+ network for target recognition and classification to obtain fixed elements and dynamic elements in the target area; wherein the dynamic elements at least include vegetation and water body.

[0016] Normalized vegetation index NDVI and normalized water index NDWI are extracted in each frame of multi-spectral image data.

[0017] According to the shooting dates of the current frame RGB image data and multi-spectral image data, spatio-temporal labels are added to the dynamic elements; wherein the spatio-temporal labels at least include time stamp, season code and weather code.

[0018] Based on the nearest point search algorithm, the laser radar point cloud data collected in a plurality of historical periods are registered, and a three-dimensional reference model of the target area in a plurality of historical periods is constructed according to the registered laser radar point cloud data.

[0019] The fixed elements and dynamic elements are all labeled in the three-dimensional reference model, and the normalized vegetation index NDVI, the normalized water index NDWI and the spatio-temporal labels are all embedded into the dynamic elements of the three-dimensional reference model to obtain the multi-modal pre-model.

[0020] In a feasible implementation, according to real-time environment parameters, the confidence weights of each element in the multi-modal pre-model are dynamically adjusted to obtain a dynamic pre-model, specifically including:

[0021] After cascading several layers of MLP networks and connecting with a Softmax network, a weight generation network is constructed and trained;

[0022] Real-time environment parameters of a target region are collected by an environment parameter sensor; wherein the real-time environment parameters at least include current illumination intensity, current weather code, current season code, current average temperature and current average humidity;

[0023] According to the real-time environment parameters and historical environment parameters in the multi-modal pre-model, multi-dimensional input features are constructed and preprocessed;

[0024] The preprocessed multi-dimensional input features are input into the weight generation network, and the confidence weights of each element in the multi-modal pre-model at the current time are output;

[0025] Based on the confidence weights, a weighted fusion model of real-time data of the unmanned aerial vehicle and the multi-modal pre-model is constructed to obtain the dynamic pre-model.

[0026] In a feasible implementation, according to the real-time environment parameters and historical environment parameters in the multi-modal pre-model, multi-dimensional input features are constructed and preprocessed, specifically including:

[0027] According to the real-time environment parameters and historical environment parameters in the multi-modal pre-model, multi-dimensional input features are constructed: X = [L, W, ||S now -S hist ||2, ΔT avg , ΔH avg ]; wherein L is the current illumination intensity, W is the current weather code, ||S now -S hist ||2 is the Euclidean distance between the current season code S now and the historical season code S hist in the multi-modal pre-model; ΔT avg is the difference between the current average temperature and the historical average temperature of the same period in the multi-modal pre-model; ΔH avg is the difference between the current average humidity and the historical average humidity of the same period in the multi-modal pre-model;

[0028] Each feature channel in the multi-dimensional input features is standardized to obtain the final input features.

[0029] In a feasible implementation, based on the confidence weight, a weighted fusion model of the unmanned aerial vehicle real-time data and the multi-modal prefabricated model is constructed to obtain the dynamic prefabricated model, specifically including:

[0030] According to The dynamic prefabricated model M is obtained by weighted fusion of the unmanned aerial vehicle real-time data and the multi-modal prefabricated model r ;

[0031] Wherein, ω i represents the confidence weight of the i-th element, N represents the total number of fixed elements and dynamic elements in the multi-modal prefabricated model, Align is a data alignment operation, is the unmanned aerial vehicle real-time data, is the historical data in the multi-modal prefabricated model;

[0032] According to the real-time confidence weight of each element calculated according to the real-time environmental parameters collected each time, the dynamic prefabricated model in the prefabricated model library is updated in real time.

[0033] In a feasible implementation, based on the U-Net network, the Transformer network and the cross attention mechanism, a space-time coding change detection model is constructed, specifically including:

[0034] Based on the U-Net convolutional neural network, a basic space network is constructed; wherein the basic space network includes an input layer, a multi-layer encoder, a bridge layer, a multi-layer decoder and an output layer;

[0035] An alternating convolution layer with a preset hole rate is introduced in the bridge layer to expand the receptive field; a channel attention is added at the jump connection of each layer to suppress invalid band interference, and a space change detection branch network is obtained;

[0036] Based on the Transformer neural network architecture, a time change detection branch network is constructed; the time change detection branch network includes a block embedding layer for dividing each image in the input time sequence image sequence into 16x16 blocks;

[0037] A cross attention module is constructed for projecting the multi-scale feature maps output by the space change detection branch network into a feature matrix, and fusing the feature matrix with the time sequence features output by the time change detection branch network through a space-time attention calculation formula to output fusion features;

[0038] A change detection head is constructed for identifying the probability of change of each pixel according to the fusion features to output a change probability map;

[0039] The spatial variation detection branch network, the temporal variation detection branch network, the cross attention module, and the variation detection head constitute the spatio-temporal coding variation detection model.

[0040] In an available embodiment, feature extraction is performed on the unmanned aerial vehicle real-time data of the target area and the multi-modal historical data loaded in the dynamic pre-prepared model respectively to obtain real-time multi-modal features and historical multi-modal feature sequences, specifically including:

[0041] In actual unmanned aerial vehicle surveying, unmanned aerial vehicle real-time data of the target area is collected in real time by an unmanned aerial vehicle; wherein the unmanned aerial vehicle real-time data at least includes: RGB image data, laser radar point cloud data and multi-spectral image data;

[0042] The dynamic pre-prepared model corresponding to the target area is extracted in the pre-prepared model library, the multi-modal historical data therein is loaded, and the multi-modal historical data of each element is multiplied by the corresponding confidence weight to obtain multi-modal weighted historical data; wherein the multi-modal historical data at least includes geometric data, spectral data and spatio-temporal labels;

[0043] The unmanned aerial vehicle real-time data and the multi-modal weighted historical data are respectively feature-encoded and aligned to obtain real-time multi-modal features and historical multi-modal feature sequences; wherein the feature encoding at least includes at least one or more encoding methods in point cloud feature extraction, spectral time series encoding and semantic embedding.

[0044] In an available embodiment, the real-time multi-modal features and historical multi-modal feature sequences are input into the spatio-temporal coding variation detection model to obtain the surveying change information of the target area, specifically including:

[0045] The real-time multi-modal features and historical multi-modal feature sequences are simultaneously input into the spatio-temporal coding variation detection model for spatial multi-scale feature extraction and time series feature extraction;

[0046] The multi-scale features are projected into a Query matrix by convolution, and the time series features are expanded into a Key-Value matrix by a full connection layer;

[0047] Based on the spatio-temporal attention calculation formula in the spatio-temporal coding variation detection model, the fusion features of the Query matrix and the Key-Value matrix are output;

[0048] The fusion features are input into the variation detection head to determine the probability of each pixel in the real-time RGB image data and the real-time multi-spectral image data changing, and a corresponding change probability map is generated;

[0049] In the change probability map, mapping change information of the target region is extracted; wherein the mapping change information at least includes a position of an important change region in the target region and a corresponding change type.

[0050] In an available implementation, after obtaining the mapping change information of the target region, the method further includes:

[0051] When there is an important change region in the target region, real-time UAV data of the current frame is inserted into the dynamic pre-model to replace the oldest historical frame in the dynamic pre-model;

[0052] For an unchanged region in the target region, only the time stamp of each element in the unchanged region in the dynamic pre-model is updated, and the geometric feature and spectral feature thereof are retained, so that incremental update of the dynamic pre-model is realized.

[0053] In another aspect, the embodiment of the present application also provides a UAV mapping information processing system of a pre-model, characterized in that the system comprises:

[0054] a pre-model construction module, configured to construct a multi-modal pre-model based on historical UAV data collected in a historical period, and dynamically adjust the confidence weight of each element in the multi-modal pre-model according to real-time environmental parameters to obtain a dynamic pre-model;

[0055] a real-time mapping module, configured to construct a spatio-temporal coding change detection model based on a U-Net network, a Transformer network and a cross attention mechanism, extract features from UAV real-time data of a target region and multi-modal historical data loaded in the dynamic pre-model respectively to obtain real-time multi-modal feature and historical multi-modal feature sequences, and input the real-time multi-modal feature and the historical multi-modal feature sequences into the spatio-temporal coding change detection model to obtain mapping change information of the target region.

[0056] Compared with the prior art, the UAV mapping information processing method and system of the pre-model provided by the embodiment of the present application have the following beneficial effects:

[0057] 1. Efficiency is significantly improved: by introducing a pre-model, the present application effectively reduces most of the repeated mapping regions in the UAV mapping process, thereby greatly reducing unnecessary flight of the UAV when performing a mapping task, and reducing the time consumption of a single task. This improvement not only significantly improves the efficiency of mapping work, but also reduces the cost of mapping, and provides strong support for large-scale and high-frequency mapping operations.

[0058] 2. Strong adaptive capability: The application innovatively introduces a dynamic weight mechanism, breaking through the limitations of traditional static pre-made models and realizing environment-adaptive multi-source data fusion. This mechanism enables the UAV to dynamically adjust the confidence weight of each element data in the pre-made model according to environmental factors such as seasonal changes, thereby effectively reducing the false alarm rate caused by seasonal changes and improving the accuracy and reliability of the surveying and mapping results.

[0059] 3. Spatio-temporal joint coding innovation: The application also adopts spatio-temporal joint coding technology, which can capture spatial details and temporal evolution rules simultaneously, thereby significantly improving the detection capability of small changes. This innovation provides more comprehensive and detailed data support for UAV surveying and mapping, which helps to discover potential environmental problems and change trends.

[0060] In summary, the application provides a new solution for UAV surveying and mapping with high precision, low redundancy, and strong adaptability through the deep coupling of pre-made models and real-time perception. It not only performs well in efficiency, accuracy, and adaptability, but also further improves the comprehensiveness and accuracy of the surveying and mapping results through the innovative application of spatio-temporal joint coding technology. Compared with existing technologies, the application has significant advantages and wide application prospects. BRIEF DESCRIPTION OF DRAWINGS

[0061] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without creative labor. In the drawings:

[0062] Figure 1 A flow chart of a UAV surveying and mapping information processing method of a pre-made model provided for the embodiments of the present application;

[0063] Figure 2 A structural schematic diagram of a UAV surveying and mapping information processing system of a pre-made model provided for the embodiments of the present application. DETAILED DESCRIPTION

[0064] To enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0065] The embodiment of the application provides a UAV surveying and mapping information processing method for a prefabricated model, as shown in the figure Figure 1 The UAV surveying and mapping information processing method for the prefabricated model specifically comprises steps S101-S105:

[0066] S101, based on historical UAV data collected in a historical period, a multi-modal prefabricated model is constructed.

[0067] Specifically, historical UAV data collected in a plurality of historical periods of a target area is acquired; wherein the historical UAV data at least comprises RGB image data, laser radar point cloud data and multispectral image data.

[0068] Further, the RGB image data and the multispectral image data are input into a DeepLabV3+ network for target recognition and classification, to acquire fixed elements and dynamic elements in the target area; wherein the dynamic elements at least comprise vegetation and water. Normalized vegetation index NDVI and normalized water index NDWI are extracted in each frame of multispectral image data.

[0069] Further, according to the shooting date of the current frame RGB image data and the multispectral image data, a space-time label is added to the dynamic elements. The space-time label at least comprises a timestamp, a season code and a weather code.

[0070] In one embodiment, 12 periods of UAV inspection data of a park in the past year are acquired, a high-definition camera, a multispectral camera and a laser radar are carried on the UAV, and the RGB image data, the laser radar point cloud data and the multispectral image data of the inspection area can be acquired simultaneously. The historical data acquired is input into the DeepLabV3+ network, to extract fixed markers (such as street lamps, public facilities, buildings, etc.) and dynamic elements (such as vegetation, water, etc.) in the park. Then, according to the inspection time of each period of data, a timestamp t and a season code S∈R 4 (Spring / Summer / Autumn / Winter) are added to each element.

[0071] Further, based on a nearest point search algorithm, the laser radar point cloud data collected in a plurality of historical periods is registered, and a three-dimensional reference model of the target area in the plurality of historical periods is constructed according to the registered laser radar point cloud data.

[0072] Finally, the fixed elements and the dynamic elements are all labeled in the three-dimensional reference model, and the normalized vegetation index NDVI, the normalized water index NDWI and the space-time label are all embedded into the dynamic elements of the three-dimensional reference model, to obtain a multi-modal prefabricated model.

[0073] S102, according to real-time environmental parameters, the confidence weights of elements in the multi-modal prefabricated model are dynamically adjusted, to obtain a dynamic prefabricated model.

[0074] Specifically, a weight generation network is constructed by cascading several layers of MLP networks and connecting them to a Softmax network. This network is then trained. The weight generation network can be trained using environmental parameters from historical inspection data.

[0075] Furthermore, real-time environmental parameters of the target area are collected through environmental parameter sensors; among which, real-time environmental parameters include at least the current light intensity, current weather code, current season code, current average temperature, and current average humidity.

[0076] Furthermore, based on real-time environmental parameters and historical environmental parameters in the multimodal pre-built model, multidimensional input features are constructed and preprocessed. The specific implementation method is as follows:

[0077] Based on real-time environmental parameters and historical environmental parameters in the multimodal prefabricated model, a multidimensional input feature is constructed: X=[L,W,||S now -S hist ||2,ΔT avg ΔH avg Where L is the current light intensity, W is the current weather code, and ||S now -S hist ||2 is the current season code S now Historical seasonal coding S in multimodal prefabricated models hist The Euclidean distance between them; ΔT avg ΔH represents the difference between the current average temperature and the historical average temperature for the same period in the multimodal prefabricated model. avg This represents the difference between the current average humidity and the historical average humidity for the same period in the multimodal prefabricated model.

[0078] Then, each feature channel in the multidimensional input features is standardized to obtain the final input features.

[0079] Furthermore, the preprocessed multidimensional input features are input into the trained weight generation network, which outputs the confidence weight ω of each element in the multimodal prefabricated model at the current time. i =f(L,W,||S now -S hist ||2). Where f is the objective function in the weight generation network.

[0080] Furthermore, based on confidence weights, a weighted fusion model of real-time UAV data and multimodal prefabricated models is constructed to obtain a dynamic prefabricated model. The specific implementation method is as follows:

[0081] according to The real-time data from the UAV is weighted and fused with the multimodal prefabricated model to obtain the dynamic prefabricated model M.r .

[0082] wherein ω i represents the confidence weight of the i-th element, N represents the total number of fixed elements and dynamic elements in the multi-modal pre-model, Align is a data alignment operation, is real-time data of the unmanned aerial vehicle, is historical data in the multi-modal pre-model.

[0083] As a feasible implementation, the dynamic pre-models of different target areas are stored in a pre-model library and are maintained regularly. In the daily maintenance of the pre-model library, the real-time confidence weight of each element is calculated according to the real-time environmental parameters collected each time, and is substituted into the dynamic pre-model to update the dynamic pre-model in the pre-model library in real time, so that the dynamic elements in the dynamic pre-model can adjust the confidence in a timely manner according to the changes of seasons, time and climate, and false alarms caused by the system mistaking the great change of vegetation as abnormal change due to seasonal change in the actual surveying and mapping process are avoided.

[0084] S103, based on the U-Net network, the Transformer network and the cross attention mechanism, a space-time coding change detection model is constructed.

[0085] Specifically, a basic space network is constructed based on a U-Net convolutional neural network; wherein the basic space network includes an input layer, a plurality of encoding layers, a bridging layer, a plurality of decoding layers and an output layer.

[0086] An alternating convolution layer with a preset hole rate is introduced at the bridging layer to expand the receptive field; a channel attention is added at the skip connection of each layer to suppress invalid band interference, and the basic space network is improved to obtain a space change detection branch network. The U-Net convolutional neural network can extract multi-level spatial features from single temporal data and capture detailed textures and global semantics.

[0087] Further, a time change detection branch network is constructed based on a Transformer neural network architecture; the time change detection branch network includes a block embedding layer for dividing each image in the input time sequence image sequence into 16x16 blocks. The Transformer neural network can encode the time sequence dependency relationship between multi-temporal data and model the change evolution law.

[0088] Further, a cross attention module is constructed for projecting the multi-scale feature maps output by the space change detection branch network into a feature matrix, and fusing the feature matrix with the time sequence features output by the time change detection branch network through a space-time attention calculation formula to output fused features.

[0089] Further, a change detection head is constructed to identify the probability of each pixel changing according to the fusion features, and output a change probability map.

[0090] Finally, the spatial change detection branch network, the temporal change detection branch network, the cross-attention module and the change detection head are combined to form a spatio-temporal coding change detection model.

[0091] S104, the unmanned aerial vehicle real-time data of the target area and the multi-modal historical data loaded in the dynamic pre-model are respectively subjected to feature extraction, and real-time multi-modal features and historical multi-modal feature sequences are obtained.

[0092] Specifically, during actual unmanned aerial vehicle surveying and mapping, unmanned aerial vehicle real-time data of the target area is collected in real time by the unmanned aerial vehicle; wherein the unmanned aerial vehicle real-time data at least includes: RGB image data, laser radar point cloud data and multi-spectral image data.

[0093] Further, the dynamic pre-model corresponding to the target area is extracted in the pre-model library, the multi-modal historical data therein is loaded, and the multi-modal historical data of each element is multiplied by the corresponding confidence weight to obtain multi-modal weighted historical data. Wherein the multi-modal historical data at least includes geometric data, spectral data and spatio-temporal label.

[0094] Further, the unmanned aerial vehicle real-time data and the multi-modal weighted historical data are respectively subjected to feature coding and alignment to obtain real-time multi-modal features and historical multi-modal feature sequences; wherein the feature coding at least includes at least one or more coding modes in point cloud feature extraction, spectral time series coding and semantic embedding.

[0095] As a feasible implementation, the multi-modal historical data of the target area is loaded from the pre-model library, including:

[0096] Geometric data: registered laser radar point cloud data, containing three-dimensional coordinates, reflection intensity and other information;

[0097] Spectral data: multi-temporal NDVI, NDWI, raster data and other information;

[0098] Spatio-temporal label: time stamp, season coding, weather code and other information.

[0099] The feature coding process includes:

[0100] Point cloud feature extraction: using PointNet++ to generate point cloud global feature vector;

[0101] Spectral time series coding: input NDVI / NDWI sequence into LSTM, and output time series features;

[0102] Semantic embedding: encode the spatio-temporal label of the element by BERT to obtain the semantic encoding features of the spatio-temporal label.

[0103] S105, input the real-time multi-modal features and the historical multi-modal feature sequence into the spatio-temporal coding change detection model to obtain the surveying and mapping change information of the target region.

[0104] Specifically, the real-time multi-modal features and the historical multi-modal feature sequence are simultaneously input into the spatio-temporal coding change detection model for spatial multi-scale feature extraction and time sequence feature extraction. The multi-scale features are projected into a Query matrix by convolution, and the time sequence features are expanded into a Key-Value matrix by a fully connected layer.

[0105] Further, based on the spatio-temporal attention calculation formula in the spatio-temporal coding change detection model, the fusion features Attention(Q, K, V) of the Query matrix and the Key-Value matrix are output.

[0106] Further, the fusion features are input into the change detection head to determine the probability P of each pixel in the real-time RGB image data and the real-time multi-spectral image data changing change ∈[0,1], and generate the corresponding change probability map.

[0107] Further, in the change probability map, the surveying and mapping change information of the target region is extracted; wherein the surveying and mapping change information at least includes the important change region position in the target region and the corresponding change type.

[0108] As a feasible implementation, in the change probability map, by comparing with the change probability threshold, the pixels exceeding the threshold can be determined, and then the change region composed of the pixels is determined. According to the difference between the element type in this change region and the historical change parameter and the change parameter, the change type of this change region can be determined, such as abnormal growth of vegetation, abnormal increase of river area, etc.

[0109] In one embodiment, if a summer inspection is performed on a park, the spring dynamic pre-model of the park is first called, and then the real-time data features collected by the unmanned aerial vehicle and the historical multi-modal feature sequence extracted from the spring dynamic pre-model are simultaneously input into the spatio-temporal coding change detection model. The unmanned aerial vehicle detects that the NDVI of a certain region rises by 0.3 (suspected rapid growth of vegetation); the result obtained after spatio-temporal coding analysis is: compared with the historical data of the same period, it is found that the NDVI increment of this region in previous years is only 0.1. The newly added shrub area is identified through the cross attention mechanism. The laser radar point cloud data and multi-spectral data of this area are inserted into the dynamic pre-model for updating, and the change type is marked as "high-risk vegetation encroachment area", triggering the pruning work order.

[0110] Further, when there is an important change region in the target region, the real-time UAV data of the current frame is inserted into the dynamic pre-model to replace the oldest historical frame in the dynamic pre-model. For the unchanged region in the target region, only the time stamp of each element in the unchanged region in the dynamic pre-model is updated, and the geometric feature and spectral feature thereof are retained, so as to realize incremental update of the dynamic pre-model.

[0111] In addition, the embodiment of the present application also provides a UAV surveying and mapping information processing system of a pre-model, as shown in the figure, the UAV surveying and mapping information processing system 200 of the pre-model specifically comprises: Figure 2

[0112] The pre-model construction module 210 is used for constructing a multi-modal pre-model based on historical UAV data collected in a historical period, and dynamically adjusting the confidence weight of each element in the multi-modal pre-model according to real-time environmental parameters to obtain a dynamic pre-model.

[0113] The real-time surveying and mapping module 220 is used for constructing a spatio-temporal coding change detection model based on a U-Net network, a Transformer network and a cross attention mechanism, performing feature extraction on the real-time UAV data of the target region and the multi-modal historical data loaded in the dynamic pre-model respectively to obtain real-time multi-modal feature and historical multi-modal feature sequences, and inputting the real-time multi-modal feature and the historical multi-modal feature sequences into the spatio-temporal coding change detection model to obtain surveying and mapping change information of the target region.

[0114] Each embodiment in the present application is described in a progressive manner, and the same and similar parts between each embodiment can be referred to each other. Each embodiment mainly describes the difference from other embodiments. Especially, for the device, equipment and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the related parts can be referred to the part of the method embodiment.

[0115] The above describes specific embodiments of the present application. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0116] The above only describes the embodiments of the present application and is not used to limit the present application. The embodiments of the present application can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present application shall be included in the protection scope of the present application.​

Claims

1. A method for processing unmanned aerial vehicle surveying information of a prefabricated model, characterized in that, The method comprises: Based on the historical data of unmanned aerial vehicle collected in the historical period, a multi-modal prefabricated model is constructed; According to the real-time environmental parameters, the confidence weights of each element in the multi-modal prefabricated model are dynamically adjusted to obtain a dynamic prefabricated model, specifically comprising: Cascade several layers of MLP network and connect with Softmax network to construct weight generation network and train; Collect real-time environmental parameters of the target area through environmental parameter sensors; wherein the real-time environmental parameters at least include current illumination intensity, current weather code, current season code, current average temperature and current average humidity; According to the real-time environmental parameters and the historical environmental parameters in the multi-modal prefabricated model, multi-dimensional input features are constructed and preprocessed, specifically comprising: According to the real-time environment parameter and the historical environment parameter in the multi-modal pre-prepared model, a multi-dimensional input feature is constructed: ; wherein, L is the current light intensity, W is the current weather code, is the current season code and the Euclidean distance between the historical season code in the multi-modal pre-prepared model ; and is the difference between the current average temperature and the same period historical average temperature in the multi-modal pre-prepared model; is the difference between the current average humidity and the same period historical average humidity in the multi-modal pre-prepared model. Standardize each feature channel in the multi-dimensional input feature to obtain the final input feature; Input the preprocessed multi-dimensional input feature into the weight generation network to output the confidence weight of each element in the multi-modal prefabricated model at the current time; Based on the confidence weight, a weighted fusion model of unmanned aerial vehicle real-time data and multi-modal prefabricated model is constructed to obtain the dynamic prefabricated model, specifically comprising: According to , the unmanned aerial vehicle real-time data is weighted and fused with the multi-modal pre-prepared model to obtain the dynamic pre-prepared model ; wherein, represents the confidence weight of the i-th element, N represents the total number of fixed elements and dynamic elements in the multi-modal pre-made model, Align is a data alignment operation, is real-time data of the unmanned aerial vehicle, is historical data in the multi-modal pre-made model; According to the real-time environmental parameters collected each time, the real-time confidence weight of each element is calculated and substituted into the dynamic prefabricated model to update the dynamic prefabricated model in the prefabricated model library in real time; Based on U-Net network, Transformer network and cross attention mechanism, a spatio-temporal coding change detection model is constructed; The real-time multi-modal features and historical multi-modal feature sequences are obtained by respectively extracting features from the unmanned aerial vehicle real-time data of the target area and the multi-modal historical data loaded in the dynamic prefabricated model; Input the real-time multi-modal features and historical multi-modal feature sequences into the spatio-temporal coding change detection model to obtain the surveying and mapping change information of the target area.

2. The unmanned aerial vehicle surveying information processing method of a prefabricated model according to claim 1, wherein, Based on the historical data of unmanned aerial vehicle collected in the historical period, a multi-modal prefabricated model is constructed, specifically comprising: Obtain the historical data of unmanned aerial vehicle collected in multiple historical periods of the target area; wherein the historical data of unmanned aerial vehicle at least includes: RGB image data, laser radar point cloud data and multispectral image data; Input the RGB image data and the multispectral image data into DeepLabV3+ network for target recognition and classification to obtain fixed elements and dynamic elements in the target area; wherein the dynamic elements at least include vegetation and water body; Extract normalized vegetation index NDVI and normalized water index NDWI in each frame of multispectral image data; According to the shooting date of the current frame of RGB image data and multispectral image data, add spatio-temporal label to the dynamic elements; wherein the spatio-temporal label at least includes timestamp, season code and weather code; Based on the nearest point search algorithm, register the laser radar point cloud data collected in multiple historical periods, and construct a three-dimensional reference model of the target area in multiple historical periods according to the registered laser radar point cloud data; The fixed elements and dynamic elements are all labeled in the three-dimensional reference model, and the normalized vegetation index NDVI, the normalized water index NDWI and the space-time label are all embedded into the dynamic elements of the three-dimensional reference model to obtain the multi-modal prefabricated model.

3. The unmanned aerial vehicle surveying information processing method of a prefabricated model according to claim 1, wherein, Based on the U-Net network, the Transformer network and the cross-attention mechanism, a space-time coding change detection model is constructed, which specifically includes: Based on the U-Net convolutional neural network, a basic spatial network is constructed; wherein the basic spatial network includes an input layer, multiple layers of encoders, a bridging layer, multiple layers of decoders and an output layer; An alternating convolution layer with a preset hole rate is introduced at the bridging layer to expand the receptive field; a channel attention is added at the jump connection of each layer to suppress invalid band interference, obtaining a spatial change detection branch network; Based on the Transformer neural network architecture, a time change detection branch network is constructed; the time change detection branch network includes a block embedding layer for dividing each image in the input time sequence image sequence into 16x16 blocks; A cross-attention module is constructed for projecting the multi-scale feature maps output by the spatial change detection branch network into a feature matrix, and fusing the feature matrix with the time sequence features output by the time change detection branch network through a space-time attention calculation formula to output fused features; A change detection head is constructed for identifying the probability of change of each pixel according to the fused features to output a change probability map. The spatial change detection branch network, the time change detection branch network, the cross-attention module and the change detection head constitute the space-time coding change detection model.

4. The unmanned aerial vehicle surveying information processing method of a prefabricated model according to claim 1, wherein, Real-time drone data of a target area and multi-modal historical data loaded in the dynamic prefabricated model are respectively subjected to feature extraction to obtain real-time multi-modal features and historical multi-modal feature sequences, specifically including: During actual drone mapping, real-time drone data of a target area is collected by a drone; wherein the real-time drone data at least includes: RGB image data, laser radar point cloud data and multispectral image data; A dynamic prefabricated model corresponding to the target area is extracted from the prefabricated model library, and the multi-modal historical data therein are loaded and multiplied by the confidence weight of each element to obtain multi-modal weighted historical data; wherein the multi-modal historical data at least includes geometric data, spectral data and space-time labels; The real-time drone data and the multi-modal weighted historical data are respectively subjected to feature coding and alignment to obtain real-time multi-modal features and historical multi-modal feature sequences; wherein the feature coding at least includes at least one or more coding methods such as point cloud feature extraction, spectral time sequence coding and semantic embedding.

5. The unmanned aerial vehicle surveying and mapping information processing method of a prefabricated model according to claim 1, characterized in that, The real-time multi-modal features and historical multi-modal feature sequences are input into the space-time coding change detection model to obtain mapping change information of the target area, specifically including: The real-time multi-modal feature and the historical multi-modal feature sequence are input into the spatio-temporal coding change detection model at the same time, spatial multi-scale feature extraction and time sequence feature extraction are performed; The multi-scale feature is projected into a Query matrix through convolution, and the time sequence feature is expanded into a Key-Value matrix through a full connection layer; Based on the spatio-temporal attention calculation formula in the spatio-temporal coding change detection model, the fusion feature of the Query matrix and the Key-Value matrix is output; The fusion feature is input into the change detection head, the probability of each pixel in the real-time RGB image data and the real-time multi-spectral image data changing is determined, and the corresponding change probability map is generated; In the change probability map, the surveying and mapping change information of the target region is extracted; wherein the surveying and mapping change information at least includes the important change region position in the target region and the corresponding change type.

6. The unmanned aerial vehicle surveying information processing method of a prefabricated model according to claim 5, wherein, After obtaining the surveying and mapping change information of the target region, the method further comprises: When there is an important change region in the target region, the current frame of unmanned aerial vehicle real-time data is inserted into the dynamic pre-model to replace the oldest historical frame in the dynamic pre-model; For the unchanged region in the target region, only the time stamp of each element in the unchanged region in the dynamic pre-model is updated, and the geometric feature and spectral feature are retained, so as to realize the incremental update of the dynamic pre-model.

7. A pre-model unmanned aerial vehicle surveying information processing system, applying a pre-model unmanned aerial vehicle surveying information processing method according to any one of claims 1-6, characterized in that, The system comprises: A pre-model construction module is configured to construct a multi-modal pre-model based on historical unmanned aerial vehicle data collected in a historical period, and dynamically adjust the confidence weight of each element in the multi-modal pre-model according to real-time environmental parameters to obtain a dynamic pre-model; A real-time surveying and mapping module is configured to construct a spatio-temporal coding change detection model based on a U-Net network, a Transformer network and a cross attention mechanism, extract features from unmanned aerial vehicle real-time data of a target region and multi-modal historical data loaded in the dynamic pre-model to obtain real-time multi-modal features and a historical multi-modal feature sequence, and input the real-time multi-modal features and the historical multi-modal feature sequence into the spatio-temporal coding change detection model to obtain surveying and mapping change information of the target region.

Images