High-altitude aerial image road identification method based on deep learning

Abstract

The invention discloses a high-altitude aerial image road recognition method based on deep learning. The high-altitude aerial image road recognition method comprises the following steps: step 1, acquiring multi-source data; step 2, high-altitude aerial image enhancement; step 3, carrying out feature extraction and rasterization processing on the collected vehicle trajectory data; step 4, adding a SimAM attention mechanism module behind an encoder of the U2Net network, constructing the improved U2Net network, and respectively obtaining road prediction images corresponding to the high-altitude aerial image and the track raster image; and step 5, fusing the track grid image obtained in the step 4 and a road prediction image corresponding to the high-altitude aerial image in a pixel-level weighted combination mode to obtain a merged road identification prediction result. The invention belongs to the technical field of visual and remote sensing image processing, and solves the problems of insufficient multi-source data fusion and insufficient identification accuracy when coping with complex backgrounds and diversified ground feature features in the prior art.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual and remote sensing image processing, and relates to a high-altitude aerial image road recognition method based on deep learning. Background Art

[0002] With the rapid development of drone technology and satellite remote sensing technology, aerial images have been widely used in urban planning, traffic management, disaster emergency response and other fields. Aerial photos and drone images are widely used to generate and update road information, but due to the diversity of land use and land cover types and the complex details of the roads to be extracted, such as complex structures and texture features, it is still a technical challenge to quickly and accurately detect and extract road features from aerial images.

[0003] In recent years, deep learning technology has made significant progress in the field of image processing and computer vision, especially convolutional neural network (CNN) has shown excellent performance in target detection and image segmentation tasks. Although some existing road recognition methods based on deep learning have improved the recognition accuracy to a certain extent, they still have problems such as high demand for large-scale datasets, long model training time and insufficient robustness in different environments.

[0004] Therefore, there is an urgent need to develop a new method that can perform data enhancement on original high-altitude aerial images, and can cope with complex image backgrounds by fusing multi-source data to improve the performance of road recognition in high-altitude aerial images. Summary of the invention

[0005] The purpose of the present invention is to provide a high-altitude aerial image road recognition method based on deep learning, which solves the problems of insufficient multi-source data fusion and insufficient recognition accuracy in the prior art when dealing with complex backgrounds and diverse land features.

[0006] The technical solution adopted by the present invention is a method for road recognition based on high-altitude aerial images based on deep learning, which is implemented according to the following steps:

[0007] Step 1, multi-source data collection;

[0008] Step 2: high-altitude aerial image enhancement;

[0009] Step 3, extracting features and rasterizing the collected vehicle trajectory data;

[0010] Step 4: Add the SimAM attention mechanism module after the encoder of the U2Net network to build an improved U2Net network, and obtain the road prediction images corresponding to the aerial images and trajectory grid images respectively;

[0011] Step 5: The trajectory grid image obtained in step 4 and the road prediction image corresponding to the high-altitude aerial image are fused by pixel-level weighted combination to obtain a merged road recognition prediction result.

[0012] The beneficial effect of the present invention is that by collecting vehicle trajectory data and high-altitude aerial photography data, the trajectory data and aerial images are combined to provide more comprehensive and accurate road recognition information; at the same time, in order to reduce the influence of noise and irrelevant features on the recognition results, a fractional-order calculus image enhancement method and a SimAM attention mechanism module are introduced in the image enhancement and feature extraction module to selectively enhance the useful features of the road and suppress irrelevant features; then, the fusion of the trajectory data and the aerial image road recognition results is achieved through a pixel-level weighted fusion method. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of the overall process of the method of the present invention;

[0014] Figure 2 is the original high-altitude aerial image mentioned in the method of the present invention;

[0015] Figure 3 The method of the present invention is directed to Figure 2 Image enhanced using fractional derivatives;

[0016] Figure 4 is a flow chart of road prediction and recognition in the method of the present invention;

[0017] Figure 5 The aerial image of the urban road in Embodiment 1 of the present invention;

[0018] Figure 6 is the road recognition result of Example 1 of the present invention under the deeplabv3+ algorithm;

[0019] Figure 7 is the road recognition result of Example 1 of the present invention under the SegFormer algorithm;

[0020] Figure 8 is the road recognition result of Example 1 of the present invention under the method of the present invention;

[0021] Fig. 9 is a high-altitude aerial image of a mountain road in Example 2 of the present invention;

[0022] Fig.10 is the road recognition result of Example 2 of the present invention under the deeplabv3+ algorithm;

[0023] Fig.11 is the road recognition result of Example 2 of the present invention under the SegFormer algorithm;

[0024] Fig.12 is the road recognition result of Example 2 of the present invention under the method of the present invention;

[0025] Fig.13 The aerial image of the rural road in Embodiment 3 of the present invention;

[0026] Fig.14 is the road recognition result of Example 3 of the present invention under the deeplabv3+ algorithm;

[0027] Fig.15 is the road recognition result of Example 3 of the present invention under the SegFormer algorithm;

[0028] Fig.16 This is the road recognition result of Example 3 of the present invention using the method of the present invention. DETAILED DESCRIPTION

[0029] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] The high-altitude aerial image road recognition method of the present invention based on deep learning is implemented according to the following steps:

[0031] Step 1: Multi-source data collection,

[0032] The collected data includes vehicle trajectory data and high-altitude aerial photography data. The vehicle trajectory data comes from the vehicle's GPS records, and the (drone) high-altitude aerial photography data includes high-resolution visible light images. By combining trajectory data and aerial images, more comprehensive and accurate road identification information is provided.

[0033] Step 2: Enhance the aerial image.

[0034] The fractional calculus image enhancement method is used to enhance the high-altitude aerial image data. The specific process is as follows:

[0035] 2.1) Determine whether the image is a grayscale image. If it is not a grayscale image, the image needs to be converted into a grayscale image;

[0036] 2.2) Find the main direction of the image to be enhanced and determine the weighted multiple of the main direction;

[0037] 2.3) According to the weighted multiple of the main direction of the image to be enhanced, the fractional order template is modified accordingly;

[0038] 2.4) Apply the new fractional order template to the image to be enhanced to obtain the enhanced result image.

[0039] Step 3: trajectory feature selection and trajectory data rasterization.

[0040] The collected vehicle trajectory data is subjected to feature extraction and rasterization processing to be converted into image data. The specific process is as follows:

[0041] 3.1) Format the vehicle trajectory data into a standard form, filter and denoise the standard form trajectory data to improve the accuracy of the data. The expression of the vehicle trajectory data is:

[0042] Trajectory={t1, t2, t3…t n}

[0043] Among them, t k represents a trajectory point in the trajectory data, k = 1, 2, 3, ..., n, whose attributes consist of vehicle number, collection time, longitude coordinate, latitude coordinate, driving speed and driving direction at the time of collection;

[0044] 3.2) Calculate and obtain the characteristics of vehicle trajectory data, including speed, direction, density, average speed, standard deviation of speed, etc.;

[0045] 3.3) Using the rasterization method, the discrete trajectory point data is converted into grayscale image data as the input of the neural network according to the characteristics of the trajectory data. The function F(x,y) is used to represent the digital image. F(x,y) represents the grayscale of the image at the coordinate (x,y). The longitude and latitude coordinates (p lon ,p lat ) is converted to digital image coordinates (x, y) as shown below:

[0046] x=floor((p lon -Lon min )*Interval lon )

[0047] y=floor((Lat max -p lat )*Interval lat )

[0048] Among them, floor is the rounding function, Lon min Represents the minimum longitude value, Lat max Represents the maximum latitude value; Interval lon Indicates the number of pixels corresponding to 1 longitude, Interval lat Represents the number of pixels corresponding to 1 degree of latitude.

[0049] Step 4: Build the U2Net network to extract image features.

[0050] Add the SimAM attention mechanism module after the encoder of the U2Net network to enhance the existing U2Net network's attention to key features and build an improved U2Net network. The specific process is as follows:

[0051] 4.1) Build a U2Net network (full name: double-layer nested U-type network) and use the U2Net network to obtain feature information of high-altitude aerial photography data. The residual U block captures multi-scale information without reducing the resolution of the feature map, which to a certain extent makes up for the lack of attention to local details compared with global information. The expansion mode of the residual U block is used in the 5th layer (En5) and 6th layer (En6) of the encoder of the U2Net network and the 5th layer (De5) of the decoder of the U2Net network. The expansion convolution is used instead of the pooling and upsampling operations to obtain a residual U block (RSU-4F) with four feature levels, so that all intermediate feature maps of the residual U block have the same resolution as its input feature map;

[0052] 4.2) In order to filter out feature combinations that are helpful for feature recognition, enhance the U2Net network’s ability to express road features, and reduce the interference of background noise, a SimAM attention mechanism module is added after the encoder of the U2Net network. The SimAM attention mechanism module is a 3D attention module. The SimAM attention mechanism module directly infers 3-D weights from neurons, that is, it considers both spatial and channel dimensions, refines neurons, and derives a closed-form solution by defining an energy function to infer effective 3-D weights. The energy function of each neuron is defined as follows:

[0053]

[0054] In the formula, and is a linear transformation of t and xi, t is the target neuron in a single channel of the input feature, xi is the other neurons in a single channel of the input feature; i is the index in the spatial dimension, M = H × W is the number of neurons in the channel, ω t and b t are the transformed weights and biases, where all values ​​are scalars, and the improved U2Net network is obtained;

[0055] 4.3) The high-altitude aerial image and trajectory raster image enhanced in step 2 are respectively input into the improved U2Net network to obtain the road prediction images corresponding to the high-altitude aerial image and the trajectory raster image.

[0056] Step 5: fusion of results.

[0057] The trajectory grid image obtained in step 4 and the road prediction image corresponding to the high-altitude aerial image are fused by pixel-level weighted combination to obtain the merged road recognition prediction result. The specific process is:

[0058] The road recognition results of the trajectory grid image and the aerial image are fused using pixel-level weighted fusion. The intensity value of each pixel in the fused image is the weighted average of the intensity values ​​of the corresponding pixels in the trajectory data road extraction image and the remote sensing image road extraction image.

[0059] Assume that the pixel intensity value matrix of the trajectory data road extraction image is T, the pixel intensity value matrix of the remote sensing image road extraction image is R, and the pixel intensity value matrix of the fused image is F. The function of the fusion process is as follows:

[0060] F(i,j)=α·T(i,j)+β·R(i,j)

[0061] Where, F(i,j) represents the pixel intensity value of the fused image at position (i,j), T(i,j) represents the pixel intensity value of the trajectory data road extraction image at position (i,j), R(i,j) represents the pixel intensity value of the remote sensing image road extraction image at position (i,j), α and β represent the weighting coefficients of trajectory data and remote sensing image, respectively, and satisfy α+β=1;

[0062] From then on, the final road recognition prediction result is obtained.

[0063] Example 1

[0064] The implementation object of this embodiment 1 is a high-altitude aerial image of a town road, and a high-altitude aerial image road recognition method based on deep learning is adopted, and the following steps are performed:

[0065] Step 1, multi-source data collection: collect the road vehicle trajectory data and (drone) high-altitude aerial photography data of Example 1, the trajectory data comes from the vehicle's GPS record, and the high-altitude aerial photography data includes high-resolution visible light images.

[0066] The collected data includes vehicle trajectory data and aerial photography data. The trajectory data comes from the vehicle's GPS record, including vehicle number, collection time, longitude coordinates, latitude coordinates, driving speed and driving direction at the time of collection. The aerial photography data includes high-resolution visible light images. By combining trajectory data and aerial images, more comprehensive and accurate road recognition information is provided.

[0067] Step 2: High-altitude aerial image enhancement: The high-altitude aerial image data of Example 1 are enhanced using a fractional-order calculus image enhancement method.

[0068] 2.1) After confirming that it is not a grayscale image, convert the image into a grayscale image first.

[0069] 2.2) Find the main direction of the aerial data and determine the weighted multiple of the main direction. The main direction is determined by the direction histogram of the gradient of the aerial data. Let the image gradient be Calculate the horizontal gradient of an image and vertical gradient Its direction θ is:

[0070]

[0071] 2.3) According to the weighted multiples of the main direction of the high-altitude aerial photography data, the fractional order template is modified accordingly. Assuming the weighted multiple of the main direction is k, the formula of the fractional order template H(x,y) is as follows:

[0072] H(x,y)=H0(x,y)·k(θ)

[0073] Among them, H0(x,y) is the original fractional-order template, k(θ) is the weighted multiple adjusted according to the main direction, and (x,y) represents the pixel coordinates of the image.

[0074] 2.4) The new fractional order template is applied to the high-altitude aerial photography data of Example 1 to obtain an enhanced result. The specific operation is to perform a convolution operation on the image as shown in the following formula:

[0075] I enhanced (x,y)=I(x,y)*H(x,y)

[0076] Among them, * is the convolution operation, I(x,y) is the original image, I enhanced (x,y) is the enhanced image.

[0077] Step 3: trajectory feature selection and trajectory data rasterization. The collected vehicle trajectory data is subjected to feature extraction and rasterization processing to be converted into image data.

[0078] 3.1) The vehicle trajectory data is formatted into a standard form, and then the standard trajectory data is filtered and denoised. The vehicle's driving trajectory data is expressed as:

[0079] Trajectory={t1, t2, t3…t n}

[0080] Among them, t k Represents a trajectory point in the trajectory data. Its attributes consist of vehicle number, collection time, longitude coordinate, latitude coordinate, driving speed and driving direction at the time of collection.

[0081] 3.2) Calculate and obtain trajectory data features, including speed, direction, density, average speed, standard deviation of speed, etc.

[0082] 3.3) Use the rasterization method to convert the discrete trajectory point data into grayscale image data as the input of the neural network according to the trajectory point data characteristics. The function F(x,y) is used to represent the digital image. F(x,y) represents the grayscale of the image at the coordinate (x,y), and the longitude and latitude coordinates (p lon ,p lat ) is converted to digital image coordinates (x, y) as shown below:

[0083] x=floor((p lon -Lon min )*Interval lon )

[0084] y=floor((Lat max -p lat )*Interval lat )

[0085] Among them, floor is the rounding function, Lon min and Lat max Represents the minimum longitude value and the maximum latitude value respectively. lon Indicates the number of pixels corresponding to 1 longitude, Interval lat Represents the number of pixels corresponding to 1 degree of latitude.

[0086] Step 4: Build the U2Net network. Add the SimAM attention mechanism module after the encoder of the U2Net network to enhance the U2Net network's attention to key features and build an improved U2Net network.

[0087] 4.1) U2Net network (full name: double-layer nested U-shaped network), uses a double-layer nested U-shaped network to obtain the feature information of high-altitude aerial photography data. The residual U block captures multi-scale information without reducing the resolution of the feature map, which to a certain extent makes up for the lack of attention to local details compared with global information. The expansion mode of the residual U block is used in the 5th layer (En5), 6th layer (En6) of the encoder of the U2Net network and the 5th layer (De5) of the U2Net network decoder network, and the expansion convolution is used instead of the pooling and upsampling operations, so that all intermediate feature maps of the residual U block (RSU-4F) with four feature levels have the same resolution as its input feature map.

[0088] 4.2) In order to filter out feature combinations that are helpful for feature recognition, enhance the U2Net network’s ability to express road features, and reduce the interference of background noise, the SimAM attention mechanism module is integrated after the encoder of the U2Net network. The SimAM attention mechanism module is a 3D attention module. The SimAM attention mechanism module directly infers 3-D weights from neurons, that is, it considers both spatial and channel dimensions, refines neurons, and infers effective 3-D weights by deriving a closed form solution by defining an energy function. The energy function of each neuron is defined as follows:

[0089]

[0090] In the formula, and is a linear transformation of t and xi, where t and xi are the target neuron and other neurons in a single channel of the input feature. i is the index in the spatial dimension, M = H × W is the number of neurons in the channel, ω t and b t are the weights and biases of the transformation, where all values ​​are scalars.

[0091] 4.3) The trajectory raster image after rasterization of the trajectory data and the high-altitude aerial image enhanced by step 2 are respectively input into the improved U2Net network to obtain the road prediction images corresponding to the high-altitude aerial image and the trajectory raster image.

[0092] Step 5: fusion of results. The trajectory raster image and the aerial photography data enhanced by step 2 are fused with the corresponding road prediction image obtained by step 4 using a pixel-level weighted combination method to obtain a combined road recognition prediction result.

[0093] Using pixel-level weighted fusion, the trajectory raster image after rasterization of the trajectory data and the road recognition result of the high-altitude aerial photography data after image enhancement in step 2 are fused. Each pixel intensity value in the fused image is the weighted average of the road extraction image of the trajectory raster image after rasterization of the trajectory data and the corresponding pixel intensity value of the high-altitude aerial photography image after image enhancement in step 2. Let the pixel intensity value matrix of the road extraction image of the trajectory raster image after rasterization of the trajectory data be T, the pixel intensity value matrix of the high-altitude aerial photography data after image enhancement in step 2 be R, and the pixel intensity value matrix of the fused image be F. The fusion formula is as follows:

[0094] F(i,j)=α·T(i,j)+β·R(i,j)

[0095] Among them, F(i,j) represents the pixel intensity value of the fused image at position (i,j), T(i,j) represents the pixel intensity value of the trajectory grid image at position (i,j) after the trajectory data is rasterized, R(i,j) represents the pixel intensity value of the high-altitude aerial photography data at position (i,j) after image enhancement in step 2, α and β represent the weighting coefficients of the trajectory grid image and the high-altitude aerial photography data after image enhancement in step 2, respectively, and satisfy α+β=1.

[0096] Below Figure 5-Figure 8 The image shown in the figure is used as an example for experimental explanation. According to the above steps, the road recognition results of the method of the present invention for urban high-altitude aerial images are obtained, and compared with the road recognition results of the existing high-altitude aerial road recognition methods deeplabv3+ algorithm and SegFormer algorithm for urban high-altitude aerial images. Figure 5 High-altitude aerial images of town roads taken by drones; Figure 6 This is the road recognition result of the existing deeplabv3+ algorithm; Figure 7 It is the road recognition result of the prior art SegFormer algorithm; Figure 8 1 is the road recognition result of the method of the present invention. It can be seen that the method of the present invention has better effect on the road recognition result of the aerial image of the town.

[0097] At the same time, this embodiment 1 compares the method of the present invention with the prior art deeplabv3+ algorithm and SegFormer algorithm through precision (P), recall (R), F1 score (F1) and mean absolute error (MAE). The precision (P) measures the proportion of pixels predicted by the algorithm as roads that are actually roads. A higher precision means that the algorithm has fewer errors in road prediction. The calculation formula is:

[0098]

[0099] Among them, TP (True Positives) is the true positive, the number of pixels correctly detected as roads, FP (False Positives) is the false positive, the number of pixels incorrectly detected as roads, and the recall rate measures the proportion of pixels that are actually roads that are correctly detected as roads. The calculation formula is:

[0100]

[0101] Among them, TP (True Positives) represents true positives, the number of pixels correctly detected as roads, FN (False Negatives) represents false negatives, the number of pixels not correctly detected as roads, and F1 score (F1) is used to strike a balance between precision and recall. When one indicator is high and the other is low, the F1 score can provide a comprehensive evaluation. The calculation formula is as follows:

[0102]

[0103] Among them, P is the precision and R is the recall.

[0104] The mean absolute error (MAE) measures the average of the absolute values ​​of the difference between the predicted value and the true value, and is used to evaluate the accuracy of road detection. The calculation formula is as follows:

[0105]

[0106] Where n is the total number of pixels, y i is the true value of the ith pixel (road or non-road), is the predicted value (road or non-road) of the ith pixel.

[0107] Table 1 shows the experimental results of the method of the present invention compared with the prior art deeplabv3+ algorithm and SegFormer algorithm. As can be seen from Table 1, the precision, recall and F1 score values ​​of the method of the present invention are greater than the results of the two prior art methods, and the MAE value is less than the results of the two prior art methods, that is, the method of the present invention is better for implementing road recognition in urban aerial images.

[0108] Table 1. Comparison of experimental results of different methods in Example 1

[0109]

[0110] Example 2

[0111] According to the steps of Example 1, road recognition is performed on the aerial image of the mountain road.

[0112] Below Figure 9-12 The image shown is used as an example for experimental description. The road recognition results of the method of the present invention for high-altitude aerial images of mountain roads are obtained according to the steps of Example 1, and compared with the road recognition results of the prior art deeplabv3+ algorithm and SegFormer algorithm for high-altitude aerial images of mountain roads. Fig. 9 It is an aerial image of mountain roads; Fig.10 This is the road recognition result of the existing deeplabv3+ algorithm; Fig.11It is the road recognition result of the prior art SegFormer algorithm; Fig.12 This is the road recognition result of the present invention. It can be seen that the method of the present invention has better effect on the road recognition result of the high-altitude aerial image in the mountainous area.

[0113] Table 2 shows the experimental results of the method of the present invention compared with the prior art deeplabv3+ algorithm and SegFormer algorithm. As can be seen from Table 2, the precision, recall and F1 score values ​​of the method of the present invention are greater than the results of the two prior art methods, and the MAE value is less than the results of the two prior art methods, that is, the method of the present invention is better for road recognition in high-altitude aerial images of mountain roads.

[0114] Table 2. Comparison of experimental results of different methods in Example 2

[0115]

[0116] Example 3

[0117] According to the steps of Example 1, road recognition is performed on the aerial image of rural roads.

[0118] Below Figure 13-Figure 16 The image shown in the figure is used as an example for experimental description. The road recognition results of the method of the present invention for aerial images of rural roads are obtained according to the steps of Example 1, and compared with the road recognition results of the prior art deeplabv3+ algorithm and SegFormer algorithm for high-altitude aerial images of rural roads. Fig.13 High-altitude aerial images of rural roads; Fig.14 This is the road recognition result of the existing deeplabv3+ algorithm; Fig.15 It is the road recognition result of the prior art SegFormer algorithm; Fig.16 This is the road recognition result of the present invention. It can be seen that the method of the present invention has better road recognition effect on the aerial photography image of rural roads.

[0119] Table 3 shows the experimental results comparison between the method of the present invention and the prior art deeplabv3+ algorithm and SegFormer algorithm. As can be seen from Table 3, the precision, recall and F1 score values ​​of the method of the present invention are greater than the results of the two prior art methods, and the MAE value is less than the results of the two prior art methods, that is, the method of the present invention has a better effect on the road recognition of rural road aerial images.

[0120] Table 3. Comparison of experimental results of different methods in Example 3

[0121]

[0122] In summary, the method of the present invention uses a fractional-order calculus image enhancement method to enhance the original image data, effectively enhances road features, and reduces noise interference; at the same time, a SimAM attention mechanism module is introduced into the feature extraction module to further enhance the model's ability to express key features; through a pixel-level weighted fusion method, the trajectory data and the recognition results of the aerial image are fused to optimize the final road recognition effect, which significantly improves the problem of inaccurate road feature extraction in complex environments in the existing high-altitude aerial image road recognition method, and improves the accuracy and robustness of the model.

Claims

1. Road recognition method for aerial images based on deep learning, It is characterized in that Follow these steps to implement: Step 1, multi-source data collection; Step 2: high-altitude aerial image enhancement; Step 3, extracting features and rasterizing the collected vehicle trajectory data; Step 4: Add the SimAM attention mechanism module after the encoder of the U2Net network to build an improved U2Net network, and obtain the road prediction images corresponding to the aerial images and trajectory grid images respectively; Step 5: The trajectory grid image obtained in step 4 and the road prediction image corresponding to the high-altitude aerial image are fused by pixel-level weighted combination to obtain a merged road recognition prediction result.

2. The method for road recognition in high-altitude aerial images based on deep learning according to claim 1, It is characterized in that In step 1, the specific process is: The collected data include vehicle trajectory data and high-altitude aerial photography data. The vehicle trajectory data comes from the vehicle's GPS records, and the high-altitude aerial photography data includes high-resolution visible light images.

3. The method for road recognition in high-altitude aerial images based on deep learning according to claim 1, It is characterized in that In step 2, the high-altitude aerial image data is enhanced using the fractional calculus image enhancement method. The specific process is as follows: 2.1) Determine whether the image is a grayscale image. If not, convert the image into a grayscale image. 2.2) Find the main direction of the image to be enhanced and determine the weighted multiple of the main direction; 2.3) According to the weighted multiple of the main direction of the image to be enhanced, the fractional order template is modified accordingly; 2.4) Apply the new fractional order template to the image to be enhanced to obtain the enhanced result image.

4. The method for road recognition in high-altitude aerial images based on deep learning according to claim 1, It is characterized in that In step 3, the specific process is: 3.1) Format the vehicle trajectory data into a standard form, filter and denoise the standard form trajectory data, and the expression of the vehicle trajectory data is: Trajectory={t 1 、t 2 、t 3 …t n } Among them, t k represents a trajectory point in the trajectory data, k = 1, 2, 3, ..., n, whose attributes consist of vehicle number, collection time, longitude coordinate, latitude coordinate, driving speed and driving direction at the time of collection; 3.2) Calculate the characteristics of vehicle trajectory data, including speed, direction, density, average speed, and standard deviation of speed; 3.3) Using the rasterization method, the discrete trajectory point data is converted into grayscale image data as the input of the neural network according to the characteristics of the trajectory data. The function F(x,y) is used to represent the digital image. F(x,y) represents the grayscale of the image at the coordinate (x,y). The longitude and latitude coordinates (p lon ,p lat ) is converted to digital image coordinates (x, y) as shown below: x=floor((p lon -Lon min )*Interval lon ) y=floor((Lat max -p lat )*Interval lat ) Among them, floor is the rounding function, Lon min Represents the minimum longitude value, Lat max Represents the maximum latitude value; Interval lon Indicates the number of pixels corresponding to 1 longitude, Interval lat Represents the number of pixels corresponding to 1 degree of latitude.

5. The method for road recognition in high-altitude aerial images based on deep learning according to claim 1, It is characterized in that In step 4, the specific process is: 4.1) Build a U2Net network and use it to obtain feature information of aerial photography data. The residual U block captures multi-scale information without reducing the resolution of the feature map. The dilation mode of the residual U block is used in the 5th and 6th layers of the encoder of the U2Net network and the 5th layer of the decoder of the U2Net network. The dilated convolution is used instead of the pooling and upsampling operations to obtain a residual U block with four feature levels, so that all intermediate feature maps of the residual U block have the same resolution as its input feature map. 4.2) The SimAM attention mechanism module is added after the encoder of the U2Net network. The SimAM attention mechanism module directly infers 3-D weights from neurons, that is, it considers both spatial and channel dimensions, refines neurons, and infers effective 3-D weights by defining an energy function to derive a closed form solution. The energy function of each neuron is defined as follows: In the formula, and is a linear transformation of t and xi, t is the target neuron in a single channel of the input feature, and xi is the other neurons in a single channel of the input feature; i is the index in the spatial dimension, M = H × W is the number of neurons in the channel, ω t and b t are the transformed weights and biases, where all values ​​are scalars, and the improved U2Net network is obtained; 4.3) The high-altitude aerial image and trajectory raster image enhanced in step 2 are respectively input into the improved U2Net network to obtain the road prediction images corresponding to the high-altitude aerial image and the trajectory raster image.

6. The method for road recognition in high-altitude aerial images based on deep learning according to claim 1, It is characterized in that In step 5, the specific process is: The road recognition results of the trajectory grid image and the aerial image are fused using pixel-level weighted fusion. The intensity value of each pixel in the fused image is the weighted average of the intensity values ​​of the corresponding pixels in the trajectory data road extraction image and the remote sensing image road extraction image. Assume that the pixel intensity value matrix of the trajectory data road extraction image is T, the pixel intensity value matrix of the remote sensing image road extraction image is R, and the pixel intensity value matrix of the fused image is F. The function of the fusion process is as follows: F(i,j)=α·T(i,j)+β·R(i,j) Where, F(i,j) represents the pixel intensity value of the fused image at position (i,j), T(i,j) represents the pixel intensity value of the trajectory data road extraction image at position (i,j), R(i,j) represents the pixel intensity value of the remote sensing image road extraction image at position (i,j), α and β represent the weighting coefficients of trajectory data and remote sensing image, respectively, and satisfy α+β=1; From then on, the final road recognition prediction result is obtained.

Images