Video landslide monitoring positioning method based on geographic position positioning algorithm

Abstract

The application discloses a video landslide monitoring and positioning method based on a geographic position positioning algorithm, and comprises the following steps: obtaining a landslide detection model by training a target detection network; obtaining a landslide positioning model by training a landslide positioning network; selecting a to-be-measured area, acquiring remote sensing images of the to-be-measured area and real-time to-be-measured photos; sequentially inputting the to-be-measured photos into the landslide detection model, outputting position coordinates of a landslide area in the to-be-measured photos when the landslide area is identified in the to-be-measured photos, and cutting out to-be-measured image slices from the to-be-measured photos; and inputting the to-be-measured image slices and the remote sensing images of the to-be-measured area into the landslide positioning model to obtain longitude and latitude coordinates of the landslide area. The application can realize real-time monitoring of the to-be-measured area and automatically output accurate positioning of the landslide area. The application is not affected by environment and weather, has simple requirements on equipment, is accurate in positioning, and has a small error range.

Description

TECHNICAL FIELD

[0001] The present application relates to a landslide positioning method, in particular to a video landslide monitoring positioning method based on a geographic position positioning algorithm. BACKGROUND

[0002] The existing landslide automatic monitoring method based on image recognition technology can be roughly divided into two categories.

[0003] One kind is to use real-time monitoring data such as network monitoring camera and unmanned aerial vehicle patrol collected pictures, and use image detection method for real-time analysis, which can analyze whether landslide occurs in the monitoring picture. If only using camera to monitor landslide, when landslide occurs, we only know that landslide occurs in the picture of the current camera, and the actual landslide point may be thousands of kilometers away from the camera erection position, and the specific landslide point position usually needs to be determined manually.

[0004] The other kind is based on remote sensing image, and uses image detection algorithm to identify landslide on remote sensing image. The method based on real-time monitoring picture can analyze whether landslide occurs in the picture in real time, but cannot accurately locate the longitude and latitude coordinate specific position information of the landslide point. The method based on remote sensing image cannot monitor landslide in real time because the acquisition of remote sensing image is not real-time or there is cloud cover. SUMMARY

[0005] The purpose of the present application is to provide a video landslide monitoring positioning method based on a geographic position positioning algorithm, which can analyze whether landslide occurs in the current area in real time through monitoring picture, and obtain the longitude and latitude coordinate information of the landslide point in real time.

[0006] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows: a video landslide monitoring positioning method based on a geographic position positioning algorithm, comprising the following steps:

[0007] (1) training a target detection network to obtain a landslide detection model;

[0008] The target detection network is trained by using the field photos marked with landslide area and position coordinates to obtain a landslide detection model, the landslide area is a rectangular area marked on the field photo, the position coordinates are the coordinates of the landslide area on the field photo, and the landslide detection model is used to input the field photo marked with landslide area and position coordinates and output the position coordinates;

[0009] (2) training a landslide positioning network to obtain a landslide positioning model;

[0010] (21) For each field photo, obtain the corresponding remote sensing image, cut out an image slice containing only the landslide area from the field photo, find the area corresponding to the image slice on the remote sensing image, and mark the latitude and longitude coordinates of the area as the latitude and longitude coordinates of the image slice;

[0011] (22) Construct a landslide positioning network, use the image slice and the remote sensing image marked with latitude and longitude coordinates as input, and the latitude and longitude coordinates as output, train the landslide positioning network to obtain a landslide positioning model;

[0012] (3) Select a to-be-detected area, obtain the remote sensing image of the to-be-detected area, and arrange video acquisition devices in the to-be-detected area to obtain real-time video of the to-be-detected area, and divide the video into a plurality of to-be-detected photos according to frames;

[0013] (4) The to-be-detected photos of step (3) are sequentially input into the landslide detection model, and when a landslide area is identified in the to-be-detected photo, the position coordinates of the landslide area in the corresponding to-be-detected photo are output;

[0014] (5) The landslide area is cut out from the to-be-detected photo to obtain a to-be-detected image slice, which is input into the landslide positioning model together with the remote sensing image of the to-be-detected area to obtain an output, and a latitude and longitude coordinate is output after a non-maximum suppression module, which is used as the latitude and longitude coordinates of the landslide area.

[0015] As preferred: the step (1) is specifically;

[0016] (11) Collect a plurality of field photos in different areas, each field photo containing at least one landslide area;

[0017] (12) For each field photo, manually mark the landslide area and record the coordinates of each landslide area in the field photo as the position coordinates of the landslide area, and use the position coordinates as the label of the field photo to constitute sample data, and all sample data constitute a landslide detection dataset;

[0018] (13) Establish a target detection network, input the sample data of the landslide detection dataset in sequence, and use the corresponding label as the expected output to train the target detection network to obtain a landslide detection model.

[0019] As preferred: step (22) is specifically;

[0020] The landslide positioning network includes a deep learning backbone network, a ResNet residual module, and an input detection head, and the training process is as follows:

[0021] (22-1) A group of slice images and corresponding remote sensing images are input into the landslide positioning network;

[0022] (22-2) the remote sensing image is output after being subjected to a deep learning backbone network and down-sampling, and the output size of a first feature map is AxAxB, wherein A is a multiple of 4, A is greater than or equal to 64, B is a multiple of 2, and B is greater than or equal to 16;

[0023] the slice image is output after being subjected to a resnet residual module and down-sampling, and the output size of a second feature map is 3x3xB;

[0024] the second feature map is subjected to a convolution operation with the first feature map as a convolution kernel, and a third feature map with a size of AxAxB is obtained;

[0025] the third feature map is subjected to twice down-sampling and convolution operation, and a fourth feature map with a size of 1 / 2A x 1 / 2A x B is obtained in the first time, and a fifth feature map with a size of 1 / 4A x 1 / 4A x B is obtained in the second time;

[0026] the third feature map, the fourth feature map and the fifth feature map are simultaneously input into a detection head, and a predicted latitude and longitude coordinate of the slice image is output;

[0027] (22-3) the predicted latitude and longitude and the latitude and longitude coordinate marked in step (21) are input into a loss function to calculate a loss value, and the network weight of the landslide positioning network is updated according to the loss value;

[0028] (22-4) steps (22-1) to (22-3) are repeated until the loss value no longer decreases, and a landslide positioning model is obtained.

[0029] As preferred: the deep learning backbone network is a ResNet backbone network, an EfficientNet network or a MoblieNet network.

[0030] Compared with the prior art, the present application has the following advantages:

[0031] The present application trains two network models, the first one is a landslide detection model, which is trained by a target detection network using a field photo marked with a landslide area and a position coordinate, and has the ability to identify a landslide area and output the position coordinate of the landslide area in the field photo. The second one is a landslide positioning model, which uses an image slice containing a landslide area and a remote sensing image as input, and outputs the latitude and longitude coordinate of the image slice. In this way, during the actual measurement, we only need to collect the video of the area to be measured in real time, input it into the landslide detection model for monitoring, and when a landslide area is detected, the position coordinate of the landslide area in the corresponding photo to be measured is output, which is subjected to slicing processing and sent into the landslide positioning model together with the remote sensing image for positioning.

[0032] In summary, the present application overcomes the defects in the prior art that the image detection method can only analyze whether a landslide occurs in the monitoring picture, but cannot obtain the latitude and longitude information of the landslide position, and the remote sensing image is not real-time, and cannot be monitored in real time due to the influence of weather and environment, so that accurate positioning cannot be achieved. The present application can monitor the to-be-detected area in real time and automatically output accurate positioning of the landslide area through simple real-time video monitoring and two network models. It is not affected by the environment and weather, has simple requirements for equipment, accurate positioning, and small error range. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a flowchart of the present application;

[0034] Figure 2 is a flowchart of the training and use of the landslide positioning network;

[0035] Figure 3 is a schematic diagram of marking the landslide area in the field photo and cutting an image slice;

[0036] Figure 4 is a schematic diagram of training the landslide positioning network using an image slice and a remote sensing image as input. EMBODIMENT

[0037] The present application will be further described below with reference to the accompanying drawings.

[0038] Example 1: see Figure 1 and Figure 2 A video landslide monitoring and positioning method based on a geographic position positioning algorithm, comprising the following steps:

[0039] (1) training a target detection network to obtain a landslide detection model;

[0040] The target detection network is trained using field photos marked with landslide areas and position coordinates to obtain a landslide detection model. The landslide area is a rectangular area marked on the field photo, and the position coordinates are the coordinates of the landslide area on the field photo. The landslide detection model is used to input field photos marked with landslide areas and position coordinates, and output position coordinates. Step (1) is implemented by the following three steps:

[0041] (11) collect multiple field photos in different areas, each field photo containing at least one landslide area;

[0042] (12) for each field photo, manually mark the landslide area and record the coordinates of each landslide area in the field photo as the position coordinates of the landslide area, and use the position coordinates as the label of the field photo to form sample data, and all sample data form a landslide detection data set;

[0043] (13) A target detection network is established, sample data of the landslide detection data set is input into the target detection network in sequence, and the corresponding label is taken as the expected output, so that the landslide detection model is obtained by training the target detection network.

[0044] (2) The landslide positioning network is trained to obtain the landslide positioning model.

[0045] (21) For each field photo, the corresponding remote sensing image is obtained, the image slice containing only the landslide area is cut out from the field photo, the area corresponding to the image slice is found on the remote sensing image, and the latitude and longitude coordinates of the area are marked as the latitude and longitude coordinates of the image slice.

[0046] (22) A landslide positioning network is constructed, the image slice and the remote sensing image with the marked latitude and longitude coordinates are taken as the input, and the latitude and longitude coordinates are taken as the output, so that the landslide positioning model is obtained by training the landslide positioning network.

[0047] (3) A to-be-detected area is selected, the remote sensing image of the to-be-detected area is obtained, and a video acquisition device is arranged in the to-be-detected area to obtain the video of the to-be-detected area in real time, and the video is divided into a plurality of to-be-detected photos by frames.

[0048] (4) The to-be-detected photos of step (3) are input into the landslide detection model in sequence, and when a landslide area is identified in the to-be-detected photo, the position coordinates of the landslide area in the corresponding to-be-detected photo are output.

[0049] (5) The landslide area is cut out from the to-be-detected photo to obtain a to-be-detected image slice, which is input into the landslide positioning model together with the remote sensing image of the to-be-detected area to obtain an output, and a latitude and longitude coordinate is output after a non-maximum suppression module, which is taken as the latitude and longitude coordinates of the landslide area.

[0050] In this embodiment, step (22) is specifically;

[0051] The landslide positioning network comprises a deep learning backbone network, a ResNet residual module and an input detection head, and the training process thereof is as follows:

[0052] (22-1) The slice image and the corresponding remote sensing image are input into the landslide positioning network.

[0053] (22-2) After the remote sensing image is input into the deep learning backbone network and down-sampled, a first feature map with a size of AxAxB is output, wherein A is a multiple of 4 and A is greater than or equal to 64, and B is a multiple of 2 and B is greater than or equal to 16.

[0054] After the slice image is input into the resnet residual module and down-sampled, a second feature map with a size of 3x3xB is output.

[0055] Convolve the second feature map as a convolution kernel with the first feature map to obtain a third feature map with a size of AxAxB;

[0056] Repeat the downsampling and convolution operation twice on the third feature map, to obtain a fourth feature map with a size of 1 / 2A*1 / 2A*B in the first time and a fifth feature map with a size of 1 / 4A*1 / 4A*B in the second time;

[0057] After the third feature map, the fourth feature map and the fifth feature map are simultaneously sent into the detection head, the predicted latitude and longitude coordinates of the slice image are outputted;

[0058] (22-3) The predicted latitude and longitude are sent into a loss function with the latitude and longitude coordinates marked in step (21) to calculate a loss value, and the network weight of the landslide positioning network is updated according to the loss value;

[0059] (22-4) Repeat steps (22-1) to (22-3) until the loss value no longer decreases, and obtain a landslide positioning model.

[0060] The deep learning backbone network is a ResNet backbone network, an EfficientNet network or a MoblieNet network.

[0061] Embodiment 2: Referring to Figures 1 to 4 , on the basis of embodiment 1, a more specific implementation is given.

[0062] Regarding step (1) of training the landslide detection model. When constructing the landslide detection data set, we need to collect a large number of field photos. Specifically, we can collect them in the field at different locations. After discovering a landslide, a camera is set up or a mobile phone is directly used to take a picture, and a field photo of the landslide can be obtained. The field photo is manually marked to mark the landslide area, which is generally realized by framing a rectangular area. In a field photo, there may be multiple landslide areas, which are marked one by one. For each landslide area, the position coordinates in the field photo are recorded, and the position coordinates are taken as the label of the landslide area, which is one-to-one corresponding to form a sample pair. This sample pair is a sample data. We can make several thousand sample data by collecting several thousand field photos in the field. Referring to Figure 3 , a schematic diagram of marking three landslide areas in a field photo is given. When training the target detection network, a sample pair is sent into the network, and the label in the sample pair is taken as the expected output to train the network. The network can automatically update the internal parameters to obtain a landslide detection model based on Figure 3 . For three landslide areas based on

[0063] As for step (2), in this embodiment, the deep learning backbone network in the landslide positioning model is a ResNet backbone network.

[0064] Since the input of the landslide positioning model is two images, the image slice and the remote sensing image. Therefore, we need to obtain the corresponding matching image slice and remote sensing image. As for the image slice, it is directly cut from the field photo, and only the landslide area is cut out, which can be used as the image slice. When a field photo contains multiple landslide areas, we can get multiple image slices. See Figure 3 , Figure 3 The right side shows a schematic diagram of cutting an image slice, and the rest of the image slices are the same as this method. As for the remote sensing image, it is obtained by satellite to obtain the remote sensing image of the area where the field photo is located. The remote sensing image can at least completely cover the geographic area in the field photo.

[0065] In step (21), before inputting into the landslide positioning network, we need to pre-process the data, that is, to find the area of the image slice on the remote sensing image and mark the latitude and longitude coordinates of the area as the latitude and longitude coordinates of the image slice. In step (22) of training, we take the image slice and the remote sensing image with marked latitude and longitude coordinates as input, and the latitude and longitude coordinates as expected output to train the network and adjust the network parameters. See Figure 4 , Figure 4 A schematic diagram of training with an image slice and a remote sensing image as input is shown, Figure 4 The larger image in the middle is a remote sensing image, and the smaller image is an image slice obtained in step (21).

[0066] When training the landslide positioning network, first to fifth feature maps need to be generated. In this embodiment, A=112, B=32, the first feature map size is 112x112x32, the second feature map size is 3x3x32, the third feature map size is 112x112x32, the fourth feature map size is 56x56x32, and the fifth feature map size is 28x28x32.

[0067] After the model of the present application is trained, it can be used for actual measurement. In actual measurement, a to-be-measured area is selected, and a video acquisition device is arranged, which is generally a camera or the like, as long as it can obtain the video of the to-be-measured area in real time. The monitoring area of the video acquisition device is large, and its arrangement position may be thousands of kilometers away from the landslide point. After the to-be-measured area is determined, the remote sensing image of the to-be-measured area is obtained from the satellite remote sensing data. In this way, the monitoring preparation is completed.

[0068] In actual monitoring, only the landslide detection model works, the field video source is continuously sent to the upper computer, is divided into a plurality of to-be-detected photos according to frames, and then is sent into the landslide detection model in sequence, if there is no landslide area, the landslide detection model has no output, when a landslide area in a to-be-detected photo is recognized, the position coordinates of the landslide area in the to-be-detected photo are output, which is beneficial to cutting out the landslide area from the to-be-detected photo to obtain a to-be-detected image slice.

[0069] At this time, the landslide positioning model is started again, the to-be-detected image slice and the remote sensing image are input, and the longitude and latitude coordinates of the landslide area are output.

[0070] Based on the method of the present application, not only can the landslide be monitored in real time, but also the longitude and latitude coordinates of the landslide can be output, and the accuracy is high.

[0071] The above only describes the preferred embodiments of the present application and is not used to limit the present application, any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A video landslide monitoring positioning method based on a geographic position positioning algorithm, characterized in that: Comprise the following steps; (1) training target detection network to obtain landslide detection model; Train the target detection network with the field photos marked with landslide area and position coordinates, the landslide area is a rectangular area marked on the field photo, the position coordinates are the coordinates of the landslide area on the field photo, and the landslide detection model is used to input the field photo marked with the landslide area and the position coordinates and output the position coordinates; (2) training landslide positioning network to obtain landslide positioning model; (21) for each field photo, obtain its corresponding remote sensing image, cut out an image slice containing only the landslide area from the field photo, find the area corresponding to the image slice on the remote sensing image, and mark the latitude and longitude coordinates of the area as the latitude and longitude coordinates of the image slice; (22) construct a landslide positioning network, input the image slice and the remote sensing image marked with latitude and longitude coordinates, and output the latitude and longitude coordinates, train the landslide positioning network to obtain the landslide positioning model; (3) select a to-be-detected area, obtain the remote sensing image of the to-be-detected area, and arrange video acquisition devices in the to-be-detected area to obtain real-time video of the to-be-detected area, and divide the video into a plurality of to-be-detected photos according to frames; (4) input the to-be-detected photos of step (3) into the landslide detection model in sequence, and when a landslide area is recognized in the to-be-detected photo, output the position coordinates of the landslide area in the corresponding to-be-detected photo; (5) cut out the landslide area from the to-be-detected photo to obtain a to-be-detected image slice, input the to-be-detected image slice and the remote sensing image of the to-be-detected area into the landslide positioning model to obtain an output, and output a latitude and longitude coordinate after a non-maximum suppression module, as the latitude and longitude coordinates of the landslide area; Step (22) is specifically that the landslide positioning network comprises a deep learning backbone network, a ResNet residual module and an input detection head, and the training process is as follows: (22-1) a group of slice images and corresponding remote sensing images are input into the landslide positioning network; (22-2) after the remote sensing image is input into the deep learning backbone network and down-sampled, a first feature map with a size of AxAxB is output, wherein A is a multiple of 4 and A is greater than or equal to 64, and B is a multiple of 2 and B is greater than or equal to 16; After the slice image is input into the resnet residual module and down-sampled, a second feature map with a size of 3x3xB is output; The second feature map is used as a convolution kernel to perform convolution operation on the first feature map to obtain a third feature map with a size of AxAxB; The third feature map is down-sampled and convolved twice, the first time to obtain a fourth feature map with a size of 1 / 2A x 1 / 2A x B, and the second time to obtain a fifth feature map with a size of 1 / 4A x 1 / 4A x B; After the third feature map, the fourth feature map and the fifth feature map are input into the detection head at the same time, the predicted latitude and longitude coordinates of the slice image are output; (22-3) the predicted latitude and longitude coordinates and the latitude and longitude coordinates marked in step (21) are input into a loss function to calculate a loss value, and the network weight of the landslide positioning network is updated according to the loss value; (22-4) repeat steps (22-1) to (22-3) until the loss value no longer decreases, and obtain the landslide positioning model.

2. The video landslip monitoring positioning method based on a geopositioning algorithm according to claim 1, characterized in that: The step (1) is specifically; (11) Collect multiple on-site photos in different areas, and each on-site photo contains at least one landslide area; (12) For each on-site photo, manually mark the landslide area, record the coordinates of each landslide area in the on-site photo as the position coordinates of the landslide area, and take the position coordinates as the label of the on-site photo to form sample data, and all sample data form a landslide detection data set; (13) Establish a target detection network, input the sample data of the landslide detection data set in turn, and take the corresponding label as the expected output, and train the target detection network to obtain a landslide detection model.

3. The video landslip monitoring positioning method based on a geopositioning algorithm according to claim 1, characterized in that: The deep learning backbone network is a ResNet backbone network, an EfficientNet network or a MoblieNet network.

Images