Sky-ground integrated remote sensing emergency monitoring method for road safety after flood

Abstract

The invention discloses a sky-ground integrated remote sensing emergency monitoring method for post-flood road safety, and relates to the technical field of remote sensing monitoring. The method comprises the following steps that an unmanned aerial vehicle collects highway pavement video image data after flood, and a terrain flood model is constructed; rainfall early warning sites are marked in the terrain flood model, and a corresponding flood early warning requirement is marked for each rainfall early warning site; and obtaining a road model in the terrain flood model, and generating an obstacle model on the road model according to the real-time data collected by the unmanned aerial vehicle. According to the invention, the terrain flood model is constructed for the video image collected by the unmanned aerial vehicle, the rainfall threshold value of each area is marked in the terrain flood model, and when the geological disaster occurs, the actual condition of the obstacle model is displayed on the terrain flood model, so that the timeliness and accuracy of flood early warning are improved, and the occurrence of the geological disaster is displayed in time. Obstacles on the road are visually displayed, and rescue after disasters can be conveniently and rapidly carried out.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing monitoring, and in particular relates to a sky-ground integrated remote sensing emergency monitoring method for highway safety after a flood. Background Art

[0002] Floods are a common and extremely destructive natural disaster, and their frequency is on the rise. The strong impact of water flow can directly destroy the roadbed and road surface, causing cracks, collapse and other serious damage to the road surface, destroying the structural integrity of the road. The large amount of silt and debris carried by the flood will also accumulate on the road, blocking traffic and affecting the normal passage of the road. Mountain roads may also cause secondary disasters such as landslides and mud-rock flows during floods, burying or destroying the road, posing a serious threat to road safety and causing huge property losses and casualties.

[0003] The limitations of traditional highway safety monitoring methods include the following specific defects:

[0004] (1) Limited monitoring scope: Traditional manual inspection methods can only inspect local areas on the road that are accessible to personnel, and it is difficult to cover large areas of disaster-stricken areas. Manual inspections are even more difficult to carry out in remote mountainous areas or areas with blocked traffic. Even if fixed ground monitoring equipment is used, its monitoring range is relatively small, and it is impossible to grasp the overall disaster situation of the highway from a macro perspective.

[0005] (2) Poor timeliness: After a flood occurs, infrastructure such as transportation and communications are often damaged. Manual inspections and data transmission are slow, making it difficult to promptly transmit road disaster information to relevant departments, resulting in delays in emergency rescue and repair work.

[0006] (3) Insufficient data accuracy: Manual judgment of road damage is subjective and erroneous. Some hidden damage, such as erosion and voids inside the roadbed, is difficult to detect through manual inspections. The data obtained by ground monitoring equipment may also be inaccurate due to interference from environmental factors.

[0007] With the continuous development of remote sensing technology, a single remote sensing data source can no longer meet the needs of highway safety monitoring after floods. It is necessary to integrate multi-source data such as satellite remote sensing, aerial remote sensing and ground monitoring to give full play to their respective advantages and achieve all-round and multi-level monitoring of highway safety. Therefore, there is an urgent need for a method that combines artificial intelligence, big data and other technologies to analyze and process the massive remote sensing data obtained by integrating the sky and the ground, realize the functions of automatic identification and risk assessment of highway disasters, improve the intelligent level of monitoring, and better ensure highway safety. Summary of the invention

[0008] The purpose of the present invention is to provide a sky-ground integrated remote sensing emergency monitoring method for highway safety after floods. A terrain flood model is constructed based on video images collected by unmanned aerial vehicles, and the rainfall threshold of each area is marked in the terrain flood model. When a geological disaster occurs, the actual situation of the obstacle model is displayed on the terrain flood model, thereby solving the problems of limited monitoring range, poor timeliness and insufficient data accuracy of existing flood warning equipment.

[0009] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0010] The present invention is a sky-ground integrated remote sensing emergency monitoring method for highway safety after a flood, comprising the following steps:

[0011] Step S1: The drone collects video image data of the road surface after the flood;

[0012] Step S2: pre-processing the highway pavement video image data to construct a three-dimensional highway model;

[0013] Step S3: calibrate the three-dimensional highway model and construct a terrain flood model;

[0014] Step S4: Count the real-time rainfall and calculate the total rainfall, and map it into the constructed terrain flood model;

[0015] Step S5: marking rainfall warning sites in the terrain flood model, and marking corresponding flood warning requirements for each rainfall warning site;

[0016] Step S6: Obtain the highway modeling in the terrain flood model. When a natural disaster occurs and an obstacle blocks the highway, an obstacle model is generated on the highway modeling according to the real-time data collected by the drone.

[0017] Step S7: When the collected real-time rainfall exceeds the preset threshold, early warning processing is performed on the terrain flood model. When a geological disaster occurs, a visualization of the obstacle model is also performed on the terrain flood model.

[0018] As a preferred technical solution, in step S1, the highway pavement video image data is collected by multiple drones for high-altitude image acquisition; when the drones shoot video images, the flight altitude of the drones needs to be set according to the drone working parameters, and the specific setting formula is as follows:

[0019] H = f × GSD / a;

[0020] Where W H It represents the relative altitude in meters, GSD represents the ground resolution of the image, f represents the focal length of the camera, and a represents the pixel size.

[0021] As a preferred technical solution, in step S2, the construction process of the three-dimensional highway model is as follows:

[0022] Step S21: converting the original video image data into point cloud data, and performing denoising, filtering and registration operations on the point cloud data;

[0023] Step S22: using a Poisson surface reconstruction algorithm to fit the point cloud data into a smooth surface, and constructing a three-dimensional surface model of the highway;

[0024] Step S23: combining the semantic segmentation results in the image, assigning different semantic labels to different areas of the road, and marking and displaying them in the three-dimensional model;

[0025] Step S24: Conduct field measurements of the location and features of the highway, compare the measured data with the corresponding data in the three-dimensional model, calculate the error, and evaluate the model accuracy.

[0026] As a preferred technical solution, in step S22, the Poisson surface reconstruction algorithm calculates the normal vector for each point cloud point by analyzing the covariance matrix of its neighborhood points. The specific formula is as follows:

[0027]

[0028] In the formula, is the neighborhood point set N i The center of mass, For the covariance matrix, p i is a point in the point cloud;

[0029] In the discrete case, for a point (i, j, k) in a three-dimensional grid, assuming that its neighboring points are (i±1, j, k), (i, j±1, k), (i, j, k±1), the normal vector field N = (N x ,N y ,N z ) is calculated as follows:

[0030]

[0031] Where h is the grid spacing;

[0032] In the marching cubes algorithm, for a cube unit, the scalar values ​​of its eight vertices are f i (i=1,2,3,...,8), assuming the threshold is, the points on the isosurface are calculated by linear interpolation.

[0033] As a preferred technical solution, in step S3, the terrain flood model calculation formula is:

[0034]

[0035] Where P is the rainfall, q is the intermediate parameter, which represents the precipitation coefficient k and the water level rise speed V. g , h is the road surface height, P 0 is the rainfall on the road surface, M is the water vapor flux of water evaporation, h * It indicates the height when the road surface water reaches saturation, mb is a parameter, generally taken as 0.0255, V indicates the road surface water flow velocity, θ is the angle between the water flow direction and the slope direction, and α is the slope.

[0036] As a preferred technical solution, in step S4, the calculation formula for the total rainfall is as follows:

[0037]

[0038] In the formula, the rainfall area is divided into j grids, L nj Indicates the radius of rainfall calculation within the grid, P cj represents the rainfall at the rainfall center, P nj Represents the amount of rainfall at the edge of the grid.

[0039] As a preferred technical solution, in step S5, the study area is divided into 20, 18, 15, 13, 10 and 8 grids respectively, and the area less than half of the grid area is merged into the adjacent grid. The mean rainfall of the rain warning station in each grid is calculated, and the rain warning station closest to the mean rainfall in the grid is selected to further deduce the rainfall of other rain warning stations in the grid. The total rainfall calculation formula in step S4 is used to calculate the surface rainfall. When the accuracy guarantee rate of the surface rainfall calculation result is greater than 90%, the number of grid divisions is determined, and finally the number of rain warning stations to be set up in the area is determined.

[0040] As a preferred technical solution, in step S6, the specific formula for highway modeling is:

[0041]

[0042] In the formula, x and y are the coordinates of the point on the horizontal plane of the geographic model, and Z 1 is the elevation value corresponding to the point in the horizontal plane, a, b, c, d, f, g are constant coefficients used to control the undulation of the benchmark terrain in the highway model;

[0043] The calculation formula for generating obstacle model in highway modeling is as follows:

[0044]

[0045] In the formula, Z 2 (x,y) is the obstacle terrain function, x i ,yi represents the center coordinate of the ith obstacle, h i is the terrain parameter used to control the obstacle height, x si ,y si are the attenuation and control slope of the ith obstacle along the x-axis and y-axis respectively, and H represents the number of obstacles.

[0046] The present invention has the following beneficial effects:

[0047] The present invention constructs a terrain flood model based on video images collected by drones, marks the rainfall threshold of each area in the terrain flood model, and displays the actual situation of the obstacle model on the terrain flood model when a geological disaster occurs, thereby improving the timeliness and accuracy of flood warning, timely displaying the occurrence of geological disasters, and visually displaying obstacles on the road, facilitating and quickly carrying out post-disaster rescue.

[0048] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0050] Figure 1 The present invention is a flow chart of a method for emergency monitoring of highways after floods using integrated sky-ground remote sensing. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0053] In order to make the purpose, technical solutions and advantages of this application more clear, the following Figure 1 It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] See also Figure 1 As shown, the present invention is a sky-ground integrated remote sensing emergency monitoring method for highway safety after floods, comprising the following steps:

[0055] Step S1: The drone collects video image data of the road surface after the flood;

[0056] Step S2: pre-processing the highway pavement video image data to construct a three-dimensional highway model;

[0057] Step S3: calibrate the three-dimensional highway model and construct a terrain flood model;

[0058] Step S4: Count the real-time rainfall and calculate the total rainfall, and map it into the constructed terrain flood model;

[0059] Step S5: marking rainfall warning sites in the terrain flood model, and marking corresponding flood warning requirements for each rainfall warning site;

[0060] Step S6: Obtain the highway modeling in the terrain flood model. When a natural disaster occurs and an obstacle blocks the highway, an obstacle model is generated on the highway modeling according to the real-time data collected by the drone.

[0061] Step S7: When the collected real-time rainfall exceeds the preset threshold, early warning processing is performed on the terrain flood model. When a geological disaster occurs, a visualization of the obstacle model is also performed on the terrain flood model.

[0062] In step S1, the highway pavement video image data is collected by multiple drones for high-altitude image acquisition; when the drones are shooting video images, the flight altitude of the drones needs to be set according to the drone working parameters, and the specific setting formula is as follows:

[0063] H = f × GSD / a;

[0064] Where W H It represents the relative altitude in meters, GSD represents the ground resolution of the image, f represents the focal length of the camera, and a represents the pixel size.

[0065] The preprocessing of highway pavement video image data includes:

[0066] (1) Image denoising, including median filtering, Gaussian filtering and wavelet denoising;

[0067] Median filtering: Using the median of the grayscale values ​​of the pixel's neighborhood to replace the grayscale value of the pixel can effectively remove discrete noise such as salt and pepper noise and protect the edge information of the image. For example, it is effective in removing isolated noise points in highway images caused by sensor failures and other reasons.

[0068] Gaussian filtering: Performs weighted average filtering on the image based on the Gaussian function, which has a good suppressive effect on noise that follows Gaussian distribution and can make the image smoother. For example, it is more suitable for processing Gaussian noise caused by light interference.

[0069] Wavelet denoising: decompose the image into different wavelet scales, and remove noise by thresholding the wavelet coefficients. This can remove noise while retaining image detail information, such as the texture of the road surface.

[0070] (2) Grayscale processing, including histogram equalization and adaptive histogram equalization

[0071] Histogram equalization: By adjusting the histogram of the image, the grayscale value distribution of the image is stretched to the entire grayscale range, so that the contrast of the image is enhanced and the details of the road surface, such as cracks and potholes, are displayed more clearly.

[0072] Adaptive histogram equalization: Divide the image into several small blocks and perform histogram equalization on each small block separately, which can better adapt to the contrast requirements of different areas in the image and avoid the problem of over-enhancement or under-enhancement during overall histogram equalization.

[0073] (3) Geometric correction, including tilt correction, perspective correction and image registration

[0074] Tilt correction: Detect the tilt angle of the road in the image, and then rotate the image to make the road horizontal or vertical in the image to facilitate subsequent analysis and processing. For example, the tilt angle can be calculated by detecting the straight line features of the road edge.

[0075] Perspective correction: Due to shooting angle and other reasons, the road in the image may appear perspective distorted. The image can be corrected to a frontal effect through perspective transformation, making the shape and size of the road closer to the actual situation. The perspective transformation matrix can be calculated using known points or features in the image.

[0076] Image registration: If there are multiple road surface images, they need to be registered spatially so that their corresponding points can be accurately aligned for subsequent comparison and analysis operations. Registration methods based on feature points or regions can be used.

[0077] (4) Image normalization

[0078] Linear normalization: Map the grayscale values ​​of an image to a specific range, such as [0,255] or [0,1].

[0079] In step S2, the construction process of the three-dimensional highway model is as follows:

[0080] Step S21: converting the original video image data into point cloud data, and performing denoising, filtering and registration operations on the point cloud data;

[0081] Step S22: using a Poisson surface reconstruction algorithm to fit the point cloud data into a smooth surface, and constructing a three-dimensional surface model of the highway;

[0082] Step S23: combining the semantic segmentation results in the image, assigning different semantic labels to different areas of the highway, such as lanes, sidewalks, green belts, etc., and annotating and displaying them in the three-dimensional model to make the model more semantically informative and readable;

[0083] Step S24: Use traditional surveying equipment such as total stations and levels to conduct field measurements of some key locations and features of the highway, compare the measured data with the corresponding data in the three-dimensional model, calculate the error, and evaluate the accuracy of the model.

[0084] In step S22, the Poisson surface reconstruction algorithm calculates the normal vector for each point cloud point by analyzing the covariance matrix of its neighborhood points. The specific formula is as follows:

[0085]

[0086] In the formula, is the neighborhood point set N i The center of mass, For the covariance matrix, p i is a point in the point cloud;

[0087] In the discrete case, for a point (i, j, k) in a three-dimensional grid, assuming that its neighboring points are (i±1, j, k), (i, j±1, k), (i, j, k±1), the normal vector field N = (N x ,N y ,N z ) is calculated as follows:

[0088]

[0089] Where h is the grid spacing;

[0090] In the marching cubes algorithm, for a cube unit, the scalar values ​​of its eight vertices are f i (i=1,2,3,...,8), assuming the threshold is, calculate the points on the isosurface by linear interpolation; select the appropriate isosurface threshold, and extract the isosurface from the solved scalar field function. The commonly used algorithm is the marching cube algorithm, which generates corresponding triangular patches for each cube in the three-dimensional grid according to the relationship between the scalar value of its vertex and the threshold. These triangular patches constitute the three-dimensional surface grid model of the highway.

[0091] In step S3, the terrain flood model calculation formula is:

[0092]

[0093] Where P is the rainfall, q is the intermediate parameter, which represents the precipitation coefficient k and the water level rise speed V. g , h is the road surface height, P 0 is the rainfall on the road surface, M is the water vapor flux of water evaporation, h * It indicates the height when the road surface water reaches saturation, mb is a parameter, generally taken as 0.0255, V indicates the road surface water flow velocity, θ is the angle between the water flow direction and the slope direction, and α is the slope.

[0094] In step S4, the calculation formula for the total rainfall is as follows:

[0095]

[0096] In the formula, the rainfall area is divided into j grids, L nj Indicates the radius of rainfall calculation within the grid, P cj represents the rainfall at the rainfall center, P nj Represents the amount of rainfall at the edge of the grid.

[0097] In step S5, the study area is divided into 20, 18, 15, 13, 10 and 8 grids respectively, and the area less than half of the grid area is merged into the adjacent grid. The mean rainfall of the rain warning station in each grid is calculated, and the rain warning station closest to the mean rainfall in the grid is selected to further deduce the rainfall of other rain warning stations in the grid. The total rainfall calculation formula in step S4 is used to calculate the surface rainfall. When the accuracy guarantee rate of the surface rainfall calculation result is greater than 90%, the number of grid divisions is determined, and finally the number of rain warning stations that should be set up in the area is determined.

[0098] In step S6, the specific formula for highway modeling is:

[0099]

[0100] In the formula, x and y are the coordinates of the point on the horizontal plane of the geographic model, and Z 1 is the elevation value corresponding to the point in the horizontal plane, a, b, c, d, f, g are constant coefficients used to control the undulation of the benchmark terrain in the highway model;

[0101] When secondary disasters such as landslides and mud-rock flows occur, obstacles appear on the road. Video images are collected by drones to generate obstacle models on the road model in a timely manner. The specific calculation formula is as follows:

[0102]

[0103] In the formula, Z 2 (x,y) is the obstacle terrain function, x i ,y i represents the center coordinate of the ith obstacle, h i is the terrain parameter used to control the obstacle height, x si ,y si are the attenuation and control slope of the ith obstacle along the x-axis and y-axis respectively, and H represents the number of obstacles.

[0104] It is worth noting that in the above system embodiment, the various units included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0105] In addition, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by instructing related hardware through a program, and the corresponding program can be stored in a computer-readable storage medium.

[0106] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for emergency monitoring of highway safety after floods based on integrated air-ground remote sensing, characterized in that: The steps include: Step S1: The drone collects video image data of the road surface after the flood; Step S2: pre-processing the highway pavement video image data to construct a three-dimensional highway model; Step S3: calibrate the three-dimensional highway model and construct a terrain flood model; Step S4: Count the real-time rainfall and calculate the total rainfall, and map it into the constructed terrain flood model; Step S5: marking rainfall warning sites in the terrain flood model, and marking corresponding flood warning requirements for each rainfall warning site; Step S6: Obtain the highway modeling in the terrain flood model. When a natural disaster occurs and an obstacle blocks the highway, an obstacle model is generated on the highway modeling according to the real-time data collected by the drone. Step S7: When the collected real-time rainfall exceeds the preset threshold, early warning processing is performed on the terrain flood model. When a geological disaster occurs, a visualization of the obstacle model is also performed on the terrain flood model.

2. The method of emergency monitoring of highway safety after floods based on sky-ground integrated remote sensing according to claim 1 is characterized in that: In step S1, the highway pavement video image data is collected by multiple drones for high-altitude image acquisition; when the drones are shooting video images, the flight altitude of the drones needs to be set according to the drone working parameters, and the specific setting formula is as follows: H = f × GSD / a; Where W H It represents the relative altitude in meters, GSD represents the ground resolution of the image, f represents the focal length of the camera, and a represents the pixel size.

3. The method of emergency monitoring of highway safety after floods based on sky-ground integrated remote sensing according to claim 1 is characterized in that: In step S2, the construction process of the three-dimensional highway model is as follows: Step S21: converting the original video image data into point cloud data, and performing denoising, filtering and registration operations on the point cloud data; Step S22: using a Poisson surface reconstruction algorithm to fit the point cloud data into a smooth surface, and constructing a three-dimensional surface model of the highway; Step S23: combining the semantic segmentation results in the image, assigning different semantic labels to different areas of the road, and marking and displaying them in the three-dimensional model; Step S24: Conduct field measurements of the location and features of the highway, compare the measured data with the corresponding data in the three-dimensional model, calculate the error, and evaluate the model accuracy.

4. The method of emergency monitoring of highway safety after floods based on sky-ground integrated remote sensing according to claim 3 is characterized in that: In step S22, the Poisson surface reconstruction algorithm calculates the normal vector for each point cloud point by analyzing the covariance matrix of its neighborhood points. The specific formula is as follows: In the formula, is the neighborhood point set N i The center of mass, For the covariance matrix, p i is a point in the point cloud; In the discrete case, for a point (i, j, k) in a three-dimensional grid, assuming that its neighboring points are (i±1, j, k), (i, j±1, k), (i, j, k±1), the normal vector field N = (N x ,N y ,N z ) is calculated as follows: Where h is the grid spacing; In the marching cubes algorithm, for a cube unit, the scalar values ​​of its eight vertices are f i (i=1,2,3,...,8), assuming the threshold is, the points on the isosurface are calculated by linear interpolation.

5. The method of emergency monitoring of highway safety after floods based on sky-ground integrated remote sensing according to claim 1 is characterized in that: In step S3, the terrain flood model calculation formula is: Where P is the rainfall, q is the intermediate parameter, which represents the precipitation coefficient k and the water level rise speed V. g , h is the road height, P0 is the rainfall on the road, M is the water vapor flux of water evaporation, h * It indicates the height when the road surface water reaches saturation, mb is a parameter, generally taken as 0.0255, V indicates the road surface water flow velocity, θ is the angle between the water flow direction and the slope direction, and α is the slope.

6. The method of emergency monitoring of highway safety after floods based on integrated sky-ground remote sensing according to claim 1, characterized in that: In step S4, the calculation formula of the total rainfall is as follows: In the formula, the rainfall area is divided into j grids, L nj Indicates the radius of rainfall calculation within the grid, P cj represents the rainfall at the rainfall center, P nj Represents the amount of rainfall at the edge of the grid.

7. The method of emergency monitoring of highway safety after floods based on integrated sky-ground remote sensing according to claim 1 is characterized in that: In step S5, the study area is divided into 20, 18, 15, 13, 10 and 8 grids respectively, and the area less than half of the grid area is merged into the adjacent grid. The mean rainfall of the rain warning station in each grid is calculated, and the rain warning station closest to the mean rainfall in the grid is selected to further deduce the rainfall of other rain warning stations in the grid. The total rainfall calculation formula in step S4 is used to calculate the surface rainfall. When the accuracy guarantee rate of the surface rainfall calculation result is greater than 90%, the number of grid divisions is determined, and finally the number of rain warning stations that should be set up in the area is determined.

8. The method of emergency monitoring of highway safety after floods based on sky-ground integrated remote sensing according to claim 1 is characterized in that: In step S6, the specific formula for highway modeling is: In the formula, x, y are the coordinates of the point in the geographic model on the horizontal plane, Z1 is the elevation value corresponding to the point in the horizontal plane, a, b, c, d, f, g are constant coefficients used to control the ups and downs of the benchmark terrain in the highway model; The calculation formula for generating obstacle model in highway modeling is as follows: Where Z2(x,y) is the obstacle terrain function, x i ,y i represents the center coordinate of the ith obstacle, h i is the terrain parameter used to control the obstacle height, x si ,y si are the attenuation and control slope of the ith obstacle along the x-axis and y-axis respectively, and H represents the number of obstacles.

Images