A river obstacle monitoring and identifying method based on millimeter wave radar

Abstract

The application relates to the field of low-altitude inspection, and discloses a river obstacle monitoring and identifying method based on a millimeter wave radar, which is used for improving the real-time performance and robustness of river obstacle monitoring. The method comprises the following steps: collecting millimeter wave radar original intermediate frequency data and high-overlap-rate aerial image streams synchronously to generate a multi-source data set; filtering water surface clutter by using a distance-Doppler spectrum, extracting a strong scattering target, and calculating a spatial coordinate to generate an effective target point cloud; selecting an elevation peak point as a rigid control point through a geographic grid, and combining with aerial triangulation calculation to generate a river correction digital surface model. The application adopts a hard-wired synchronous trigger mechanism to ensure the space-time alignment of the multi-source data, optimizes the stability of the elevation control point through neighborhood density checking and anisotropic weighted adjustment, and outputs a monitoring result including the position, height and projected area of the obstacle, so that the real-time performance, accuracy and robustness of the river obstacle monitoring are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of low-altitude inspection, and more particularly to a method for monitoring and identifying obstacles in river channels based on millimeter-wave radar. Background Technology

[0002] Rivers, as an important component of natural water systems, serve multiple functions, including flood control, irrigation, navigation, and ecological protection. However, the presence of obstacles within river channels (fallen trees, debris, illegal structures, etc.) can significantly affect water flow characteristics, leading to a decrease in the river's flow capacity and consequently causing abnormal rises in local water levels, increasing the risk of flooding. Especially during the flood season or under extreme weather conditions, obstacles can become a key trigger for sudden floods, seriously threatening the lives and property of residents along the river and the stability of infrastructure. Therefore, real-time and accurate monitoring and identification of river obstacles is of great significance for improving flood control and disaster reduction capabilities and ensuring the safety of river flood passage.

[0003] In recent years, multi-source data fusion technology has provided new ideas for river monitoring, but existing solutions still have the following shortcomings: Strong clutter generated by water surface reflection can easily mask the signals of real obstacles, making it difficult for existing algorithms to effectively separate the target from the background. Image-based 3D reconstruction relies on dense feature points, but in areas such as riverbanks or tops of obstacles, sparse feature points lead to large elevation estimation errors. Traditional digital surface model (DSM) generation relies on camera pose estimation, but accumulated errors can easily lead to overall model shift, affecting the accuracy of water level fitting; Existing methods rely solely on a single elevation or image feature to identify obstacles, making them susceptible to terrain undulations or image noise, resulting in high false positive / false negative rates.

[0004] Therefore, we propose a method for monitoring and identifying obstacles in river channels based on millimeter-wave radar to solve the above problems. Summary of the Invention

[0005] This invention provides a method for monitoring and identifying obstacles in river channels based on millimeter-wave radar, which improves the real-time performance and robustness of obstacle monitoring in river channels.

[0006] The first aspect of this invention provides a method for monitoring and identifying river obstacles based on millimeter-wave radar, applied to a low-altitude inspection platform equipped with a millimeter-wave radar sensor and an optical camera. The method includes: controlling the low-altitude inspection platform to perform inspection tasks along the river channel, simultaneously acquiring raw intermediate-frequency (IF) data from the millimeter-wave radar and high-overlap aerial image streams of the river area, and outputting a multi-source dataset; calling the raw IF data from the millimeter-wave radar in the multi-source dataset, extracting the range-Doppler spectrum, filtering out background clutter noise from the water surface, and extracting strong scattering signals with a signal-to-noise ratio higher than a preset threshold, and... The spatial coordinates are calculated, and a strong scattering effective target point cloud is output. The strong scattering effective target point cloud is processed, and the point with the largest elevation value in each geographic grid of a preset size is selected as the rigid control point of the area. A sparse set of elevation control points distributed on the riverbank and the top of obstacles is output. The high overlap aerial image stream in the multi-source dataset is read and aerial triangulation is performed to output a river channel corrected digital surface model. Based on the river channel corrected digital surface model, the lowest elevation line of the river channel section is extracted to fit the instantaneous water level and the monitoring results are output.

[0007] Optionally, in a first implementation of the first aspect of the present invention, the method includes: reading the planar coordinate data of the strong scattering effective point cloud, projecting it onto a horizontal geographic plane of a preset resolution to generate a horizontal projection grid dataset; traversing each rectangular cell in the horizontal projection grid dataset, retrieving all point cloud data falling within that cell, and generating a local elevation peak candidate sequence; constructing a spherical spatial search neighborhood with each candidate point in the local elevation peak candidate sequence as the sphere center and a preset spatial continuity verification radius as the distance constraint; counting the number of point clouds falling within the spherical spatial search neighborhood, calculating the neighborhood point cloud density value, and comparing the neighborhood point cloud density value with a preset entity support threshold to generate verified rigid feature points; collecting all verified rigid feature points, retaining their original three-dimensional geodetic coordinate information, and assigning them a unique index identifier to generate a sparse elevation control point set.

[0008] Optionally, in the second implementation of the first aspect of the present invention, the method includes: extracting features from each frame of high-overlap aerial imagery in the multi-source dataset, identifying corresponding feature points between images and establishing a correspondence, and constructing a visual feature association chain; reading the three-dimensional coordinates of the sparse elevation control point set, projecting them onto the image plane covered by the visual feature association chain, and generating vertical constraint anchor point pairs; establishing an error equation including camera pose parameters and three-dimensional point coordinate parameters, and generating a weighted joint adjustment equation; solving the weighted joint adjustment equation until the projection error converges to a preset range, updating the external orientation elements of the original image using the calculated correction parameters, and outputting the corrected camera pose parameters; using the corrected camera pose parameters to perform multi-view stereo matching and dense matching processing on the original aerial imagery, calculating the spatial three-dimensional coordinates corresponding to each pixel in the image, generating point cloud data, and encapsulating and outputting an absolute digital surface model of the river channel.

[0009] Optionally, in a third implementation of the first aspect of the present invention, in order to assign a weight to each control point in the adjustment, the weight value of the i-th anchor point in the vertical direction is set to W. z,i : ; in, S represents the standard deviation of radar ranging. i R represents the local slope value. i This represents the local roughness value. and The normalization coefficient is... The threshold value is used to describe the difference between different points in the ground.

[0010] Optionally, in the fourth implementation of the first aspect of the present invention, the forced elevation stretching and correction of the three-dimensional point cloud reconstructed from the image includes the following steps: taking each control point in the sparse elevation control point set as the center, performing local dense matching using a high overlap rate aerial image stream to reconstruct the three-dimensional geometric surface within a small area around the control point, generating a local visual verification patch; performing geometric analysis on the local visual verification patch to generate a terrain complexity index characterizing the terrain smoothness of the control point's landing area; setting a terrain error threshold, and determining control points whose terrain complexity index is higher than the threshold as elevation sensitive. Unstable points are identified and removed, while control points falling in flat or gently sloping areas are retained to generate a stable elevation anchor point set. An anisotropic weight matrix is ​​constructed for each anchor point in the stable elevation anchor point set, assigning high weights to the vertical elevation constraint components and low weights to the horizontal planar constraint components, generating an anisotropic weighted equation. The anisotropic weighted equation is substituted into the bundle adjustment model for iterative solution. The high-weighted vertical constraints are used to lock the absolute elevation datum of the model, while the low-weighted horizontal constraints allow the model to be fine-tuned in the horizontal direction to adapt to the image texture, outputting an absolute digital surface model of the river channel.

[0011] Optionally, in the fifth implementation of the first aspect of the present invention, the method includes: generating a centerline along the river channel flow direction, and slicing the absolute digital surface model of the river channel at preset intervals along the normal direction of the centerline to generate a set of orthogonal cross-section height parameters; traversing the set of orthogonal cross-section height parameters, extracting the elevation minimum points in each cross-section as local water level sampling points, and generating a continuous hydraulic slope surface; performing a raster difference operation on the absolute digital surface model of the river channel and the continuous hydraulic slope surface to generate a normalized relative elevation model; applying a preset flood obstruction height threshold to the normalized relative elevation model for binarization processing to generate a vector mask of suspected obstacle connected components; projecting and superimposing the vector mask of suspected obstacle connected components onto the aerial image stream in the multi-source dataset, counting the number of pixels within the mask area to calculate the projected area, and combining and outputting obstacle monitoring result data.

[0012] Optionally, in the sixth implementation of the first aspect of the present invention, the method further includes: performing time-domain waveform analysis on the raw millimeter-wave radar echo data in the multi-source dataset to generate a secondary echo delay sequence; based on the secondary echo delay sequence, combined with the beam pointing angle of the millimeter-wave radar and the instantaneous pose of the inspection platform, calculating the spatial three-dimensional coordinates of the reflection point of the secondary wave peak to generate a forest penetration point cloud; projecting the spatial coordinates of the forest penetration point cloud onto the absolute digital surface model of the river channel to generate a vegetation thickness difference model characterizing the distribution of surface vegetation cover thickness; using the vegetation thickness difference model to generate a vegetation shading mask, and within the mask-covered area, removing the lifting effect of the vegetation canopy on the terrain to generate a bare surface digital elevation model; performing terrain abrupt change detection on the bare surface digital elevation model to identify areas that are covered by vegetation texture in the optical image but show abnormal protrusion features on the bare surface model, and generating a forest cover obstacle notification.

[0013] Optionally, in the seventh implementation of the first aspect of the present invention, the output of the concealed obstacle notification includes the following steps: calling the signal amplitude information in the secondary echo delay sequence and mapping it as the intensity dimension to each three-dimensional spatial coordinate point of the forest penetration point cloud to generate a four-dimensional attribute penetration point cloud; performing local spatial neighborhood traversal on the four-dimensional attribute penetration point cloud, calculating the mean and standard deviation of the intensity values ​​within the neighborhood, and generating material dispersion feature parameters; performing plane fitting detection on the spatial coordinates of the forest penetration point cloud, calculating the vertical distance residual from the point cloud to the fitting plane, and generating a geometric regularity score; weightedly fusing the material dispersion feature parameters and the geometric regularity score, and determining that the concealed object is a man-made hard facility when the dispersion is lower than a preset uniformity threshold and the regularity is higher than a preset plane threshold, otherwise determining it as a natural accumulation, and marking the attribute label in the notification, and outputting an enhanced concealed obstacle notification.

[0014] The mechanism of this invention is as follows: It overcomes the shortcomings of single optical photogrammetry in weak textured water surface environment elevation drift (bowl effect) and single millimeter wave radar angular resolution, and realizes accurate relative elevation measurement of obstacles above the dynamic water surface of the river without external control points. Beneficial effects: The proposed equidistant trigger waypoint generation and hard-wired synchronous triggering technology achieves millisecond-level synchronous acquisition of radar and camera through level pulses and feedback pulses, avoiding the delay error of traditional software synchronization, significantly improving data fusion accuracy, reducing target offset or mismatch caused by timing asynchrony, and laying the foundation for subsequent high-precision 3D reconstruction. Reliable control points are generated in sparsely characterized areas such as riverbanks and tops of obstacles. Unstable points (steep slopes and vegetated areas) are eliminated by using the terrain complexity index and terrain difference threshold, while stable anchor points in flat or gentle slope areas are retained, reducing reliance on artificial ground control points and lowering deployment costs. An anisotropic weighted bundle adjustment model is designed, which assigns high weights to the vertical direction and low weights to the horizontal direction to stable elevation anchor points. This locks the absolute elevation benchmark while allowing for horizontal fine-tuning. Combined with visual verification patches generated by local dense matching, the stability of the control points is further verified. By combining river cross-section water level fitting, obstacles exceeding the safe height are automatically identified, and the scale of obstacles is quantified by calculating the projected area, providing data support for flood control decisions. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of an embodiment of the river obstacle monitoring and identification method based on millimeter-wave radar in this invention. Figure 2 This is a schematic diagram of another embodiment of the river obstacle monitoring and identification method based on millimeter-wave radar in this invention. Figure 3 An optical image of a location B at a given time A; Figure 4 A DSM generated at a certain time (A) and location (B). Detailed Implementation

[0016] This invention provides a method for monitoring and identifying river obstacles based on millimeter-wave radar, which improves the real-time performance and robustness of river obstacle monitoring. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the river obstacle monitoring and identification method based on millimeter-wave radar in this invention includes: 101. Acquire spatiotemporal synchronous observation data: Control the low-altitude inspection platform to perform inspection tasks along the river channel, and synchronously acquire the raw intermediate frequency data of millimeter-wave radar and the high overlap aerial image stream of the river channel area through hardware-triggered synchronization mechanism; output multi-source datasets including timestamps and location information; It is understood that the executing entity of this invention can be a river obstacle monitoring and identification device based on millimeter-wave radar, or it can be a terminal or a server; the specific implementation is not limited here. This embodiment of the invention will be described using a server as an example.

[0018] It should be noted that in the Yangtze River downstream Nanjing section obstacle monitoring project, a straight section of the river approximately 2 kilometers long was selected, with the starting point at 32.05 degrees north latitude and 118.75 degrees east longitude and the ending point at 32.04 degrees north latitude and 118.78 degrees east longitude. The main execution unit was a server, which remotely controlled a DJI Matrice 300 RTK drone as a low-altitude inspection platform. The drone was equipped with a TI AWR1843 millimeter-wave radar and a Zenmuse P1 aerial camera, and the inspection route along the centerline of the river was preset via a ground station before flight.

[0019] Data collection began at 9:00 AM on October 15, 2023, under clear and windless conditions. The drone flew automatically at a speed of 5 m / s and a relative altitude of 50 meters. The synchronization mechanism was implemented by a hardware trigger in the drone's flight control system: whenever the radar transmitted a frequency-modulated continuous wave with a period of 100 milliseconds and received the echo to generate raw intermediate frequency data, the trigger simultaneously sent a pulse signal to the camera, ensuring that the aerial images were exposed at exactly the same time. The radar intermediate frequency data was recorded in real time in 16-bit complex form, while the camera acquired images at a frequency of 2 frames per second, with an image overlap rate set to 80% to meet subsequent processing requirements.

[0020] Throughout the 20-minute inspection mission, the server acquired and recorded precise spatiotemporal information for each data point via the drone's RTK module. At 9:05:30, the drone's position was 32.047 degrees North latitude, 118.765 degrees East longitude, and 55.2 meters above sea level. At this time, the radar acquired a 0.1-second intermediate frequency signal, and the camera captured an aerial photograph with a resolution of 8192×5460. All data was transmitted back to the server in real time via a 4G link.

[0021] The output multi-source dataset is a structured directory comprising two core parts: first, the raw millimeter-wave radar data files, each file including acquisition start and end timestamps (accurate to microseconds) and corresponding WGS84 coordinates; and second, the aerial image stream files, each image also embedding the same timestamps and location information. The dataset totals 1200 radar data segments and 2400 aerial images, covering the entire target river area.

[0022] 102. Solving strong scattering feature point cloud: Call the original intermediate frequency data of millimeter-wave radar in the multi-source dataset, perform fast Fourier transform to extract the range-Doppler spectrum, use the constant false alarm rate detection algorithm to filter out water surface background clutter noise, extract strong scattering signals with a signal-to-noise ratio higher than the preset threshold, and solve their spatial coordinates; output the strong scattering effective target point cloud composed of non-water surface hard objects; It should be noted that in the Nanjing section of the river obstacle monitoring project, after the server completes step 101, it calls the raw intermediate frequency data of the millimeter-wave radar from the multi-source dataset to specifically implement step 102, "decomposing the strong scattering feature point cloud". The server reads a radar data file with a data segment timestamped at 9:05:30. This segment is 0.1 seconds long and includes 256 complex sampling points, corresponding to the UAV's location at 32.047 degrees north latitude, 118.765 degrees east longitude, and an elevation of 55.2 meters. Next, a Fast Fourier Transform (FFT) is performed on this intermediate frequency data: a 128-point FFT is used in the range dimension to convert the signal to the frequency domain, and then a 64-point FFT is used to analyze the velocity information in the Doppler dimension, generating a 128×64-cell range-Doppler map. Each cell represents a range-Doppler cell with a range resolution of 0.5 meters and a Doppler resolution of 0.1 meters per second.

[0023] A constant false alarm rate (CFAR) detection algorithm was applied to filter out background clutter noise from the water surface. The algorithm set the false alarm probability to 1 / 10,000, and slides a detection window across the range-Doppler spectrum, comparing the power value of each cell with that of the surrounding background cells. Water surface echoes are typically weak, with an average power of -50 dBm, while obstacle echoes are stronger. Through calculation, the algorithm eliminated clutter points with power below a dynamic threshold; points below -40 dBm were considered noise. Subsequently, strong scattered signals with a signal-to-noise ratio (SNR) 10 dB higher than a preset threshold were extracted. A signal with an SNR of 15 dB was detected, corresponding to a range cell of 30 (representing a distance of 15 meters from the radar) and a Doppler cell of 5 (representing a relative velocity of 0.5 m / s).

[0024] When calculating spatial coordinates, the server combines radar parameters and the UAV's position and attitude information. Based on radar polar coordinate measurements, the distance and Doppler values ​​are converted into coordinates in a local coordinate system, and then converted to the WGS84 geographic coordinate system using GPS and IMU data from the UAV. The spatial coordinates obtained after processing a strong scattering signal are: 32.0465°N, 118.7658°E, and 56.3 meters above sea level. This process is repeated for all radar data segments, outputting a point cloud of strong scattering effective targets consisting of non-water-surface rigid objects, including approximately 500 points. Each point includes latitude, longitude, elevation, and signal-to-noise ratio (SNR) attributes. These points correspond to rigid targets such as bridge pillars in the river channel and the tops of buildings on the riverbank. One point cloud point represents the top of a bridge pillar, with coordinates of 32.048°N, 118.767°E, elevation of 60.1 meters, and a SNR of 18 dB.

[0025] 103. Construct an elevation-constrained control network: Perform rasterization spatial thinning on the strong scattering effective target point cloud, select the point with the largest elevation value in each geographic raster of a preset size as the rigid control point of the area, and remove isolated noise points that do not meet the spatial continuity verification; output a sparse set of elevation control points distributed on the riverbank and the top of obstacles. It should be noted that in the Nanjing section of the river obstacle monitoring project, after the server obtains the strong scattering effective target point cloud output in step 102, it begins to execute step 103, "Constructing an elevation constraint control network". The input point cloud includes 500 points, each with latitude, longitude, elevation, and signal-to-noise ratio attributes, covering a river area approximately 2 kilometers long and 200 meters wide. Point A has coordinates of 32.0465 degrees north latitude, 118.7658 degrees east longitude, and an elevation of 56.3 meters.

[0026] Rasterization and spatial thinning were performed. The server converted the geographic area covered by the point cloud into UTM planar coordinates for uniform division, setting the size of each geographic raster to 5 meters × 5 meters. The entire area was divided into 4000 rasters, with one raster corresponding to a rectangular area from 118.7650 to 118.7655 degrees east longitude and 32.0460 to 32.0465 degrees north latitude. Next, control points were selected within each raster: if the raster contained point cloud data, the point with the highest elevation value was selected as the rigid control point for that area. If three points were detected in a certain raster with elevations of 55.1 meters, 56.3 meters, and 54.8 meters, the point with an elevation of 56.3 meters was selected as the control point; if the raster was empty, it was skipped. After this step, 320 control points were initially selected from 500 original points. These points were mostly located at higher positions, such as the top of bridge pillars (elevation 60.1 meters) or the top of riverbank slopes (elevation 58.5 meters).

[0027] Spatial continuity checks were performed to remove isolated noise points. The server set a neighborhood radius of 10 meters and checked whether at least two other control points existed within a 10-meter radius of each control point. One control point located at 32.0401°N, 118.7510°E, and an elevation of 62.0 meters, but with no other control points within a 10-meter radius, was identified as an isolated noise point and removed. Another control point located at 32.0475°N, 118.7670°E, and an elevation of 59.8 meters, with five control points surrounding it, was retained. This process removed 170 isolated points, outputting a sparse elevation control point set of 150 points.

[0028] These control points are evenly distributed along the riverbank and on top of obstacles, with 8 points in the bridge area and an average elevation of 60.5 meters; and 20 points on the top of the left bank slope with an average elevation of 58.2 meters. This set of control points provides crucial elevation constraints for subsequent steps, ensuring the elimination of visual reconstruction errors.

[0029] 104. Generate a controlled digital surface model: Read aerial image streams with high overlap from multi-source datasets and perform aerial triangulation. During bundle adjustment, introduce a sparse set of elevation control points as weighted observations of absolute elevation. Force elevation stretching and correction are applied to the 3D point cloud reconstructed from the images to eliminate the bowl-shaped deformation of pure visual reconstruction. Output a river channel corrected digital surface model with absolute elevation accuracy. It should be noted that the high-overlapping aerial image stream from the multi-source dataset was read. This image stream includes 2,400 aerial images, each with a resolution of 8192×5460 and an overlap rate of 80%. It also includes precise timestamps and location information. The images were captured from 9:00:00 to 9:20:00, covering the entire 2-kilometer river area.

[0030] Aerial triangulation was initiated, and a feature matching algorithm was used to automatically identify corresponding points between images. Preliminary calculations of the exterior orientation elements for each image were performed, including camera position and attitude angles. Image number 1200 was located at 32.045°N, 118.760°E, with an elevation of 55.0 meters and attitude angles of 1.2 degrees pitch and 0.5 degrees roll. However, the 3D point cloud reconstructed solely from visual data exhibited significant bowl-shaped deformation, manifested as a systematic underestimation of the elevation in the central river channel area, with a maximum deviation of 2.0 meters. The visually reconstructed elevation of one point was 56.5 meters, while the actual elevation should be approximately 58.5 meters.

[0031] During bundle adjustment, the server introduces a sparse set of elevation control points as weighted observations of absolute elevation. The control point set comprises 150 points, each with known WGS84 coordinates and elevation. Control point CP001 has coordinates of 32.0475°N, 118.7670°E, and an elevation of 59.8 meters. During adjustment, the elevation observations of the control points are assigned a weight of 0.1 to strongly constrain the optimization process. The algorithm iteratively adjusts image parameters and 3D point coordinates, minimizing reprojection error while forcing the model elevation to align with the control point elevations.

[0032] Through forced elevation stretching and correction, the model's elevation was adjusted point-by-point to match the control points. In the bridge area, the elevation of the visually reconstructed point before correction was 58.3 meters, while the control point elevation was 60.1 meters. After correction, the elevation of this point was precisely stretched to 60.1 meters. The bowl-shaped deformation of the entire river channel area was eliminated, outputting a corrected digital surface model of the river channel with absolute elevation accuracy. This model is in grid format, with a horizontal resolution of 0.1 meters, covering an area of ​​2 square kilometers, and including 20 million grid points.

[0033] Table 1 below shows the correction results for three typical control points: Table 1 The final model's elevation accuracy was evaluated, with a root mean square error of 0.2 meters.

[0034] 105. Determine obstacle attributes and parameters: Based on the river channel correction digital surface model, extract the lowest elevation line of the river channel section to fit the instantaneous water level, calculate the relative height of each point on the model surface relative to the instantaneous water level, identify connected areas with relative heights exceeding the preset flood obstruction threshold as suspected obstacles, and map them back to aerial images to confirm texture attributes; output monitoring results including obstacle location, projected area, and water obstruction height.

[0035] It should be noted that, based on the river channel correction digital surface model generated in step 104, step 105, "determining obstacle attributes and parameters," is performed. This model covers a 2-kilometer river channel area, with a horizontal resolution of 0.1 meters and an elevation accuracy of 0.2 meters, and includes 20 million grid points.

[0036] The lowest elevation lines of the river channel cross-sections were extracted to fit the instantaneous water level. A cross-section was established every 10 meters along the river centerline, generating a total of 200 cross-sections. On each cross-section, 100 points were sampled to find the lowest elevation point. At the cross-section 500 meters from the starting point, the lowest elevation point was 55.2 meters. Connecting all the lowest elevation points, an instantaneous water level line was fitted using linear interpolation. This water level gradually rises from the starting elevation of 54.5 meters to the ending elevation of 54.8 meters, reflecting the slight slope of the river channel.

[0037] Calculate the relative height of each point on the model surface with respect to the instantaneous water level. For each grid point, subtract the corresponding water level elevation from its elevation. For example, a grid point with coordinates of 32.0460°N, 118.7660°E has an elevation of 60.1 meters, and the water level elevation at that location is 55.0 meters, resulting in a relative height of 5.1 meters. The relative height of the entire model ranges from 0.1 meters to 10.5 meters.

[0038] Identifying potential obstacles: A preset flood threshold of 0.5 meters was set, and all grid points with a relative height exceeding 0.5 meters were screened and grouped using 8-neighborhood connectivity analysis. A total of 5 connected regions were identified, each considered a potential obstacle. Region 1 included 5000 grid points with an average relative height of 3.2 meters; Region 2 included 3000 points with an average relative height of 2.8 meters.

[0039] Mapping back to aerial imagery confirms texture attributes. The server maps the geographic coordinates of suspected obstacle areas onto synchronously acquired aerial imagery, and confirms attributes through manual visual inspection or image classification. Area 1 appears as a concrete structure on the imagery and is identified as an abandoned bridge pier; Area 2 appears as green vegetation and is identified as accumulated trees.

[0040] The output monitoring results include obstacle location, projected area, and water-blocking height. The projected area is calculated based on the number of grid points and resolution, and the water-blocking height is the maximum relative height within the area. Table 2 below shows the results for three typical obstacles: Table 2 These results identify flood risk points and provide quantitative data for river management.

[0041] Please see Figure 2 and Figure 4 Another embodiment of the river obstacle monitoring and identification method based on millimeter-wave radar in this invention includes: 201. Acquire spatiotemporal synchronous observation data: Control the low-altitude inspection platform to perform inspection tasks along the river channel, and synchronously acquire the raw intermediate frequency data of millimeter-wave radar and the high overlap aerial image stream of the river channel area through hardware-triggered synchronization mechanism; output multi-source datasets including timestamps and location information; Specifically, the process involves: planning equidistant trigger waypoints: analyzing the geographical area of ​​the river channel, combining the field of view of the optical camera with the preset heading overlap rate, and calculating and generating a series of equidistant trigger waypoints with fixed spatial intervals along the river centerline; generating a position synchronization trigger signal: using the high-precision positioning unit on the low-altitude inspection platform to monitor the platform's current spatial position in real time, when the distance between the platform's spatial position and the equidistant trigger waypoints is detected to be less than the preset tolerance, a level pulse is sent through the general input / output interface to generate a position synchronization trigger signal; and executing cascaded hardware exposure: transmitting the position synchronization trigger signal to the external trigger port of the optical camera to control the optical camera to open the shutter for exposure; simultaneously, outputting an exposure feedback pulse through the flash output port of the optical camera. The system first transmits a pulse and then transmits the feedback pulse directly to the synchronization trigger pin of the millimeter-wave radar via a hardwired connection, serving as the radar acquisition activation command. Next, it acquires attitude-tagged data: upon receiving the radar acquisition activation command, the millimeter-wave radar immediately transmits a frequency-modulated continuous wave sequence and receives the echo. Simultaneously, the inertial measurement unit records the current pitch, roll, and yaw angle data, binding the data to the optical image and radar echo of the current frame, respectively, to generate raw data pairs with attitude tags. Finally, it encapsulates the spatiotemporal synchronized dataset: reading the raw data pairs with attitude tags, removing abnormal data pairs whose pitch or roll angles exceed a preset stability threshold based on the attitude tags, and indexing and associating the remaining valid data pairs according to the acquisition sequence number, writing them to the storage medium, and outputting a multi-source dataset.

[0042] It should be noted that this is an example of constructing a low-altitude inspection system based on a multi-rotor UAV. The system is equipped with a 77GHz millimeter-wave radar, a high-resolution industrial camera, and a high-precision RTK positioning module. The following is the specific implementation process for obstacle monitoring along a 5-kilometer-long mountain river: Technicians imported GIS vector data of the river's centerline into the ground station software. The flight altitude was set to 100 meters above the river, and the selected industrial camera had a vertical field of view of 60 degrees. To ensure the accuracy of the subsequently generated 3D model, the forward overlap rate was set to 80%. The system automatically calculated that, to meet this overlap rate, the distance between adjacent photo points should be 25 meters. Based on this, the software automatically generated 200 equidistant waypoints with latitude and longitude coordinates along the river's centerline and uploaded the waypoint list to the UAV's onboard computer.

[0043] The drone takes off and enters the inspection route. The onboard computer reads RTK positioning data at a frequency of 10Hz. When the drone reaches the first waypoint, the system calculates the spatial distance between the current coordinates and the preset waypoint in real time. Once a distance of less than 0.5 meters is detected within the preset tolerance range, the onboard computer's GPIO port immediately outputs a 3.3V high-level pulse signal with a duration of 5 milliseconds.

[0044] The aforementioned 3.3V pulse signal is hardwired to the external trigger interface of the industrial camera, forcing the camera to open its shutter and capture the first frame of optical image at a shutter speed of 1 / 2000 second. Crucially, the camera's flash output port (Hot Shoe) simultaneously outputs a feedback level signal the instant the shutter fully opens. This feedback signal is directly connected to the millimeter-wave radar's hardware synchronization pin (SYNC_IN). The radar is activated within microseconds of receiving this signal, ensuring strict physical synchronization between "light" and "wave".

[0045] Once activated, the millimeter-wave radar immediately transmits a FM continuous wave sequence consisting of 128 chirs and receives the echo data. Simultaneously, the onboard inertial measurement unit (IMU) records the UAV's attitude at that moment: pitch angle 2.5 degrees, roll angle 0.8 degrees, and yaw angle 120 degrees. The system assigns this set of attitude data, the current raw radar intermediate frequency data, and the high-resolution image captured by the camera the same "serial number 001," completing the hardware-triggered physical binding.

[0046] After the inspection, the data processing unit read all data pairs from the storage card. The system set the attitude stability threshold to 10 degrees. During the inspection of the 45th data set, it was found that due to gusts of wind, the UAV's instantaneous roll angle reached 12 degrees, exceeding the stability threshold. The system determined that this data set would cause distortion in subsequent point cloud stitching, therefore automatically discarding the radar and image data from the 45th set. The remaining 199 valid data sets, after integrity verification, were stored sequentially in folders named after the acquisition time, outputting a multi-source dataset including precise position, attitude, imagery, and radar echoes.

[0047] 202. Solving Strong Scattering Feature Point Cloud: Call the original intermediate frequency data of millimeter-wave radar in the multi-source dataset, perform Fast Fourier Transform (FFT) to extract the range-Doppler spectrum, use the Constant False Alarm Rate Detection (CFAR) algorithm to filter out water surface background clutter noise, extract strong scattering signals with a signal-to-noise ratio higher than a preset threshold, and solve their spatial coordinates; output the strong scattering effective target point cloud composed of non-water surface hard objects; Specifically, the range-Doppler response matrix is ​​constructed as follows: The raw millimeter-wave radar echo data undergoes windowing preprocessing, and Fast Fourier Transform is performed in both the fast and slow time dimensions to convert the time-domain signal into a frequency-domain signal, generating a range-Doppler response matrix that includes target range and relative velocity information. A strong reflection candidate target index is extracted: A sliding window is used to traverse the range-Doppler response matrix using a cell-average constant false alarm rate detector to estimate the local background noise power level in real time. Cells with power amplitudes exceeding a preset multiple of the local background noise power level are marked as valid detection cells, generating a strong reflection candidate target index set. The radar spherical coordinate space vector is solved: Based on the strong reflection candidate target index set, the corresponding multi-channel receiving antenna phase data is extracted, and the phase interferometry method is used to solve the radar spherical coordinate space vector. The target's horizontal azimuth and vertical elevation angles, combined with corresponding range information, generate a radar spherical coordinate space vector set with the radar antenna phase center as the origin. A geographic discrete point cloud is generated: real-time inertial navigation data from the millimeter-wave radar platform is acquired, a rotation and translation transformation matrix is ​​constructed, and the radar spherical coordinate space vector set is mapped to a unified geographic coordinate system, generating a geographic discrete point cloud reflecting the target's true spatial location. Spatial density clustering filtering is performed: Euclidean distance neighborhood search is performed on the geographic discrete point cloud, the number of neighboring points within a preset spatial radius for each point is counted, local point cloud density is calculated, isolated outliers with local point cloud density below a preset continuity threshold are removed, and high-density point clusters with continuous spatial distribution are retained, outputting a strongly scattering effective target point cloud composed of non-water-surface hard objects.

[0048] It should be noted that, following the "serial number 001" data pair collected in the previous steps, the specific implementation example is as follows: The data processing unit reads the raw intermediate frequency (IF) data from the millimeter-wave radar in serial number 001. This data consists of 128 consecutive chirps, each sampled at 256 points. The system applies a Hamming window function to this 256×128 data matrix to suppress sidelobe interference, and then performs Fast Fourier Transform (FFT) sequentially in the fast time dimension (range dimension) and the slow time dimension (velocity dimension). The processed data generates a two-dimensional range-Doppler heatmap, where the horizontal axis represents relative velocity and the vertical axis represents range. The heatmap shows a significant energy accumulation zone within a range of 30 to 35 meters from the radar.

[0049] The Constant False Alarm Rate (CA-CFAR) detector is activated. The reference cell window size is set to 8 cells, the protection cell size to 2 cells, and the false alarm rate to 1 / 100,000. The algorithm slides across the spectrum, calculating the average power of local background noise in real time. When scanning to position 50 at distance index, the system detects that the signal power amplitude at that location is 15 dB higher than the surrounding background noise, exceeding the preset detection threshold of 13 dB. The system determines that the signal is not a weak diffuse reflection from the water surface, but a strong reflection from a hard object, and therefore marks the cell as a valid candidate target.

[0050] Based on the extracted valid detection unit index, the system backtracks to read the multi-channel antenna phase data corresponding to that location. Using the principle of phase interferometry, the phase difference between the receiving antennas is calculated, and then the physical spatial angle of the target is determined. The calculation results show that the target's horizontal azimuth angle relative to the radar antenna normal is 3.5 degrees to the left, and its vertical elevation angle is 42 degrees downward. Combined with the previously calculated radial distance of 32.4 meters, a spherical coordinate vector with the radar as the origin is generated.

[0051] The inertial navigation data associated with "serial number 001" was read to determine the UAV's location at 113.5 degrees east longitude and 24.2 degrees north latitude, at an altitude of 100 meters, and with a pitch angle of 2.5 degrees. A rotation and translation matrix was constructed to transform the spherical coordinate vectors to a geocentrically fixed coordinate system. Calculations showed that the reflection point was located at an absolute elevation of 68.5 meters somewhere in the river channel, forming an isolated geographic point. This process was repeated for all strong reflection signals in a frame of data, generating an initial point cloud containing 120 scattered points.

[0052] The initial point cloud still contains a small number of noise points caused by water splashes or multipath effects. The system performs density clustering filtering, setting the search radius to 1.5 meters and the continuity threshold to 5 points. The algorithm finds a cluster of 35 points tightly distributed within a 1-meter radius, whose local density meets the threshold, and is identified as a boulder protruding from the water. Around this cluster, within 5 meters, there are 3 scattered points with only 0 or 1 neighbors within a 1.5-meter neighborhood, below the threshold, and are therefore identified as false noise and removed. The output only includes the strong scattering effective target point cloud of those 35 high-confidence points.

[0053] 203. Construct an elevation-constrained control network: Perform rasterization spatial thinning on the strong scattering effective target point cloud, select the point with the largest elevation value in each geographic raster of a preset size as the rigid control point of the area, and remove isolated noise points that do not meet the spatial continuity verification; output a sparse set of elevation control points distributed on the riverbank and the top of obstacles. Specifically, the process involves: constructing a horizontal projection grid: reading the planar coordinate data of the strong scattering effective point cloud, projecting it onto a horizontal geographic plane of a preset resolution, and dividing the plane into non-overlapping rectangular cells to generate a horizontal projection grid dataset; extracting local peak candidates: traversing each rectangular cell in the horizontal projection grid dataset, retrieving all point cloud data falling within that cell, comparing their vertical coordinate values, and selecting the point with the largest vertical coordinate value to generate a local elevation peak candidate sequence; and constructing a spatial verification neighborhood: using each candidate point in the local elevation peak candidate sequence as the center and a preset spatial continuity verification radius as the distance constraint, the original data of the strong scattering effective point cloud is used for verification. The system centrally constructs a spherical spatial search neighborhood; performs density support verification: counts the number of point clouds falling within the spherical spatial search neighborhood, calculates the neighborhood point cloud density value, and compares this neighborhood point cloud density value with a preset entity support threshold; when the neighborhood point cloud density value is lower than the entity support threshold, the candidate point is determined to be a false multipath noise point and is removed; when the neighborhood point cloud density value is higher than or equal to the entity support threshold, the candidate point is determined to be a real ground feature point with entity support, generating verified rigid feature points; outputs a sparse control set: collects all verified rigid feature points, retains their original 3D geodetic coordinate information, assigns a unique index identifier, and combines them to generate a sparse elevation control point set.

[0054] It should be noted that at this point, the data processing unit has obtained a series of discrete radar points along the entire 5-kilometer river channel. In order to provide an absolute elevation benchmark for subsequent optical image reconstruction, the system needs to sift out a very small number of "anchor points" from these messy point clouds, but whose positions are extremely precise and stable.

[0055] The projection resolution was set, and the geographical plane of the river area was divided into rectangular grid cells with a size of 5 meters × 5 meters. Taking a slope protection area on the right bank of the river as an example, the grid number corresponding to this area is "Grid-CN-45". The system retrieved all strong scattering points falling within this grid and found a total of 128 radar points. The system compared the vertical coordinates (elevation Z value) of these points one by one and selected the point with the highest elevation, "P_Cand_01", with an elevation value of 68.5 meters. This point usually represents the top of the slope protection or the highest point of the large rocks on the bank, and is an ideal candidate for elevation control.

[0056] To prevent "P_Cand_01" from being a false elevation noise point generated by the multipath effect of the water surface (such noise points often exist in isolation), the system performs a density support verification. The spatial continuity verification radius is set to 1.0 meter, and the solid support threshold is 10 points. The system constructs a spherical search area with a radius of 1 meter centered on "P_Cand_01," and counts the number of points in the origin cloud dataset that fall within this sphere. The results show that there are 35 neighboring points within the sphere. Since 35 is much larger than the preset threshold of 10, the system determines that this point has solid support and belongs to a real rigid slope protection structure, therefore marking it as an "effective rigid feature point."

[0057] Conversely, in the grid "Grid-CN-46" at the center of the river channel, the highest point "P_Cand_02" extracted by the system has an elevation of 66.2 meters (suspended above the water surface). However, when performing the same spherical search with a radius of 1 meter on it, only 2 points were found in the neighborhood. Since 2 is less than the threshold of 10, the system determined that this point was an occasional false multipath noise point and discarded it.

[0058] The above process was repeated for all grids throughout the entire river channel, retaining rigid points that were evenly distributed and passed density verification. Table 3 below shows an example of the processing and judgment data for some grids: Table 3 All points marked "reserved" were collected, their original geodetic coordinates (longitude 113.5xxx, latitude 24.2xxx, elevation 68.52) were preserved, and unique indices were assigned to them, generating a "sparse elevation control point set" containing approximately 200 high-quality control points.

[0059] 204. Generate a controlled digital surface model: Read aerial image streams with high overlap from multi-source datasets and perform aerial triangulation. During bundle adjustment, introduce a sparse set of elevation control points as weighted observations of absolute elevation to perform forced elevation stretching and correction on the 3D point cloud reconstructed from the images, eliminating the bowl-shaped deformation of pure visual reconstruction; output a channel-corrected digital surface model (DSM) with absolute elevation accuracy. Specifically, the process involves: establishing visual feature association chains: extracting features from each high-overlap aerial image frame in the multi-source dataset, identifying corresponding feature points between images and establishing a correspondence, and constructing a visual feature association chain describing the relative positional relationship between images; generating vertical constraint anchor point pairs: reading the 3D coordinates of the sparse elevation control point set, projecting them onto the image plane covered by the visual feature association chain, searching for the corresponding feature points closest to the projected position, establishing a correspondence index between radar measured elevation and visual feature points, and generating vertical constraint anchor point pairs; and constructing weighted joint adjustment equations: establishing error equations including camera pose parameters and 3D point coordinate parameters, in which the visual feature association chain is used as a relative position constraint term, and the vertical constraint anchor point pairs are used as... The system generates a weighted joint adjustment equation by assigning absolute position constraints a higher weight than relative position constraints. It then solves for corrected camera pose using an iterative least squares method until the projection error converges to a preset range. The calculated correction parameters are used to update the external orientation elements of the original image, outputting corrected camera pose parameters to eliminate elevation distortion. Finally, it generates an absolute elevation dense point cloud by performing multi-view stereo matching and dense matching on the original aerial image using the corrected camera pose parameters. This calculates the spatial 3D coordinates of each pixel in the image, generating high-density point cloud data with absolute elevation information. The point cloud data is then triangulated and encapsulated to output an absolute digital surface model of the river channel.

[0060] Furthermore, the 3D point cloud reconstructed from the image is subjected to forced elevation stretching and correction, including the following steps: Constructing a local visual verification patch: Before performing bundle adjustment, local dense matching is performed using a high overlap aerial image stream, centered on each control point in the sparse elevation control point set, to reconstruct the 3D geometric surface within a small area around the control point, generating a local visual verification patch; Calculating the terrain complexity index: Geometric analysis is performed on the local visual verification patch, calculating the angle between the patch surface normal vector and the gravity direction to obtain the slope value, and statistically analyzing the elevation variance within the patch to obtain the roughness value. The slope value and roughness value are normalized and fused to generate a terrain complexity index characterizing the terrain smoothness of the control point's landing area; Selecting stable elevation anchor points: A terrain complexity threshold is set to filter stable elevation anchor points. Control points with an index exceeding the threshold are identified as elevation-sensitive unstable points and removed. Control points falling in flat or gently sloping areas are retained to generate a stable elevation anchor point set. Anisotropic weighted equations are constructed: Based on the physical characteristic that the ranging accuracy of millimeter-wave radar is better than that of angle measurement, an anisotropic weight matrix is ​​constructed for each anchor point in the stable elevation anchor point set. The vertical elevation constraint component is given a high weight value, and the horizontal plane constraint component is given a low weight value, generating anisotropic weighted equations. Robust joint adjustment is performed: The anisotropic weighted equations are substituted into the bundle adjustment model for iterative solution. The high-weight vertical constraint is used to lock the absolute elevation datum of the model, while the low-weight horizontal constraint allows the model to be fine-tuned in the horizontal direction to adapt to the image texture, outputting an absolute digital surface model of the river channel.

[0061] It should be noted that the following example is based on 199 high-overlap images of a 5-kilometer-long river channel: Feature extraction was performed on 199 acquired images, yielding an average of 12,000 SIFT feature points per image, and a total of 850,000 tie points were established between adjacent images. To incorporate radar control points, the system read 200 sparse control points generated in step 203. For control point "Anchor-45" (elevation 68.52 meters), the system did not use it directly but instead first constructed a "local visual verification patch." The system called upon three images covering this point and performed high-density matching within a 2-meter radius of the point to reconstruct a miniature 3D patch.

[0062] Analysis of the surface patch “Anchor-45” revealed an angle (slope) of 3.5 degrees between its surface normal vector and the gravity direction, and an elevation variance (roughness) of 0.02 meters, classifying it as a flat area. However, the control point “Anchor-88” has a surface slope of 38 degrees (located on a steep rock face). The system selects anchor points based on the “terrain complexity index.” To assign weights to each control point in the adjustment, an anisotropic elevation confidence weighting formula is introduced: Among them, W z,i Let be the weight value of the i-th anchor point in the vertical direction. S represents the standard deviation of radar ranging (taken as 0.1m). i R represents the local slope value. i This represents the local roughness value. and The normalization coefficient is... The terrain roughness threshold is defined by this formula. This formula ensures that the flatter the terrain and the lower the roughness of the radar point, the closer its weight is to the radar's physical accuracy limit; conversely, the weight drops sharply.

[0063] The weights of all candidate points were calculated using the above formula, and a weighted joint adjustment equation was constructed. Before adding radar control points, the purely visually reconstructed river model exhibited a typical "bowl-shaped deformation," with the river center elevation being 5.2 meters lower than the actual elevation. The 150 selected "stable elevation anchor points" and their corresponding anisotropic weights were substituted into the equation. The vertical constraint weight was set to be much greater than the horizontal constraint weight (weight ratio of 100:1). After 15 iterations of least squares solution, the projection error converged to 0.6 pixels.

[0064] Table 4 below shows the processing data details for some control points: Table 4 Using the corrected camera pose parameters (eliminating bowl distortion), the system performed dense matching (MVS) on the entire river channel. The output point cloud density reached 400 points / square meter, and the elevation error of the river channel centerline was reduced from 5.2 meters to within 0.15 meters, successfully generating a digital surface model (DSM) of the river channel with absolute geographic accuracy.

[0065] 205. Determine obstacle attributes and parameters: Based on the river channel correction digital surface model, extract the lowest elevation line of the river channel section to fit the instantaneous water level, calculate the relative height of each point on the model surface relative to the instantaneous water level, identify connected areas with relative heights exceeding the preset flood obstruction threshold as suspected obstacles, and map them back to aerial images to confirm texture attributes; output monitoring results including obstacle location, projected area and water obstruction height.

[0066] Specifically, the process involves: extracting an orthogonal cross-sectional elevation set: generating a centerline along the river's flow direction, and slicing the absolute digital surface model of the river channel at preset intervals along the normal direction of the centerline to obtain the cross-sectional elevation data corresponding to each slice location, thus generating an orthogonal cross-sectional elevation set; constructing a continuous hydraulic gradient surface: traversing the orthogonal cross-sectional elevation set of the river channel, extracting the elevation minimum points in each cross-section as local water level sampling points, and using the least squares method to perform polynomial curve fitting along the river's flow direction on all local water level sampling points to generate a continuous hydraulic gradient surface reflecting the natural downward trend of the river's water level; and solving the normalized relative elevation model: performing a raster difference operation on the absolute digital surface model of the river channel and the continuous hydraulic gradient surface, i.e., subtracting the absolute elevation of each pixel in the digital surface model from the raster difference. The process involves: removing the water level elevation at the corresponding location on the hydraulic gradient surface to generate a normalized relative elevation model that eliminates the influence of topographic undulations and water level differences; generating an obstacle connectivity mask: applying a preset flood obstruction height threshold to the normalized relative elevation model for binarization, marking areas with relative elevations greater than the threshold as foreground, and performing morphological closing operations to fill internal voids, connecting adjacent broken patches to generate a suspected obstacle connectivity vector mask; encapsulating a multi-dimensional attribute monitoring report: projecting and overlaying the suspected obstacle connectivity vector mask onto the aerial image stream in the multi-source dataset, counting the number of pixels within the mask area to calculate the projected area, extracting the maximum relative elevation within the mask area as the water obstruction height, and cropping the corresponding image texture slices, combining and outputting the obstacle monitoring results data.

[0067] It should be noted that, following the digital surface model (DSM) of the river channel with absolute elevation accuracy generated in step 204, the following is a specific implementation example of intelligent obstacle screening for this 5-kilometer river channel: An orthogonal slice is generated every 10 meters along the river centerline, resulting in 500 cross-sections over a total length of 5 kilometers. Taking the cross-section at station K2+150 as an example, the system reads the topographic data on this cross-section and extracts the lowest elevation point as 58.12 meters. By performing least-squares polynomial fitting on the lowest points of the 500 cross-sections before and after it, the system filters out measurement noise and the influence of local potholes, constructing a smooth "continuous hydraulic gradient surface." The fitting calculation shows that the instantaneous theoretical water level elevation at K2+150 is 58.25 meters, which represents the current water surface benchmark.

[0068] Raster interpolation was performed, and the hydraulic gradient surface was subtracted using DSM. Near section K2+150, a raised area was found with a peak absolute elevation of 60.75 meters. The calculated relative height of this point was 2.50 meters (60.75 - 58.25). The system's preset "flood obstruction threshold" was 0.8 meters. Since 2.50 meters is much larger than 0.8 meters, this area was binarized and marked as foreground. Subsequently, the system performed morphological closing operations, connecting the fragmented pixels within this area into a complete connected region, and the projected area of ​​this connected region was calculated to be 12.4 square meters.

[0069] The vector mask of the aforementioned connected domain was back-projected onto the original high-resolution aerial image. Texture analysis confirmed that the area exhibited grayish-white man-made structure features. The system output a list of obstacle monitoring results, in which the object at K2+150 was identified as "abandoned bridge pier debris".

[0070] The following is Table 5, which shows some of the monitoring results output by the system for this river section: Table 5 Note: The water-blocking height of OBS-03 in the table is 0.45 meters, which is lower than the preset threshold of 0.8 meters. Therefore, although it is calculated in the final report, it is marked as "ignored" or "concerned" and is not considered as an emergency removal target; while OBS-01 and OBS-04 are marked as high-risk obstacles.

[0071] 206. Extracting Multipath Echo Temporal Features: Perform temporal waveform analysis on the raw millimeter-wave radar echo data from the multi-source dataset. Search for secondary peaks within a preset time window after detecting the main peak, extract the signal strength and flight time parameters of the secondary peaks, and generate a secondary echo time delay sequence. Solving Forest Penetration Point Clouds: Based on the secondary echo time delay sequence, combined with the beam pointing angle of the millimeter-wave radar and the instantaneous pose of the inspection platform, calculate the spatial three-dimensional coordinates of the reflection points of the secondary peaks, generating a forest penetration point cloud that reflects the true surface or location of concealed objects beneath the vegetation cover. Constructing a Vegetation Thickness Difference Model: Project the spatial coordinates of the forest penetration point cloud onto the absolute digital surface model of the river channel, and calculate the optical surface elevation and radar penetration at the projection points. Vertical differences between elevations are used to generate a vegetation thickness difference model representing the distribution of surface vegetation cover thickness. The bare surface elevation model is reconstructed: a vegetation shading mask is generated using the vegetation thickness difference model. Within the masked area, elevation data from the forest canopy penetration point cloud is used to replace the optical surface elevation data in the absolute digital surface model of the river channel, eliminating the uplift effect of the vegetation canopy on the terrain, thus generating a bare surface digital elevation model. Hidden obstacle notifications are output: terrain abrupt change detection is performed on the bare surface digital elevation model to identify areas covered by vegetation texture in the optical image but exhibiting abnormal protrusions in the bare surface model. These areas are determined to be forest cover facilities or deposits, generating forest cover obstacle notifications including the center coordinates and estimated volume of the hidden objects.

[0072] Specifically, the output of concealed obstacle notification includes the following steps: Constructing a four-dimensional attribute penetration point cloud: The signal amplitude information from the secondary echo delay sequence is retrieved and mapped as an intensity dimension to each three-dimensional spatial coordinate point of the forest understory penetration point cloud, establishing a correspondence between spatial location and reflected energy, generating a four-dimensional attribute penetration point cloud including three-dimensional coordinate information and one-dimensional intensity information; Calculating material dispersion features: A local spatial neighborhood traversal is performed on the four-dimensional attribute penetration point cloud, calculating the mean and standard deviation of the intensity values ​​within the neighborhood, generating material dispersion feature parameters characterizing the consistency of surface reflection; Extracting geometric rules... Geometric regularity score: Perform plane fitting detection on the spatial coordinates of the forest penetration point cloud, calculate the vertical distance residual from the point cloud to the fitting plane, and count the percentage of points with residuals within a preset allowable range to generate a geometric regularity score characterizing the flatness of the object surface; Determine obstacle attribute category: Weighted fusion of material dispersion feature parameters and geometric regularity score. When the dispersion is lower than a preset uniformity threshold and the regularity is higher than a preset plane threshold, the hidden object is determined to be a man-made hard facility; otherwise, it is determined to be a natural accumulation. The attribute label is marked in the report, and an enhanced hidden obstacle report including attribute classification information is output.

[0073] It should be noted that the following is an example of a densely vegetated area from K3+500 to K3+600, following the previous steps: At K3+520, aerial imagery revealed a dense reed bed, with the optical DSM model indicating a surface elevation of 63.5 meters. The system retrieved raw intermediate frequency (IF) data from the millimeter-wave radar corresponding to this location and performed time-domain waveform analysis. After detecting a strong primary wave peak representing the reed canopy, the system set a 20-nanosecond time window. Ten nanoseconds after the primary wave peak, the system detected a weaker but clear secondary wave peak (signal strength -85 dBm). Based on the vertical observation angle at the time, the system calculated that the reflection point generating the secondary wave peak was located 1.5 meters below the reed canopy, at an absolute elevation of 62.0 meters. The system determined the vegetation thickness at this location to be 1.5 meters and adopted 62.0 meters as the true elevation of the exposed surface.

[0074] The area was processed in batches, and the original raised optical surface was replaced with a penetrating point cloud to generate a "digital elevation model of the exposed surface." After model reconstruction, the system discovered that beneath the grass at K3+545, the exposed surface did not naturally decrease with the riverbank; instead, an abnormal rectangular protrusion appeared, measuring 3 meters long, 2 meters wide, and 1.2 meters higher than the surrounding ground. Although the optical imagery showed only green vegetation, the "penetrated" terrain model revealed this hidden object.

[0075] To determine whether the protrusion was a natural mound or a man-made structure, the system constructed a four-dimensional attribute point cloud. The system extracted the reflection intensity (fourth dimension) of the point cloud on the surface of the concealed object, finding an average reflection intensity as high as -50 dBm (significantly higher than soil's -90 dBm). Next, the system calculated the "material dispersion feature." Within a 1-square-meter area on the top of the object, the standard deviation of the reflection intensity was only 2.5, indicating a very uniform material. Simultaneously, the system performed a "geometric regularity score," fitting the point cloud to a plane, finding that 85% of the points had a residual distance of less than 5 centimeters to the fitted plane, indicating extremely high geometric regularity.

[0076] The aforementioned feature parameters were weighted and fused. Due to the object's low dispersion (uniform material) and high regularity (flat surface), the system classified it as a "man-made hard structure (suspected illegal prefabricated steel structure)". At the nearby K3+560, another hidden protrusion was found, but due to its large dispersion and irregular geometry, it was classified as a "natural accumulation".

[0077] The following is Table 6, a comparison and judgment table of concealed obstacle features output by the system: Table 6 An "Enhanced Covert Obstacle Notification" was generated, including the coordinates, volume, and attributes of HID-01 and HID-03, prompting inspection personnel to focus on checking the reed beds at K3+545.

[0078] The present invention also provides a river obstacle monitoring and identification device based on millimeter-wave radar. The river obstacle monitoring and identification device based on millimeter-wave radar includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs the steps of the river obstacle monitoring and identification method based on millimeter-wave radar in the above embodiments.

[0079] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the millimeter-wave radar-based river obstacle monitoring and identification method.

[0080] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0081] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring and identifying river obstacles based on millimeter-wave radar, characterized in that, Applications include low-altitude inspection platforms equipped with millimeter-wave radar sensors and optical cameras, including: Control the low-altitude inspection platform to perform inspection tasks along the river, simultaneously collect raw intermediate frequency data from millimeter-wave radar and high-overlap aerial image streams of the river area, and output multi-source datasets; The original intermediate frequency data of millimeter-wave radar in the multi-source dataset is called, the range-Doppler spectrum is extracted, the background clutter noise of the water surface is filtered out, the strong scattering signal with a signal-to-noise ratio higher than a preset threshold is extracted, and its spatial coordinates are calculated to output the strong scattering effective target point cloud. The strong scattering effective target point cloud is processed, and the point with the largest elevation value in each geographic grid of a preset size is selected as the rigid control point of the area, and a sparse set of elevation control points distributed on the riverbank and the top of obstacles is output. Read the high-overlap aerial image stream from the multi-source dataset, perform aerial triangulation calculations, and output a river channel corrected digital surface model. Based on the river channel correction digital surface model, the lowest elevation line of the river channel section is extracted to fit the instantaneous water level and output the monitoring results.

2. The method for monitoring and identifying river obstacles based on millimeter-wave radar according to claim 1, characterized in that, include: Read the planar coordinate data of the strong scattering effective point cloud, project it onto a horizontal geographic plane of a preset resolution, and generate a horizontal projection grid dataset; Traverse each rectangular cell in the horizontal projection grid dataset, retrieve all point cloud data falling within that cell, and generate a local elevation peak candidate sequence. Using each candidate point in the local elevation peak candidate sequence as the sphere center and a preset spatial continuity verification radius as the distance constraint, a sphere spatial search neighborhood is constructed. The number of point clouds falling into the search neighborhood of the sphere is counted, the neighborhood point cloud density value is calculated, and the neighborhood point cloud density value is compared with the preset entity support threshold to generate verified rigid feature points. Collect all verified rigid feature points, retain their original three-dimensional geodetic coordinate information, and assign them unique index identifiers to generate a sparse set of elevation control points.

3. The method for monitoring and identifying river obstacles based on millimeter-wave radar according to claim 2, characterized in that, include: Feature extraction is performed on each frame of high-overlap aerial imagery in the multi-source dataset. Corresponding feature points between images are identified and established to construct a visual feature association chain. Read the three-dimensional coordinates of the sparse elevation control point set, project them onto the image plane covered by the visual feature association chain, and generate vertical constraint anchor point pairs; Establish error equations that include camera pose parameters and 3D point coordinate parameters, and generate weighted joint adjustment equations; The weighted joint adjustment equation is solved until the projection error converges to a preset range. The external orientation elements of the original image are updated using the calculated correction parameters, and the corrected camera pose parameters are output. The original aerial images are processed by multi-view stereo matching and dense matching using the corrected camera pose parameters. The spatial three-dimensional coordinates of each pixel in the image are calculated to generate point cloud data and encapsulate and output the absolute digital surface model of the river channel.

4. The method for monitoring and identifying river obstacles based on millimeter-wave radar according to claim 3, characterized in that, To assign a weight to each control point in the adjustment, the weight of the i-th anchor point in the vertical direction is set to W. z,i : ； in, S represents the standard deviation of radar ranging. i R represents the local slope value. i This represents the local roughness value. and The normalization coefficient is... The threshold value is used to describe the difference between different points in the ground.

5. The method for monitoring and identifying river obstacles based on millimeter-wave radar according to claim 3, characterized in that, The forced elevation stretching and correction of the reconstructed 3D point cloud from the image includes the following steps: Centered on each control point in the sparse elevation control point set, local dense matching is performed using a high overlap rate aerial image stream to reconstruct the three-dimensional geometric surface within a small area around the control point, generating a local visual verification patch. Geometric analysis is performed on the local visual verification patch to generate a terrain complexity index that characterizes the flatness of the terrain in the control point landing area; A terrain complexity index threshold is set, and control points with a terrain complexity index higher than the threshold are identified as elevation-sensitive unstable points and removed. Control points that fall in flat or gentle slope areas are retained to generate a stable elevation anchor point set. An anisotropic weight matrix is ​​constructed for each anchor point in the set of stable elevation anchor points. The vertical elevation constraint component is given a high weight value, and the horizontal plane constraint component is given a low weight value, thereby generating an anisotropic weighted equation. The anisotropic weighted equations are substituted into the bundle adjustment model for iterative solution. The vertical constraint with high weight is used to lock the absolute elevation datum of the model, while the horizontal constraint with low weight allows the model to be fine-tuned in the horizontal direction to adapt to the image texture, and the absolute digital surface model of the river channel is output.

6. The method for monitoring and identifying river obstacles based on millimeter-wave radar according to claim 3, characterized in that, include: A centerline is generated along the river channel flow direction, and the absolute digital surface model of the river channel is sliced ​​and sampled along the normal direction of the centerline at preset intervals to generate a high-order sequence set of orthogonal cross-sections of the river channel. Traverse the set of orthogonal cross-sections of the river channel, extract the elevation minimum point in each cross-section as a local water level sampling point, and generate a continuous hydraulic gradient surface. Perform a raster difference operation on the absolute digital surface model of the river channel and the continuous hydraulic gradient surface to generate a normalized relative elevation model; The normalized relative elevation model is binarized using a preset flood obstruction height threshold to generate a vector mask of the connected domain of suspected obstacles. The suspected obstacle connected domain vector mask is projected and superimposed onto the aerial image stream in the multi-source dataset. The number of pixels within the mask area is counted to calculate the projected area, and the obstacle monitoring results data are combined and output.

7. The method for monitoring and identifying river obstacles based on millimeter-wave radar according to claim 1, characterized in that, Also includes: Perform time-domain waveform analysis on the raw millimeter-wave radar echo data from the multi-source dataset to generate a secondary echo delay sequence; Based on the secondary echo delay sequence, combined with the beam pointing angle of the millimeter-wave radar and the instantaneous pose of the inspection platform, the spatial three-dimensional coordinates of the reflection point of the secondary wave peak are calculated to generate a forest penetration point cloud. The spatial coordinates of the forest underpass penetration point cloud are projected onto the absolute digital surface model of the river channel to generate a vegetation thickness difference model that characterizes the distribution of surface vegetation cover thickness. The vegetation thickness difference model is used to generate a vegetation shading mask. Within the mask-covered area, the effect of the vegetation canopy on the terrain lifting is eliminated, and a digital elevation model of the bare surface is generated. Terrain abrupt change detection is performed on the exposed ground digital elevation model to identify areas that are covered by vegetation texture in optical images but show abnormal protrusion features on the exposed ground model, and to generate understory concealment obstacle notifications.

8. The method for monitoring and identifying river obstacles based on millimeter-wave radar according to claim 7, characterized in that, The output of the concealed obstacle notification includes the following steps: The signal amplitude information in the secondary echo delay sequence is called and mapped as the intensity dimension to each three-dimensional spatial coordinate point of the forest underpass penetration point cloud to generate a four-dimensional attribute penetration point cloud. Perform a local spatial neighborhood traversal on the four-dimensional attribute penetrating point cloud, calculate the mean and standard deviation of the intensity values ​​within the neighborhood, and generate material dispersion feature parameters; Perform plane fitting detection on the spatial coordinates of the forest underpass penetration point cloud, calculate the vertical distance residual from the point cloud to the fitting plane, and generate a geometric regularity score; The material dispersion feature parameter and the geometric regularity score are weighted and fused. When the dispersion is lower than the preset uniformity threshold and the regularity is higher than the preset plane threshold, the hidden object is determined to be a man-made hard facility. Otherwise, it is determined to be a natural accumulation. The attribute label is marked in the report and the enhanced hidden obstacle report is output.

Images