Machine vision-oriented method and system for identifying flat terrain of unmanned area parking

Abstract

The application discloses a method and system for identifying flat terrain in a no-man's land for machine vision, and relates to the technical field of terrain identification.The method comprises the following steps: obtaining digital elevation model data, ground optical images, multispectral remote sensing images, SAR images and near-real-time meteorological data of a window period of a target area; performing feature recognition and extraction to obtain visual recognition features; performing fusion processing to determine a static flat passable area; predicting potential catchment areas and soil moisture changes to generate a dynamic safety area; and performing spatial overlap alignment between the dynamic safety area and the static flat passable area to determine a passable flat area.The method solves the technical problem that the prior art cannot effectively respond to dynamic changes in the environment of a no-man's land, resulting in insufficient accuracy, comprehensiveness and safety of flat area identification.The method achieves the technical effect of improving the accuracy, comprehensiveness and safety of terrain identification in a no-man's land by combining multi-source data fusion and dynamic and static analysis.

Description

Technical Field

[0001] This invention relates to the field of terrain recognition technology, specifically to a method and system for recognizing flat terrain in uninhabited parking areas using machine vision. Background Technology

[0002] Autonomous parking and path planning for unmanned vehicles in complex environments is a crucial research direction in uninhabited area exploration. Among these, the identification and selection of parking terrain is a key aspect of ensuring safe parking and operation, especially in uninhabited areas lacking artificial facilities and clear road markings. Accurately and efficiently identifying flat, passable areas is therefore essential. Traditional flat area identification relies on a single data source, such as terrain analysis based on digital elevation models (DEMs) or visual feature extraction from optical images. While DEM data can reflect changes in terrain elevation, it cannot comprehensively capture surface cover types and their dynamic characteristics. Optical images are susceptible to lighting and weather conditions, potentially leading to unstable image quality and feature occlusion in uninhabited areas. Furthermore, uninhabited environments are highly dynamic, exhibiting variations in soil moisture after rainfall and seasonal runoff; relying solely on static data makes it difficult to accurately assess real-time surface passability.

[0003] Therefore, current technologies have limitations in effectively addressing dynamic changes in uninhabited environments, resulting in insufficient accuracy, comprehensiveness, and security in identifying flat areas. Summary of the Invention

[0004] This application provides a machine vision-based method and system for recognizing flat terrain in uninhabited parking areas. This solves the technical problem in existing technologies that cannot effectively cope with dynamic changes in the uninhabited environment, resulting in insufficient accuracy, comprehensiveness, and safety in recognizing flat areas. It achieves the technical effect of improving the accuracy, comprehensiveness, and safety of uninhabited parking terrain recognition through multi-source data fusion and dynamic-static combined analysis.

[0005] This application provides a machine vision-based method for identifying flat terrain in uninhabited parking areas. The method includes: acquiring digital elevation model data, surface optical images, multispectral remote sensing images, SAR images, and near-real-time meteorological data during a window period for the target area, and preprocessing the acquired multi-source data; extracting features from the acquired surface optical images, multispectral remote sensing images, and SAR images based on machine vision to obtain visual recognition features; fusing the visual recognition features with the digital elevation model data to identify surface undulation features, water body features, and soil bearing capacity features, and determining statically flat and passable areas; predicting potential catchment areas and soil moisture changes based on the near-real-time meteorological data during the window period combined with the analysis data of the SAR images, generating dynamic safe areas; and spatially overlapping and aligning the dynamic safe areas with the statically flat and passable areas to determine passable flat areas.

[0006] In a possible implementation, the machine vision-oriented method for recognizing flat terrain in uninhabited parking areas further performs the following processing: filling depressions, smoothing edges, and performing coordinate system unification on the digital elevation model data; performing distortion correction based on camera calibration parameters and brightness normalization based on histogram matching on the surface optical image; and performing radiometric calibration, speckle noise suppression, and geocoding on the SAR image to generate a standardized backscattering coefficient map.

[0007] In a possible implementation, the machine vision-oriented method for recognizing flat terrain in uninhabited parking areas further performs the following processing: using a convolutional neural network to segment the surface type of the optical image to obtain the surface material type, including rocks, bare soil, sand, grassland, and obstacles; based on the surface material type, extracting local terrain undulation features from the optical image through a monocular depth estimation network to generate a visual depth map; and using the surface material type and the corresponding visual depth map as visual recognition features.

[0008] In a possible implementation, the machine vision-oriented method for identifying flat terrain in uninhabited parking areas further performs the following processing: calculating the normalized water index, soil-regulated vegetation index, and red edge index on the multispectral remote sensing image to identify surface water bodies, wetlands, and vegetation-covered areas; calculating surface moisture characteristics based on the standardized backscattering coefficient map and the incident angle model to identify areas with abnormal surface moisture content; and decomposing and distinguishing surface roughness types based on the SAR scattering mechanism and the standardized backscattering coefficient map.

[0009] In a possible implementation, the machine vision-oriented method for identifying flat terrain in uninhabited parking areas further performs the following processing: using a threshold segmentation method to identify strong absorption regions in the backscattering coefficient map, and marking regions with backscattering coefficients below a preset threshold as suspected water bodies; using time-series SAR imagery for InSAR interferometry analysis to extract surface deformation information, detecting the degree of surface disturbance based on multi-temporal SAR coherence, and marking regions with coherence below a preset threshold as unstable regions; and using the identified water body features, humidity features, and surface deformation features as visual recognition features for SAR.

[0010] In a possible implementation, the machine vision-oriented method for recognizing flat terrain in uninhabited parking areas further performs the following processing: using a multi-scale feature pyramid network to scale-align the digital elevation model data with the visual depth features; wherein, the terrain skeleton structure of the digital elevation model data is preserved in the coarse-scale layer, visual depth detail features are injected in the fine-scale layer in a local compensation manner, and the weighting coefficients of the multi-source features are adaptively adjusted through a channel attention mechanism to form a fused elevation model.

[0011] In a possible implementation, the machine vision-oriented method for identifying flat terrain in uninhabited parking areas further performs the following processing: based on the fused elevation model, the slope value of each grid is calculated, and areas with slopes lower than a first dynamic threshold are marked as slope candidate areas; within a multi-scale sliding window, the elevation standard deviation is calculated as a terrain roughness index, and areas with roughness lower than a second dynamic threshold are marked as roughness candidate areas; based on multispectral images, a normalized water index is calculated, and combined with SAR backscattering coefficient threshold segmentation, open water areas are identified; based on the visual semantic segmentation results, shadowed pseudo-water bodies, seasonal wetlands, and snowmelt water areas are identified; based on the identified candidate areas, as well as the open water areas, shadowed pseudo-water bodies, seasonal wetlands, and snowmelt water areas, flat areas are screened to obtain the statically flat and passable areas.

[0012] In a possible implementation, the machine vision-oriented method for recognizing flat terrain in uninhabited parking areas further performs the following processing: state verification is performed on the shadowed pseudo-water bodies, seasonal wetlands, and snowmelt water areas. Specifically, for the shadowed pseudo-water bodies, consistency is determined by comparing the shadow distribution from semantic segmentation with the mountain shadows generated by the digital elevation model data. For the seasonal wetlands, active and dry states are identified. For dry seasonal wetlands, soil carrying capacity is additionally assessed; if the carrying capacity exceeds a safety threshold, the state verification is deemed inconsistent. For the snowmelt water areas, thermal infrared data is introduced, and the ongoing snowmelt process is detected by the difference between the surface temperature and the surrounding environment. The shadowed pseudo-water bodies, seasonal wetlands, snowmelt water areas, and open water areas with consistent state verification are removed to obtain the static, flat, and passable area.

[0013] In a possible implementation, the machine vision-oriented method for identifying flat terrain in unmanned parking areas further performs the following processing: Constructing a multimodal soil carrying capacity estimation model, wherein the inputs of the multimodal soil carrying capacity estimation model include multispectral reflectance features, SAR surface soil moisture inversion results, and surface material types obtained from visual semantic segmentation; outputting a soil carrying capacity distribution map through a gradient boosting decision tree regression model; based on the soil carrying capacity distribution map, coupling near-real-time meteorological data with the terrain parameters of the digital elevation model to simulate the surface runoff formation and soil moisture migration process in the target area within a future preset time window, predicting potential catchment areas and soft soil areas; performing differential interferometry processing on at least three periods of SAR images to identify high-risk areas for geological hazards where the surface deformation rate exceeds a safety threshold, wherein the SAR images should have a uniform and continuous time series with a time interval not exceeding twice the satellite revisit period, constructing a deformation time series; and removing the predicted potential catchment areas, soft soil areas, and high-risk areas for geological hazards to generate dynamic safe areas.

[0014] This application also provides a machine vision-based uninhabited area parking terrain flatness recognition system. The system includes: a multi-source data acquisition module for acquiring digital elevation model data, surface optical images, multispectral remote sensing images, SAR images, and near-real-time meteorological data during a window period for the target area, and preprocessing the acquired multi-source data; a feature recognition and extraction module for performing feature recognition and extraction on the acquired surface optical images, multispectral remote sensing images, and SAR images based on machine vision to obtain visual recognition features; a fusion processing module for fusing the visual recognition features with the digital elevation model data to identify surface undulation features, water body features, and soil bearing capacity features, and determine statically flat and passable areas; a dynamic safe area generation module for predicting potential catchment areas and soil moisture changes based on near-real-time meteorological data during a window period combined with the analysis data of the SAR images, and generating dynamic safe areas; and a spatial overlap and alignment module for spatially overlapping and aligning the dynamic safe areas with the statically flat and passable areas to determine passable flat areas.

[0015] This application proposes a machine vision-based method and system for identifying flat terrain in uninhabited parking areas. The method acquires digital elevation model data, surface optical images, multispectral remote sensing images, SAR images, and near-real-time meteorological data for a specific window period for the target area. It then extracts visual features, performs fusion processing to determine statically flat and passable areas, predicts potential catchment areas and soil moisture changes to generate dynamic safe areas, and spatially overlaps and aligns the dynamic safe areas with the statically flat and passable areas to determine the passable flat areas. This addresses the technical problem in existing technologies that cannot effectively handle dynamic changes in the uninhabited environment, leading to insufficient accuracy, comprehensiveness, and safety in flat area identification. It achieves the technical effect of improving the accuracy, comprehensiveness, and safety of uninhabited parking terrain identification through multi-source data fusion and dynamic / static combined analysis. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments of this disclosure will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0017] Figure 1 This is a schematic flowchart of a machine vision-based method for recognizing flat terrain in uninhabited parking areas, provided in an embodiment of this application.

[0018] Figure 2A schematic diagram of the structure of a machine vision-oriented unmanned area parking terrain flatness recognition system provided in an embodiment of this application.

[0019] Figure labeling: 10 for multi-source data acquisition module, 20 for feature recognition and extraction module, 30 for fusion processing module, 40 for dynamic safe region generation module, and 50 for spatial overlap alignment module. Detailed Implementation

[0020] To further illustrate the technical means and effects adopted by the present invention in order to achieve the intended purpose, the following detailed description is provided in conjunction with the accompanying drawings and preferred embodiments, based on the specific implementation methods, structures, features and effects of the present invention.

[0021] This application provides a machine vision-based method for recognizing the flatness of uninhabited parking terrain, such as... Figure 1 As shown, the method includes:

[0022] Step S100: Acquire digital elevation model data, surface optical images, multispectral remote sensing images, SAR images, and near-real-time meteorological data for the target area during the window period, and preprocess the acquired multi-source data.

[0023] Preferably, digital elevation model data, surface optical images, multispectral remote sensing images, SAR images, and near-real-time meteorological data for the target area are acquired from various satellite, airborne, or ground-based sensor platforms. Specifically, digital elevation model data refers to the acquisition of Earth's surface elevation data recorded in a regular grid array, where each pixel value represents the elevation of that point, forming a digital model of the terrain undulations; surface optical images refer to high-resolution images captured in the visible and near-infrared bands that reflect the surface reflectivity characteristics, visually displaying the appearance, texture, and color of ground features; multispectral remote sensing images refer to images acquired in multiple... Image data recorded in specific electromagnetic wave bands are used to invert the physical and chemical properties of the Earth's surface by calculating various indices; SAR imagery refers to images formed by acquiring microwaves actively emitted by a synthetic aperture radar system and receiving their backscattered signals. It does not depend on sunlight, can penetrate clouds, and is sensitive to surface roughness, dielectric properties, and geometric structure; near-real-time meteorological data for a window period refers to timely meteorological observations or forecasts acquired around the time of image data acquisition. It may include information such as precipitation, temperature, humidity, and wind speed, and its time range covers a period before data acquisition and extends to a short-term forecast window in the future.

[0024] Furthermore, step S100 also includes step S110, which fills depressions, smooths edges, and performs coordinate system unification on the digital elevation model data; step S120, which performs distortion correction based on camera calibration parameters and brightness normalization based on histogram matching on the surface optical image; and step S130, which performs radiometric calibration, speckle noise suppression, and geocoding on the SAR image to generate a standardized backscattering coefficient map.

[0025] Preferably, the acquired multi-source data is preprocessed to correct it to a unified geometric and radiometric benchmark and eliminate errors caused by the sensors themselves and the environment. Specifically, the digital elevation model data is subjected to depression filling, edge smoothing, and coordinate system unification. Depression filling refers to identifying and correcting false depressions caused by data acquisition or processing errors to ensure the accuracy of water flow path analysis; edge smoothing refers to preserving the sharp boundaries of key terrain features such as ridgelines and valleys while suppressing data noise; coordinate system unification refers to transforming all data to the same geodetic coordinate system and projection system to ensure accurate spatial matching. The method involves performing distortion correction based on camera calibration parameters and brightness normalization based on histogram matching on surface optical images. Distortion correction based on camera calibration parameters refers to using the sensor's internal geometric model, such as focal length, principal point, and lens distortion coefficient, to correct image distortion caused by lens optical characteristics. Brightness normalization based on histogram matching refers to adjusting the brightness and contrast of images acquired at different times and under different lighting conditions to ensure a consistent visual appearance and statistical distribution, thereby reducing the interference of lighting changes on machine vision analysis.

[0026] Preferably, the SAR image is subjected to radiometric calibration, speckle noise suppression, and geocoding. Radiometric calibration refers to converting the raw digital values ​​of the image into radar backscattering coefficients with physical meaning, so that data acquired at different times and from different sensors have radiometric consistency and comparability. Speckle noise suppression refers to applying a Frost filter to suppress the inherent speckle noise of the SAR system, improve the image signal-to-noise ratio, and preserve ground feature details as much as possible. Geocoding refers to accurately converting the processed SAR image from the slant range-azimuth coordinate system to the geodetic coordinate system, generating an orthorectified product with accurate geographic coordinates, and then generating a standardized backscattering coefficient map.

[0027] Step S200: Based on machine vision, feature recognition and extraction are performed on the acquired surface optical images, multispectral remote sensing images, and SAR images to obtain visual recognition features.

[0028] Step S200 further includes step S210, using a convolutional neural network to segment the surface type of the surface optical image to obtain the surface material type, including rock, bare soil, sand, grassland, and obstacles; step S220, based on the surface material type, extracting local terrain undulation features from the surface optical image through a monocular depth estimation network to generate a visual depth map; step S230, using the surface material type and the corresponding visual depth map as visual recognition features.

[0029] Preferably, a convolutional neural network is used to segment the surface type of the optical image. The convolutional neural network may be a pre-trained semantic segmentation model, such as U-Net or DeepLab. The optical image is input into the network and classified pixel by pixel, assigning a pre-defined category label to each pixel to generate a surface material type segmentation map. This map is spatially aligned with the original optical image, but the value of each pixel represents its corresponding surface material category. Specific surface categories include passable surface cover types such as rocks, bare soil, sand, and grassland, as well as impassable obstacles such as vegetation, man-made debris, or large rocks. Then, a monocular depth estimation network is used to segment the surface type. Local terrain undulation features are extracted from optical images. A monocular depth estimation network is used, a convolutional neural network model that predicts the distance / depth of each pixel from a single 2D image. Surface material type is used as an auxiliary input channel to the monocular depth estimation network, inputting it along with the original optical image of the surface. This process generates a visual depth map, a grayscale image of the same size as the input image. The grayscale value of each pixel represents its relative elevation or distance in the real world; brighter pixels may represent higher points, and darker pixels represent lower points, thus depicting the undulation features of the local terrain. This allows for the recovery of 3D terrain information from 2D images, quantifying the slope, elevation changes, and other geometric morphologies of the surface. Finally, the surface material type and the corresponding visual depth map are used as visual recognition features to provide semantic and geometric / morphological information about the surface.

[0030] Furthermore, step S200 also includes step S240, calculating the normalized water index, soil-regulated vegetation index, and red edge index on the multispectral remote sensing image to identify surface water bodies, wetlands, and vegetation-covered areas; step S250, calculating surface moisture characteristics based on the standardized backscattering coefficient map and incident angle model to identify areas with abnormal surface moisture content; and step S260, based on the scattering mechanism of SAR, decomposing and distinguishing surface roughness types according to the standardized backscattering coefficient map.

[0031] Preferably, for the acquired multispectral remote sensing images, the normalized water index, soil-regulated vegetation index, and red-edge index are calculated using the reflectance values ​​of different bands. The normalized water index is calculated using the reflectance of the near-infrared and green bands. Water bodies have strong absorption in the near-infrared band but some reflectivity in the green band. Amplifying this characteristic effectively highlights water body information and is used to identify open surface water bodies and saturated soil areas. The soil-regulated vegetation index is a vegetation index that incorporates a soil brightness correction factor to reduce the amount of bare soil. Background interference with vegetation detection is used to more accurately identify and quantify vegetation cover areas, especially in areas with low vegetation cover or obvious soil background. The red edge is the area where the reflectance of the vegetation spectral curve rises sharply in the near-infrared band. The red edge index is calculated using bands sensitive to the location of the red edge to detect the physiological state of vegetation more precisely, such as chlorophyll content and biomass, which helps to distinguish vegetation types and health conditions. Furthermore, it can comprehensively identify surface water bodies, wetlands, and vegetation cover areas in different states, which are directly related to the carrying capacity and accessibility of the land surface.

[0032] Preferably, the surface moisture information is retrieved by combining the preprocessed standardized backscattering coefficient map with the radar incident angle model. Specifically, the radar backscattering intensity is significantly affected by the surface dielectric constant. The dielectric constant of water is much higher than that of dry soil. Increased soil moisture may lead to an increase in the dielectric constant, thereby increasing the backscattering coefficient. However, the backscattering coefficient is also affected by surface roughness and radar incident angle. Correction is performed based on the incident angle model to eliminate or reduce the scattering intensity changes caused by the change in incident angle due to topographic undulation, thereby more accurately separating the influence of moisture. Through correction, surface moisture characteristics that more directly reflect the soil moisture status can be calculated, which may be the relative humidity index or volumetric water content, thereby identifying areas with abnormal surface moisture content, such as overly moist soft soil areas and potential waterlogging areas.

[0033] Preferably, the radar signal scattering mechanism reflected by the standardized backscattering coefficient map is analyzed, and the surface roughness is classified. Specifically, the interaction mechanism between radar waves and the ground surface mainly includes surface scattering, volume scattering, and angular scattering. Among them, surface scattering occurs on very smooth surfaces, such as still water surfaces and smooth road surfaces. The radar waves undergo specular reflection and return almost no energy to the radar, which appears as extremely dark areas in the image. Volume scattering occurs in media with a certain volume and internal structure, such as dense vegetation and dry snow layers. The radar waves return after multiple reflections within the medium, usually producing moderate to strong backscattering. Angular scattering occurs on rough surfaces with angular structures, such as rock piles, urban buildings, and the interface between vegetation roots and the ground. The radar waves return along the original path after secondary reflection, producing very strong backscattering, which appears as bright spots in the image. By analyzing the absolute magnitude of the backscattering coefficient and the texture features in local areas, different types of surface roughness can be decomposed and distinguished. For example, the surface can be divided into categories such as smooth, medium rough, and very rough. Finally, areas where excessive surface roughness may cause severe vehicle bumps or impassability can be identified, such as rocky areas and dense shrubbery.

[0034] Furthermore, step S200 also includes step S270, which uses a threshold segmentation method to identify strong absorption regions in the backscattering coefficient map and marks regions with backscattering coefficients below a preset threshold as suspected water bodies; step S280, which uses time-series SAR images to perform InSAR interferometry analysis, extracts surface deformation information, detects the degree of surface disturbance based on multi-temporal SAR coherence, and marks regions with coherence below a preset threshold as unstable regions; step S280, which uses the identified water body features, humidity features, and surface deformation features as visual recognition features for SAR.

[0035] Preferably, a threshold segmentation method is used to identify strong absorption regions in the backscattering coefficient map. A preset numerical threshold, such as -12dB, is set for the preprocessed standardized backscattering coefficient map. Then, the backscattering coefficient map is traversed, and all pixel regions with backscattering coefficients lower than the preset threshold are marked as suspected water bodies. This allows for the rapid and direct identification of open water bodies or extremely saturated wetlands that may pose a serious threat to vehicles.

[0036] Preferably, SAR images of the same area acquired at different times are obtained and interferometrically processed, including precise registration, calculation of interferometric phase maps, removal of flatland effects and topographic phase, and detection of surface deformation along the radar line of sight by comparing small changes in radar phase between two observations, obtaining a surface deformation rate map or cumulative deformation map. Based on the interferometric processing, interferometric coherence is calculated, where coherence is an index that measures the stability of surface scattering characteristics between two observations, with a value ranging from 0 (completely incoherent) to 1 (completely coherent). If the surface undergoes drastic changes between two observations, such as vegetation growth, soil disturbance, vehicle compaction, snow melting, or drastic changes in soil moisture content, the scattering characteristics of the radar signal will change, resulting in a significant decrease in coherence. The average coherence or differential coherence over the time series is calculated, and areas with coherence below a preset threshold are marked as unstable areas, whose surface physical properties are in an unstable state. Furthermore, areas with risks of slow subsidence, landslides, and other geological hazards, as well as areas where the surface cover itself is unstable or prone to change, are comprehensively identified. Finally, the identified water features, humidity features, and surface deformation features are used together as visual identification features for SAR, indicating areas of immediate danger, areas with potentially insufficient soil bearing capacity, and areas with geological instability and surface volatility, respectively.

[0037] Step S300: The visual recognition features are fused with the digital elevation model data to identify surface undulation features, water features, and soil bearing capacity features, and to determine static, flat, and passable areas.

[0038] Step S300 further includes step S310, which uses a multi-scale feature pyramid network to scale-align the digital elevation model data with the visual depth features; step S320, wherein the terrain skeleton structure of the digital elevation model data is preserved in the coarse-scale layer, visual depth detail features are injected in the fine-scale layer in a local compensation manner, and the weighting coefficients of the multi-source features are adaptively adjusted through a channel attention mechanism to form a fused elevation model.

[0039] Preferably, visual recognition features are fused with digital elevation model (DEM) data. This involves inputting the DEM data and visual depth features into a multi-scale feature pyramid network for scale alignment. Specifically, convolution and pooling are used to generate feature maps of the same region at different scales in parallel. Fine-scale and coarse-scale layers capture detailed and macroscopic features, respectively. Scale alignment includes spatial alignment and feature layer alignment. Spatial alignment ensures geometric registration, while feature layer alignment refers to fusing the two feature maps at each corresponding scale. In the lower-resolution coarse-scale layer, which is more reliable and accurate in describing large-scale terrain trends such as ridge orientation and valley outlines, the terrain skeleton structure of the DEM data is preserved to prevent cumulative errors in the visual depth map from distorting the overall terrain. In the higher-resolution fine-scale layer, local compensation is used... The method injects visual depth detail features as a supplement, adding the detailed information provided by the visual depth map to the terrain skeleton structure to recover micro-topographic features such as small mounds, shallow ditches, and subtle undulations of rocks that were smoothed out by LiDAR or photogrammetry. A channel attention mechanism adaptively evaluates the reliability and importance of terrain elevation features and visual depth features at the current image region and scale, and calculates corresponding weighting coefficients. For example, in areas where the optical image is covered by shadows, visual depth may be unreliable, and the attention mechanism automatically reduces the weight of visual depth features while increasing the weight of terrain elevation features. Conversely, in well-lit, textured, flat areas, the weight of visual depth details is increased. This process then generates a fused elevation model, ensuring optimal fusion results in various complex scenarios.

[0040] Furthermore, step S300 also includes step S330, which calculates the slope value of each grid based on the fused elevation model and marks areas with slopes lower than the first dynamic threshold as slope candidate areas; step S340, within a multi-scale sliding window, calculates the elevation standard deviation as a terrain roughness index and marks areas with roughness lower than the second dynamic threshold as roughness candidate areas; step S350, calculates the normalized water index based on multispectral imagery and identifies open water areas by combining SAR backscattering coefficient threshold segmentation; step S360, identifies shadowed pseudo-water bodies, seasonal wetlands, and snowmelt water areas based on visual semantic segmentation results; and step S370, performs flat area screening based on the identified candidate areas, as well as the open water areas, shadowed pseudo-water bodies, seasonal wetlands, and snowmelt water areas, to obtain the static flat passable area.

[0041] Preferably, based on the fused elevation model, a spatial analysis algorithm is used to calculate the slope value of each grid cell. A first dynamic threshold is preset based on the vehicle's climbing ability and can be dynamically adjusted according to task requirements. All grid cells with slope values ​​lower than the first dynamic threshold are marked as slope candidate areas, representing areas that are potentially passable from the perspective of slope. A multi-scale sliding window, such as a 3×3, 5×5, or 7×7 pixel window, is used to traverse the fused elevation model. Within each window, the standard deviation of all grid elevation values ​​is calculated as a terrain roughness index. The larger the standard deviation, the more severe the surface potholes and undulations, and the worse the travel experience. A second dynamic threshold is preset, and all areas within the window with an elevation standard deviation lower than this threshold are marked as roughness candidate areas, representing areas that are potentially passable from the perspective of surface smoothness. Then, the normalized water index is calculated from the multispectral image, and the backscattering coefficient is used from the SAR image for threshold segmentation. Areas that point to each other are identified as open water areas, representing absolutely impassable areas.

[0042] Preferably, based on the visual semantic segmentation results, shadowed pseudo-water bodies, seasonal wetlands, and snowmelt water areas are identified. These include areas where the shadows of mountains or buildings in optical images are dark and easily misidentified as water bodies; areas that dry up and fill with water depending on the season; and temporary water accumulation areas formed by melting snow and ice. Finally, based on the identified candidate areas, open water areas, shadowed pseudo-water bodies, seasonal wetlands, and snowmelt water areas, flat areas are screened. This involves performing spatial intersection operations on slope candidate areas and roughness candidate areas to obtain preliminary flat areas that simultaneously satisfy the requirements of gentle slope and flat surface. Open water areas, shadowed pseudo-water bodies, seasonal wetlands, and snowmelt water areas are then excluded from the preliminary flat areas. Finally, the remaining areas are determined as safe, reliable, and passable static flat and passable areas.

[0043] Furthermore, step S370 also includes step S371, which verifies the status of the shadowed pseudo-water bodies, seasonal wetlands, and snowmelt water areas. Specifically, for the shadowed pseudo-water bodies, the consistency between the shadow distribution of semantic segmentation and the mountain shadow generated by the digital elevation model data is determined. For the seasonal wetlands, the active and dry states are identified. For the dry seasonal wetlands, the soil carrying capacity is additionally assessed. When the carrying capacity is greater than the safety threshold, the status verification is determined to be inconsistent. For the snowmelt water areas, thermal infrared band data is introduced, and the snowmelt process is detected by the difference between the surface temperature and the surrounding environment. Step S372, the shadowed pseudo-water bodies, seasonal wetlands, snowmelt water areas, and open water areas with consistent status verification are removed to obtain the static flat and passable area.

[0044] Preferably, the status verification of shadow pseudo-water bodies, seasonal wetlands, and snowmelt water areas is performed. Specifically, the consistency between the shadow distribution of semantic segmentation and the mountain shadows generated by digital elevation model data is compared. If they are inconsistent, it indicates that the shadow comes from an independent object and should not be considered a water body or dangerous terrain, so it is removed from the shadow pseudo-water body list. Seasonal wetlands are identified as active or dry, and it is determined whether the area has sufficient carrying capacity for passage during the dry period. Specifically, the soil carrying capacity model is used to evaluate the dry area. If the carrying capacity is greater than the safety threshold, it means that the ground is hard enough and can be removed from the danger list. If the carrying capacity is less than the safety threshold, it means that the ground is still soft and there is a risk of vehicles getting stuck, indicating that the status verification is consistent. The status verification of snowmelt water areas is performed to confirm whether the area is in a dangerous state of snowmelt water accumulation. That is, thermal infrared data is introduced to analyze the surface temperature. The temperature of the area where snow is melting is significantly lower than the surrounding environment due to heat absorption during phase change. If a low temperature anomaly is detected, the status verification is confirmed to be consistent. If no low temperature is detected, it may just be residual solid snow or ice, which can be removed from the danger list. Finally, from the identified slope and roughness candidate areas, truly dangerous areas such as shadowy pseudo-water bodies, seasonal wetlands and snowmelt water areas, and open water areas with consistent status verification are eliminated to obtain a highly reliable and final static flat and passable area.

[0045] Step S400: Based on the near-real-time meteorological data of the window period combined with the analysis data of the SAR image, predict the potential catchment area and soil moisture changes, and generate a dynamic safe zone.

[0046] Step S400 further includes step S410, constructing a multimodal soil carrying capacity estimation model, wherein the input of the multimodal soil carrying capacity estimation model includes multispectral reflectance characteristics, SAR surface soil moisture inversion results, and surface material types obtained from visual semantic segmentation, and outputting a soil carrying capacity distribution map through a gradient boosting decision tree regression model; Step S420, based on the soil carrying capacity distribution map, coupling near-real-time meteorological data with the topographic parameters of the digital elevation model, simulating the surface runoff formation and soil moisture migration process in the target area within a future preset time window, and predicting potential catchment areas and soft soil areas; Step S430, performing differential interferometry processing on at least three periods of SAR images to identify high-risk areas for geological hazards where the surface deformation rate exceeds a safety threshold, wherein the SAR images should have a uniform and continuous time series, with the time interval not exceeding twice the satellite revisit period, and constructing a deformation time series; Step S440, removing the predicted potential catchment areas and soft soil areas, as well as the high-risk areas for geological hazards, to generate a dynamic safe area.

[0047] Preferably, based on the analysis of near-real-time meteorological data during the window period combined with SAR imagery, potential catchment areas and soil moisture changes are predicted and identified to determine the risks that dynamic environmental changes may bring in the future, generating dynamic safety zones to ensure vehicle safety throughout the parking period. Specifically, a multimodal soil carrying capacity estimation model is constructed to predict the soil carrying capacity of each geographic location, i.e., the pressure that a unit area of ​​soil can withstand. The inputs of the multimodal soil carrying capacity estimation model include multispectral reflectance features, SAR surface soil moisture inversion results, and surface material types obtained from visual semantic segmentation. Among them, multispectral reflectance features are band reflectance and derived indices extracted from multispectral images, indirectly reflecting soil composition, organic matter content, and vegetation cover; SAR surface soil moisture inversion results are used to provide a direct estimate of the current soil moisture content, which is the most critical factor affecting carrying capacity; surface material types obtained from visual semantic segmentation provide semantic information about the surface, and different types of materials have different inherent carrying capacity ranges; the input multimodal features are processed by a gradient boosting decision tree regression model to predict the estimated carrying capacity value corresponding to each pixel, and the soil carrying capacity distribution map is output.

[0048] Preferably, the soil bearing capacity distribution map is coupled with near-real-time meteorological data and topographic parameters of the digital elevation model, and hydrological and soil physical processes are simulated. This simulates the formation of surface runoff and soil moisture migration in the target area within a preset time window. Specifically, the soil bearing capacity distribution map is used as the baseline condition, and near-real-time meteorological data containing forecast or measured precipitation, as well as topographic parameters of the digital elevation model such as slope, aspect, and catchment area, are input. The hydrological model is used to simulate how precipitation generates runoff in the watershed and collects along the terrain, while simulating how water infiltrates into the soil and migrates underground. This helps to identify low-lying areas that will form temporary water accumulation or excessive surface moisture due to runoff collection in the future, i.e., predict potential catchment areas. At the same time, the simulation identifies soil areas where the bearing capacity decreases significantly due to increased saturation caused by water infiltration, i.e., predict potential soft soil areas.

[0049] Preferably, differential interferometry is used to process at least three uniform and continuous time-series SAR images to ensure interferometric coherence. Then, the deformation rate of each pixel in the radar line-of-sight direction is extracted to construct a deformation time series. The trend of slow geological movements such as surface subsidence and landslides is analyzed, and high-risk areas of geological hazards where the surface deformation rate exceeds the safety threshold are identified and marked as high-risk areas of geological hazards, such as active landslides and ground subsidence areas. The time interval of SAR images does not exceed twice the satellite revisit period. Finally, the predicted potential catchment areas, soft soil areas, and high-risk areas of geological hazards are marked as dynamic risk areas and removed. The remaining areas are output as dynamic safe areas, which are not only flat and solid at present, but also have a low risk of becoming unsafe in the foreseeable future due to rainfall, catchment, soil softening, or geological hazards. This ensures the safety and reliability of unmanned vehicles operating in complex uninhabited areas for extended periods.

[0050] Step S500: Spatially overlap and align the dynamic safety area with the static flat passable area to determine the passable flat area.

[0051] Preferably, the dynamic safe area and the static flat passable area are spatially overlapped and aligned to ensure that the static flat passable area and the dynamic safe area are under the same geographic coordinate system and projection, so that each pixel is completely corresponding in space. Then, a pixel-by-pixel logical "AND" operation is performed on them. When the same pixel is marked as "passable" in the static flat passable area layer and is also marked as "safe" in the dynamic safe area layer, the pixel is retained in the final result and determined as a passable flat area. This reduces the probability of autonomous vehicles falling into danger due to dynamic changes in the environment in complex and unknown environments and improves the safety of parking in uninhabited areas.

[0052] In the above text, refer to Figure 1 A method for recognizing the flatness of uninhabited parking terrain based on machine vision, according to an embodiment of the present invention, is described in detail. Next, reference will be made to... Figure 2 A machine vision-based terrain flatness recognition system for parking in uninhabited areas, according to an embodiment of the present invention, is described.

[0053] The machine vision-oriented unmanned area parking terrain flatness recognition system according to embodiments of the present invention addresses the technical problem in existing technologies that fail to effectively cope with dynamic changes in the unmanned area environment, resulting in insufficient accuracy, comprehensiveness, and safety in flatness area recognition. It achieves the technical effect of improving the accuracy, comprehensiveness, and safety of unmanned area parking terrain recognition through multi-source data fusion and dynamic / static combined analysis. Figure 2As shown, the unmanned parking terrain flatness recognition system for machine vision includes: a multi-source data acquisition module 10, a feature recognition and extraction module 20, a fusion processing module 30, a dynamic safe area generation module 40, and a spatial overlap alignment module 50.

[0054] The multi-source data acquisition module 10 is used to acquire digital elevation model data, surface optical images, multispectral remote sensing images, SAR images, and near-real-time meteorological data of the target area during the window period, and to preprocess the acquired multi-source data; the feature recognition and extraction module 20 is used to perform feature recognition and extraction on the acquired surface optical images, multispectral remote sensing images, and SAR images based on machine vision to obtain visual recognition features; the fusion processing module 30 is used to fuse the visual recognition features with the digital elevation model data to identify surface undulation features, water body features, and soil bearing capacity features, and to determine static flat and passable areas; the dynamic safe area generation module 40 is used to predict potential catchment areas and soil moisture changes based on the near-real-time meteorological data of the window period combined with the analysis data of the SAR images, and to generate dynamic safe areas; the spatial overlap and alignment module 50 is used to spatially overlap and align the dynamic safe areas with the static flat and passable areas to determine passable flat areas.

[0055] The specific configuration of the multi-source data acquisition module 10 will be described in detail below. The multi-source data acquisition module 10 further includes: filling depressions, smoothing edges, and performing coordinate system unification processing on the digital elevation model data; performing distortion correction based on camera calibration parameters and brightness normalization based on histogram matching on the surface optical image; and performing radiometric calibration, speckle noise suppression, and geocoding processing on the SAR image to generate a standardized backscattering coefficient map.

[0056] The specific configuration of the feature recognition and extraction module 20 will be described in detail below. The feature recognition and extraction module 20 further includes: using a convolutional neural network to segment the surface type of the optical image to obtain surface material types, including rocks, bare soil, sand, grassland, and obstacles; based on the surface material types, extracting local terrain undulation features from the optical image through a monocular depth estimation network to generate a visual depth map; and using the surface material types and the corresponding visual depth maps as visual recognition features.

[0057] The specific configuration of the feature recognition and extraction module 20 will be described in detail below. The feature recognition and extraction module 20 further includes: calculating the normalized water index, soil-regulated vegetation index, and red edge index for the multispectral remote sensing image to identify surface water bodies, wetlands, and vegetation cover areas; calculating surface moisture characteristics based on the standardized backscattering coefficient map and the incident angle model to identify areas with abnormal surface water content; and decomposing and distinguishing surface roughness types based on the SAR scattering mechanism and the standardized backscattering coefficient map.

[0058] The specific configuration of the feature recognition and extraction module 20 will be described in detail below. The feature recognition and extraction module 20 further includes: using a threshold segmentation method to identify strong absorption regions in the backscattering coefficient map, and marking regions with backscattering coefficients below a preset threshold as suspected water bodies; performing InSAR interferometry analysis using time-series SAR imagery to extract surface deformation information, detecting the degree of surface disturbance based on multi-temporal SAR coherence, and marking regions with coherence below a preset threshold as unstable regions; and using the identified water body features, humidity features, and surface deformation features as the visual recognition features for SAR.

[0059] The specific configuration of the fusion processing module 30 will be described in detail below. The fusion processing module 30 further includes: using a multi-scale feature pyramid network to perform scale alignment between the digital elevation model data and the visual depth features; wherein, the terrain skeleton structure of the digital elevation model data is preserved in the coarse-scale layer, visual depth detail features are injected in the fine-scale layer in a local compensation manner, and the weighting coefficients of the multi-source features are adaptively adjusted through a channel attention mechanism to form a fused elevation model.

[0060] The specific configuration of the fusion processing module 30 will be described in detail below. The fusion processing module 30 further includes: calculating the slope value of each grid cell based on the fused elevation model, and marking areas with slopes below a first dynamic threshold as slope candidate areas; calculating the elevation standard deviation as a terrain roughness index within a multi-scale sliding window, and marking areas with roughness below a second dynamic threshold as roughness candidate areas; calculating the normalized water index based on multispectral imagery, and identifying open water areas by combining SAR backscattering coefficient threshold segmentation; identifying shadowed pseudo-water bodies, seasonal wetlands, and snowmelt water areas based on visual semantic segmentation results; and filtering flat areas based on the identified candidate areas, as well as the open water areas, shadowed pseudo-water bodies, seasonal wetlands, and snowmelt water areas, to obtain the statically flat and passable area.

[0061] The specific configuration of the fusion processing module 30 will be described in detail below. The fusion processing module 30 further includes: performing state verification and determination on the shadow pseudo-water bodies, seasonal wetlands, and snowmelt water areas. Specifically, for the shadow pseudo-water bodies, consistency is determined by comparing the shadow distribution of semantic segmentation with the mountain shadows generated by the digital elevation model data. For the seasonal wetlands, active and dry states are identified. For dry seasonal wetlands, soil carrying capacity is additionally assessed; if the carrying capacity exceeds a safety threshold, the state verification is deemed inconsistent. For the snowmelt water areas, thermal infrared data is introduced, and the ongoing snowmelt process is detected by the difference between the surface temperature and the surrounding environment. The shadow pseudo-water bodies, seasonal wetlands, snowmelt water areas, and open water areas with consistent state verification are removed to obtain the static, flat, and passable area.

[0062] The specific configuration of the dynamic safe zone generation module 40 will be described in detail below. The dynamic safe zone generation module 40 further includes: constructing a multimodal soil bearing capacity estimation model, wherein the inputs of the multimodal soil bearing capacity estimation model include multispectral reflectance characteristics, SAR surface soil moisture inversion results, and surface material types obtained from visual semantic segmentation; outputting a soil bearing capacity distribution map through a gradient boosting decision tree regression model; based on the soil bearing capacity distribution map, coupling near-real-time meteorological data with the topographic parameters of the digital elevation model to simulate the surface runoff formation and soil moisture migration process in the target area within a future preset time window, predicting potential catchment areas and soft soil areas; performing differential interferometry processing on at least three periods of SAR images to identify high-risk areas for geological hazards where the surface deformation rate exceeds a safety threshold, wherein the SAR images should have a uniform and continuous time series, with time intervals not exceeding twice the satellite revisit period, constructing a deformation time series; and removing the predicted potential catchment areas, soft soil areas, and high-risk areas for geological hazards to generate a dynamic safe zone.

[0063] The machine vision-oriented unmanned parking terrain flatness recognition system provided in this embodiment of the invention can execute the machine vision-oriented unmanned parking terrain flatness recognition method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0064] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A machine vision-based method for recognizing the flatness of parking terrain in uninhabited areas, characterized in that, include: Acquire digital elevation model data, surface optical images, multispectral remote sensing images, SAR images, and near-real-time meteorological data for the target area during the window period, and preprocess the acquired multi-source data; Based on machine vision, feature recognition and extraction are performed on the acquired surface optical images, and feature recognition and extraction are performed on SAR images to obtain visual recognition features. The visual recognition features are fused with the digital elevation model data to identify surface undulation features. Based on the surface undulation features, water features, and soil bearing capacity features, static flat and passable areas are determined. Based on the analysis of near-real-time meteorological data during the window period combined with the SAR imagery, potential catchment areas and soft soil areas, as well as high-risk areas for geological disasters, are predicted, and dynamic safe zones are generated. The dynamic safety zone and the static flat passable zone are spatially overlapped and aligned to determine the passable flat zone. Feature extraction from acquired surface optical images based on machine vision includes: The surface optical image is segmented using a convolutional neural network to obtain the surface material types, including rocks, bare soil, sand, grassland, and obstacles. Based on the surface material type, local terrain undulation features are extracted from the surface optical image using a monocular depth estimation network to generate a visual depth map. The surface material type and the corresponding visual depth map are used as visual recognition features; Feature recognition and extraction from SAR images includes: Surface humidity characteristics are calculated based on standardized backscattering coefficient maps and incident angle models to identify areas with abnormal surface moisture content. Based on the scattering mechanism of SAR, the surface roughness type is decomposed and distinguished according to the standardized backscattering coefficient map. Also includes: A threshold segmentation method is used to identify strong absorption regions in the backscattering coefficient map, and regions with backscattering coefficients below a preset threshold are marked as suspected water bodies. InSAR interferometry analysis was performed using time-series SAR images to extract surface deformation information. The degree of surface disturbance was detected based on multi-temporal SAR coherence, and areas with coherence below a preset threshold were marked as unstable areas. The identified water body features, humidity features, and surface deformation features are used together as the visual recognition features of SAR; Determine the static, flat, and passable area, including: Based on the fused elevation model, the slope value of each grid is calculated, and areas with a slope lower than the first dynamic threshold are marked as slope candidate areas. Within a multi-scale sliding window, the elevation standard deviation is calculated as a terrain roughness index, and areas with roughness below the second dynamic threshold are marked as roughness candidate areas. Normalized water index is calculated based on multispectral imagery, and open water areas are identified by combining SAR backscattering coefficient threshold segmentation. Based on the visual semantic segmentation results, identify shadowed pseudo-water bodies, seasonal wetlands, and snowmelt water areas; Based on the identified candidate areas, as well as the open water areas, shadowed pseudo-water areas, seasonal wetlands, and snowmelt water areas, flat areas are screened to obtain the static flat passable areas. The status of the shadowed pseudo-water bodies, seasonal wetlands, and snowmelt water areas is verified and determined; the active and dry states of the seasonal wetlands are identified, and the soil carrying capacity of the dry seasonal wetlands is additionally assessed. When the carrying capacity is greater than the safety threshold, the status verification is determined to be inconsistent. Based on the soil bearing capacity distribution map, the surface runoff formation and soil moisture migration process in the target area within a future preset time window are simulated by coupling near real-time meteorological data with the topographic parameters of the digital elevation model, and potential catchment areas and soft soil areas are predicted. Differential interferometry is performed on at least three SAR images to identify high-risk areas for geological hazards where the surface deformation rate exceeds a safety threshold. The SAR images should have a uniform and continuous time series with time intervals not exceeding twice the satellite revisit period to construct a deformation time series. The predicted potential catchment areas, soft soil areas, and high-risk areas for geological hazards are marked and removed to generate dynamic safe areas.

2. The machine vision-based method for recognizing flat terrain in uninhabited parking areas according to claim 1, characterized in that, Preprocessing of the acquired multi-source data includes: The digital elevation model data is subjected to depression filling, edge smoothing, and coordinate system processing. The surface optical image is subjected to distortion correction based on camera calibration parameters and brightness normalization based on histogram matching. The SAR image is radiometrically calibrated, speckle noise suppressed, and geocoded to generate a standardized backscattering coefficient map.

3. The machine vision-based method for recognizing the flatness of parking terrain in uninhabited areas according to claim 1, characterized in that, The process of fusing the visual recognition features with the digital elevation model data includes: A multi-scale feature pyramid network is used to align the scale of the digital elevation model data with the visual recognition features. Specifically, the terrain skeleton structure of the digital elevation model data is preserved in the coarse-scale layer, visual depth detail features are injected in the fine-scale layer through local compensation, and the weighting coefficients of the multi-source features are adaptively adjusted through the channel attention mechanism to form a fused elevation model.

4. The machine vision-based method for recognizing flat terrain in uninhabited parking areas according to claim 1, characterized in that, Obtain the static, flat, passable area. include: Specifically, the consistency between the shadow pseudo-water body and the mountain shadow generated by the digital elevation model data is determined by comparing the shadow distribution of the semantic segmentation. For the snowmelt water accumulation area, thermal infrared band data is introduced to detect the snowmelt process that is taking place by the difference between the surface temperature and the surrounding environment. The shadowed pseudo-water bodies, seasonal wetlands, snowmelt water areas, and open water areas that are consistent with the status verification are removed to obtain the static flat and passable area.

5. The machine vision-based method for recognizing flat terrain in uninhabited parking areas according to claim 1, characterized in that, Based on the analysis of near-real-time meteorological data during the window period combined with the SAR imagery, potential catchment areas and soil moisture changes are predicted, generating dynamic safety zones, including: A multimodal soil carrying capacity estimation model is constructed. The inputs of the multimodal soil carrying capacity estimation model include multispectral reflectance features, SAR surface soil moisture inversion results, and surface material types obtained by visual semantic segmentation. The soil carrying capacity distribution map is output through a gradient boosting decision tree regression model.

6. A machine vision-based terrain flatness recognition system for parking in uninhabited areas, characterized in that: The system is used to implement the machine vision-oriented terrain flatness recognition method for parking in uninhabited areas as described in any one of claims 1 to 5, the system comprising: The multi-source data acquisition module is used to acquire digital elevation model data, surface optical images, multispectral remote sensing images, SAR images, and near-real-time meteorological data of the target area during the window period, and to preprocess the acquired multi-source data. The feature recognition and extraction module is used to perform feature recognition and extraction on the acquired surface optical images based on machine vision, and to perform feature recognition and extraction on SAR images to obtain visual recognition features. The fusion processing module is used to fuse the visual recognition features with the digital elevation model data, identify the surface undulation features, and determine the static flat and passable area based on the surface undulation features, water features, and soil bearing capacity features. The dynamic safety zone generation module is used to predict potential catchment areas and soft soil areas, as well as high-risk areas for geological disasters, based on near-real-time meteorological data during the window period combined with the analysis data of the SAR image, and to generate dynamic safety zones. The spatial overlap alignment module is used to spatially overlap and align the dynamic safety area with the static flat passable area to determine the passable flat area.

Images