An automatic driving obstacle identification method for a complex scenic spot road scene

Abstract

The application discloses a kind of complex scenic spot road scene-oriented automatic driving obstacle identification method, comprising: collecting scenic spot road original image;Input multi-scale feature extraction network, generate multi-scale feature map;It is input into multi-branch decoder, respectively obtain obstacle region segmentation result, obstacle position bounding box coordinates and obstacle type classification probability;Combine the surrounding environment information of road geometric shape, weather and signal light state, using attention mechanism to its feature weighting, generate initial obstacle saliency map;According to the preset obstacle type priority level, detect and process the recognition result of spatial overlap in the map, obtain the obstacle recognition result after conflict resolution;It is consistent with road path segmentation map Space collation, and based on time series smoothing processing eliminates interframe jump, finally output recognition result.The application effectively solves the problem of high-precision, real-time obstacle identification for complex scenic spot road scene.

Description

Technical Field

[0001] This invention relates to the technical fields of computer vision and intelligent transportation in scenic areas, and in particular to an obstacle recognition method for autonomous driving in complex scenic road scenarios. Background Technology

[0002] With the rapid development of artificial intelligence and computer vision technologies, environmental perception and obstacle recognition technologies in autonomous driving systems have become core research directions in the field of intelligent transportation. Traditional obstacle recognition methods mainly rely on single sensor data processing, acquiring environmental information through devices such as radar, lidar, or cameras, and using basic image processing algorithms for target detection and classification. However, with the rise of deep learning technology, visual perception methods based on convolutional neural networks have gradually become mainstream, especially showing significant advantages in multi-target recognition tasks in complex road environments. Currently, obstacle recognition frameworks typically adopt an end-to-end deep learning architecture, combining multiple sub-tasks such as feature extraction, target detection, and semantic segmentation to achieve accurate recognition of various targets such as vehicles, pedestrians, and traffic signs in road scenes. At the same time, the application of multimodal fusion technology further improves the robustness of the system under adverse weather conditions, while the introduction of temporal information effectively alleviates the misidentification problem in single-frame detection.

[0003] CN115230694A discloses an obstacle recognition method for autonomous vehicles. Although obstacle screening is achieved through Frenet coordinate system transformation, this method is mainly designed for structured roads and is not adaptable to complex terrain, diverse obstacle types, and dynamic environmental changes in scenic roads. In particular, it is prone to misjudgment when dealing with non-standard road geometry.

[0004] CN118587683A discloses an obstacle recognition method based on generative adversarial networks, which improves the model's generalization ability through data augmentation. However, its feature extraction process lacks specific consideration for special environmental factors of scenic roads, such as complex lighting conditions, changing weather conditions, and mobile obstacles unique to scenic areas (tourists, sightseeing vehicles, etc.), resulting in problems such as insufficient recognition accuracy and poor real-time performance in practical applications.

[0005] Furthermore, existing technologies generally lack effective fusion mechanisms for multi-scale features, making it difficult to simultaneously achieve accurate detection of small targets at long distances and complete segmentation of large targets at close range. Moreover, they lack effective conflict resolution strategies when dealing with spatially overlapping targets, which easily leads to false detections and missed detections. Therefore, existing autonomous driving obstacle recognition technologies suffer from insufficient adaptability to complex scenic road environments, limited multi-scale feature fusion capabilities, lack of utilization of environmental context information, and imperfect mechanisms for handling spatially overlapping targets. This invention primarily addresses the problem of high-precision, real-time obstacle recognition in complex scenic road scenarios. Summary of the Invention

[0006] The purpose of this section is to outline some aspects of the embodiments of the present invention and to briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section, as well as in the specification abstract and the title of the invention, to avoid obscuring the purpose of this section, the specification abstract, and the title of the invention. Such simplifications or omissions shall not be used to limit the scope of the invention.

[0007] In view of the aforementioned existing problems, the present invention is proposed.

[0008] To solve the above-mentioned technical problems, the present invention provides the following technical solution: acquiring original images of scenic roads and performing noise suppression, brightness equalization and contrast enhancement to obtain enhanced images with uniform resolution;

[0009] The enhanced image is input into a multi-scale feature extraction network to extract texture features, shape features, and semantic features, which are then fused through a feature pyramid network to generate a multi-scale feature map.

[0010] The multi-scale feature map is input into the multi-branch decoder to obtain the obstacle region segmentation result, the obstacle location bounding box coordinates, and the obstacle type classification probability, respectively.

[0011] Combining road geometry, weather and traffic light status information of the surrounding environment, an attention mechanism is used to perform feature weighting on the obstacle region segmentation results, obstacle location bounding box coordinates and obstacle type classification probabilities to generate an initial obstacle saliency map;

[0012] Based on the preset obstacle type priority level, the recognition results of spatial overlap in the initial obstacle salience map are detected and processed, high-priority targets are retained and low-priority targets are suppressed to obtain the obstacle recognition results after conflict resolution.

[0013] The obstacle recognition results are spatially consistent with the road path segmentation map, and inter-frame jumps are eliminated based on time series smoothing. The final output includes the recognition results containing obstacle type, location, segmentation mask and confidence level.

[0014] As a preferred embodiment of the obstacle recognition method for autonomous driving in complex scenic road scenarios described in this invention, the enhanced road image obtained includes:

[0015] Bilateral filtering is performed on the original image of the scenic road to suppress random noise while preserving edge details;

[0016] An adaptive histogram equalization method is used to locally enhance the contrast in areas with uneven brightness distribution.

[0017] The gamma coefficient is dynamically adjusted based on global brightness statistics to make the overall brightness of the enhanced image more balanced.

[0018] As a preferred embodiment of the obstacle recognition method for autonomous driving in complex scenic road scenarios described in this invention, the acquisition of the original images of the scenic roads meets the following environmental requirements, including:

[0019] Images of scenic area roads were collected under sunny, cloudy, rainy, foggy, and nighttime conditions.

[0020] For each batch of data collected under each weather condition, there should be at least three lighting orientations: facing the sun directly, facing away from the sun, and side lighting.

[0021] The scenic area road images include at least images of main roads within the scenic area, images of secondary roads within the scenic area, images of parking lots within the scenic area, and images of scenic area roads under different weather conditions, wherein:

[0022] The scenic road images under different meteorological conditions include, but are not limited to, strong light images under sunny high illumination conditions, weak light images under cloudy low illumination conditions, reflective images under rainy and dry conditions, blurred images under foggy conditions with reduced visibility, as well as clear images under daytime illumination conditions, supplementary lighting images under twilight light changes, and low illumination images under nighttime artificial lighting conditions.

[0023] As a preferred embodiment of the obstacle recognition method for autonomous driving in complex scenic road scenarios described in this invention, the generation of multi-dimensional feature maps includes:

[0024] The enhanced road image is used as input data, and features are extracted layer by layer from the input image through the shallow layer group, middle layer group and deep layer group in the multi-dimensional feature extraction network.

[0025] The shallow layer group extracts image edge texture features, the middle layer group extracts image shape and structure features, and the deep layer group extracts image semantic content features.

[0026] The feature pyramid network fuses features from different layers and unifies the feature map size through upsampling and downsampling operations, ultimately outputting a multi-dimensional feature map containing multi-dimensional feature information.

[0027] As a preferred embodiment of the obstacle recognition method for autonomous driving in complex scenic road scenarios described in this invention, the multi-dimensional feature map is input into a multi-branch decoder, and obstacle region segmentation results, obstacle location results, and obstacle type classification results are obtained through three decoding branches, respectively:

[0028] The first decoding branch is the obstacle region segmentation branch, which analyzes the image resolution on the multi-dimensional feature map through gradient inverse matrix operation, and determines the obstacle region attribution of each pixel through a pixel-level classifier to generate a binary obstacle region segmentation mask.

[0029] The second decoding branch is the obstacle location branch, which converts the multi-dimensional feature map into fixed-length features through global average pooling operation for analysis, and then uses a multi-layer connection network to regress and predict the bounding box coordinate information of the obstacle.

[0030] The third decoding branch is the obstacle classification type branch, which extracts global signal features through feature aggregation operations and then outputs the probability distribution of each obstacle through the classifier network.

[0031] As a preferred embodiment of the obstacle recognition method for autonomous driving in complex scenic road scenarios described in this invention, the obstacle region segmentation result includes: describing the boundary information of the obstacle region with pixel-level precision, each pixel corresponding to a binary label used to indicate whether the pixel belongs to the obstacle region, forming a complete obstacle segmentation mask map;

[0032] The obstacle location results include: obstacle spatial location information represented by bounding boxes, each bounding box being determined by the coordinates of the upper left corner and the lower right corner, indicating the accurate location range of the obstacle in the image, and also including the confidence score of the bounding boxes, which is used to indicate the reliability of the location results;

[0033] The obstacle type classification results include: classification labels for common obstacle types in scenic area roads, including at least pedestrian categories, traffic vehicle categories, park maintenance equipment categories, and fixed landscape facility categories. Each category label corresponds to a probability value, which is used to indicate the likelihood of the obstacle belonging to a certain category.

[0034] As a preferred embodiment of the obstacle recognition method for autonomous driving in complex scenic road scenarios described in this invention, the step of obtaining the obstacle salience feature map includes:

[0035] Extract surrounding environmental information of scenic area roads, including road geometry, surrounding road landscape layout, signal conditions, and weather conditions;

[0036] A multimodal fusion framework is constructed to convert obstacle region segmentation results into spatial saliency features, obstacle location results into location saliency features, and obstacle type classification results into semantic saliency features.

[0037] The correlation weights between different modal features are calculated using an attention mechanism, and the weights of each modal feature are adaptively adjusted and fused based on the regional path context information.

[0038] The spatial saliency features, location saliency features, and semantic saliency features are merged into a unified discrete obstacle saliency feature map through a weighted fusion operation.

[0039] As a preferred embodiment of the obstacle recognition method for autonomous driving in complex scenic road scenarios described in this invention, the obstacle location results and obstacle type classification results in the obstacle salience feature map are corrected using an obstacle type priority strategy, including:

[0040] Establish a priority level for obstacle types in road scenarios, with pedestrians as the first priority, vehicles as the second priority, park maintenance equipment as the third priority, and fixed landscape facilities as the fourth priority.

[0041] Conflict detection is performed on multiple obstacle identification results where the salient features of obstacles overlap spatially. When multiple different types of obstacles are detected in the same space, the results are filtered according to a priority strategy.

[0042] The identification results of fourth-priority obstacles are transcribed using non-maximum transcription factors, while the location and type classification results of first-priority obstacles are retained, resulting in obstacle identification results after conflict resolution.

[0043] As a preferred embodiment of the obstacle recognition method for autonomous driving in complex scenic road scenarios described in this invention, the corrected recognition results are checked for consistency by combining scenic road path segmentation information, and finally the scenic obstacle recognition results are output, including:

[0044] By segmenting the scenic area roads into road path regions using a segmentation network, a road path segmentation map is generated, which includes passable road areas, tourist pedestrian areas, green landscape areas, and building areas.

[0045] The modified obstacle recognition results are spatially consistent with the road path segmentation map to check whether the recognized location of the obstacle matches the road functional area that should appear.

[0046] Verify the identification results that do not meet the spatial consistency verification.

[0047] The system eliminates inter-frame jumps by smoothing the time series data, and finally outputs a complete obstacle recognition result that includes obstacle type labels, location coordinates, region segmentation masks, and confidence scores.

[0048] As a preferred embodiment of the obstacle recognition method for autonomous driving in complex scenic road scenarios described in this invention, the time series smoothing process specifically includes:

[0049] Based on the Kalman filter algorithm, the position and velocity of obstacles in adjacent frames are estimated;

[0050] The estimation results are fed back into the current frame recognition results to reduce instantaneous localization jitter and update the confidence level;

[0051] Output the final obstacle recognition result after time smoothing.

[0052] The beneficial effects of this invention are:

[0053] 1. Improved the stability and accuracy of subsequent feature extraction, maintaining consistent image quality input under various complex environmental conditions, and reducing the impact of environmental changes on recognition accuracy;

[0054] 2. It overcomes the limitations of insufficient single feature representation capabilities, achieves accurate detection of small targets at long distances and large targets at close range, improves the accuracy of multi-scale target recognition, and provides a unified feature representation framework for obstacles of varying sizes in scenic roads.

[0055] 3. It achieves dual optimization of detection accuracy and computational efficiency, ensuring both the speed and accuracy of obstacle boundary detection and providing positioning capabilities under real-time requirements, thus providing richer and more accurate spatial information for subsequent path planning and obstacle avoidance decisions.

[0056] 4. Effectively suppressed false detection results at unreasonable locations, and improved the robustness of identification under statistical network conditions;

[0057] 5. It effectively solves the problem of duplicate recognition caused by overlapping detection boxes in traditional methods, avoids waste of computing resources and confusion in decision-making, and ensures that each real faulty object has a unique and optimal recognition result. It is particularly suitable for complex scenarios in local roads where tourists gather and vehicles mix, and improves the reliability and consistency of recognition results. Attached Figure Description

[0058] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0059] Figure 1 This is a flowchart illustrating the obstacle recognition method for autonomous driving in complex scenic road scenarios as shown in this invention.

[0060] Figure 2 This is a diagram showing the application interface of the autonomous driving vehicle in the scenic area on the scenic road map, as described in this invention. Detailed Implementation

[0061] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0062] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort should fall within the scope of protection of this invention.

[0063] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0064] According to an embodiment of the present invention, in combination Figure 1 The flowchart and Figure 2 The scenic area map application interface shown illustrates an obstacle recognition method for autonomous driving in complex scenic road scenarios, which specifically includes the following steps:

[0065] S1. Acquire original images of the scenic area roads and perform noise suppression, brightness equalization, and contrast enhancement to obtain enhanced images with uniform resolution. Note the following in this step:

[0066] S1.1 Systematically collect images of scenic area roads under various meteorological conditions, including at least sunny, cloudy, rainy, foggy, and nighttime conditions. For each meteorological condition, images are collected sequentially in the directions of facing the sun, facing away from the sun, and side lighting. The collected images of scenic area roads should at least cover the main roads, branch roads, parking lots, and special functional areas.

[0067] In an optional implementation, image acquisition adopts a layered acquisition strategy, such as first acquiring images in stages according to meteorological conditions, wherein:

[0068] For data collection under clear or partly cloudy conditions, ensure a sufficient and stable natural data source.

[0069] For cloudy weather data collection, choose a day with partial cloud cover but no precipitation, where the light is relatively uniform but low in intensity.

[0070] Rainy day conditions were selected for data collection under light to moderate rain conditions, where there was water accumulation and reflection on the water surface.

[0071] Data collection under foggy conditions was conducted in light to moderate fog, with visibility ranging from 50 to 200 meters.

[0072] Nighttime condition data collection was conducted under artificial lighting conditions, including light sources such as streetlights and vehicle lights;

[0073] Furthermore, under the aforementioned meteorological conditions, azimuth data was acquired using a three-dimensional display:

[0074] Direct sunlight capture (e.g., 0°±15°): The camera is positioned towards the sun to simulate a backlight driving scenario;

[0075] Cameras facing away from the sun (e.g., 180°±15°): The camera is positioned away from the sun to simulate a driving scenario with the sun behind it;

[0076] Side lighting acquisition (e.g., 90°±15° and 270°±15°): The camera is at a perpendicular angle to the sun to simulate a side lighting driving scenario;

[0077] It should be noted that the driving scene data collection adopts a spatial hierarchical strategy, in which:

[0078] The main roads of the site are ≥6 meters wide, including straight sections, curved sections, and uphill and downhill sections;

[0079] Local side road routes: one-way or pedestrian access roads with a width of 3 to 6 meters, including roundabouts and side roads;

[0080] Open-air parking lots and semi-enclosed parking lots;

[0081] Special functional areas: high-density areas such as tourist distribution areas, parking lot entrances, and surrounding service facilities;

[0082] For example, light intensity is defined as: >50,000 lux on sunny days, 10,000–50,000 lux on cloudy days, and <10,000 lux on rainy days; while the range is 50–200 meters on foggy days, >200 meters on rainy days, and >1,000 meters on sunny / cloudy days; the standards for image acquisition include: resolution ≥720p, frame rate ≥20fps, and dynamic range ≥8bit.

[0083] It should be noted that the images of scenic roads under different meteorological conditions include, but are not limited to, strong light images under sunny and high illumination conditions, weak light images under cloudy and low illumination conditions, reflective images under dry rainy conditions, blurred images under foggy conditions with reduced visibility, as well as obvious images under daytime illumination conditions, supplementary lighting images under twilight light changes, and low illumination images under nighttime artificial lighting conditions.

[0084] S1.2. Perform indirect feedback processing on the acquired raw images by setting the standard deviation σ of the spatial distance weighting function. d The standard deviation σ of the pixel value variance weighting function is for a range of 5 to 9 pixels. r For values ​​ranging from 20 to 80 pixels, a 9×9 indirect feedback kernel is used to effectively increase random noise while preserving edge details;

[0085] In an optional implementation, performing indirect feedback processing includes: calculating the neighborhood spatial distance weight for each pixel, then calculating the pixel value similarity weight, and finally obtaining the adsorption result by weighted averaging.

[0086] S1.3. An adaptive histogram equalization method is adopted to divide the input image into 8×8 uniform grid blocks. The variance histogram of each sub-block is calculated independently and a shearing threshold of 2.0 to 4.0 is set. The block effect is eliminated by bilinear interpolation, and sparse fusion is used to optimize the local and global regions with non-distributed mean values.

[0087] S1.4. Based on the global brightness statistics of the image, dynamically calculate the gamma correction coefficient γ = 1 + α × (128 - μ) / 128 to obtain the gamma value, where μ is the average brightness of the image and α is an adjustment factor, which can be converted into the pixel output value I. out =255×(I in / 255) γ This makes the enhanced image gradually more balanced overall, where I in Input pixel values;

[0088] S1.5. The processed image is uniformly adjusted to a preset standard resolution (e.g., 1280×720) using bicubic interpolation. Image deformation and boundary effects are avoided by cropping strategy and mirror filling method. Finally, the enhanced image with uniform resolution is used as the input data for subsequent feature extraction.

[0089] S2. Input the enhanced image into a multi-scale feature extraction network to extract texture, shape, and semantic features respectively, and then fuse them through a feature pyramid network to generate a multi-scale feature map. Note that the following points should be noted in this step:

[0090] S2.1 Construct a feature extraction network consisting of shallow layer groups, medium layer groups, and deep layer groups. The shallow layer group contains 16 layers for extracting edge image texture features, the medium layer group contains 16 layers for extracting shape image structure features, and the deep layer group contains 18 layers for extracting image reconstruction features.

[0091] S2.2. The input enhanced image is subjected to hierarchical feature encoding. The shallow layer group uses Gabor filter bank and local binary operator to extract multi-directional and multi-frequency texture features, generating a 256-channel texture feature map; the middle layer group uses Canny edge detection, Sobel operator and Laplacian operator to extract curvature and shape contour, generating a 512-channel shape feature map; the deep layer group uses hole structure and spatial pooling module to extract multi-scale semantic information, generating a 1024-channel feature map.

[0092] S2.3. A top-down feature pyramid network is used to fuse different features. Vertical feature matching is performed through a 1×1 shape of side connections, and upsampling is performed using nearest neighbor interpolation, according to formula P. i =Conv 1×1 (C i )+Upsampie(P i+1 Layer-by-layer feature fusion is performed, where P i Let C be the pyramid feature of the i-th layer. i For the extracted features of the i-th layer, Conv 1×1 For 1×1 operations, Upsampie is an upsampling operation;

[0093] S2.4. Upsampling is performed using transpose error and downsampling is performed using max pooling. Feature maps of different scales are uniformly adjusted to a resolution of 320×180. Bilinear interpolation is used to ensure the spatial correspondence and feature accuracy of the feature maps.

[0094] S2.5. The end texture, shape features and contour features are stitched together according to the channel dimension to form a multi-dimensional feature map with 1792 channels. The feature dimension is reduced to 512 dimensions by using principal component analysis. The shape features are compressed by channel attention mechanism. The output is a standardized multi-dimensional feature map containing multi-dimensional feature information as the input data for the subsequent multi-branch decoder.

[0095] S3. Input the multi-scale feature map into the multi-branch decoder to obtain the obstacle region segmentation result, obstacle location bounding box coordinates, and obstacle type classification probability. Note that the following should be noted in this step:

[0096] The multi-dimensional feature map is input into a multi-branch decoder, and the obstacle region segmentation result, obstacle location result, and obstacle type classification result are obtained through three decoding branches, respectively:

[0097] The first decoding branch is the obstacle region segmentation branch, which analyzes the image resolution on the multi-dimensional feature map through gradient inverse matrix operation, and determines the obstacle region attribution of each pixel through a pixel-level classifier to generate a binary obstacle region segmentation mask.

[0098] The second decoding branch is the obstacle location branch, which converts the multi-dimensional feature map into fixed-length features through global average pooling and then uses a multi-layer connection network to regress and predict the bounding box coordinates of the obstacle.

[0099] The third decoding branch is the obstacle classification type branch, which extracts global signal features through feature aggregation operations and then outputs the probability distribution of each obstacle through the classifier network.

[0100] As an example, the obstacle region segmentation result includes: describing the boundary information of the obstacle region with pixel-level precision, with each pixel corresponding to a binary label to indicate whether the pixel belongs to the obstacle region, forming a complete obstacle segmentation mask map.

[0101] As an example, the obstacle location results include: spatial location information of the obstacle in the form of bounding boxes, each bounding box being determined by the coordinates of the upper left corner and the lower right corner, indicating the exact location range of the obstacle in the image, and also including the confidence score of the bounding boxes to indicate the reliability of the location results;

[0102] As an example, the obstacle type classification results include: classification labels for common obstacle types in scenic roads, including at least the categories of pedestrians, vehicles, park maintenance equipment, and fixed landscape facilities. Each category label corresponds to a probability value, which is used to indicate the likelihood of the obstacle belonging to a certain category.

[0103] It should be noted that the multi-branch decoder in this embodiment consists of three decoding branches, which are used for obstacle region segmentation, obstacle location localization, and obstacle type classification, respectively.

[0104] S4. Combining road geometry, weather, and traffic light status information with surrounding environmental data, an attention mechanism is used to weight the obstacle region segmentation results, obstacle location bounding box coordinates, and obstacle type classification probabilities to generate an initial obstacle saliency map. Note that the following points should be noted in this step:

[0105] Lane lines are detected and road geometry information is calculated using Hough transform and RANSAC algorithm. Green areas, buildings and water features are identified using image segmentation network. At the same time, signal condition information such as traffic light status and traffic signs are identified using deep learning object detection algorithm. Signal strength, visibility and sky status information are obtained through image perception analysis, enhancement evaluation and sky segmentation network.

[0106] A multimodal fusion framework is constructed to perform modal transformation on the obstacle recognition results from step S3. Specifically, the obstacle region segmentation results are converted into spatial saliency features through pixel-level weight calculation and spatial convolutional encoding, the obstacle location bounding boxes are converted into location saliency features through distance weighting function and angle offset evaluation, and the obstacles are converted into obstacle type classification probabilities through category mapping and semantic context enhancement.

[0107] The multi-head attention mechanism of the Transformer computing architecture is used to calculate the correlation weights between different modal features. Spatial, positional and semantic features are respectively used as key-value triples. Eight attention heads are used to process different feature subspaces, and the fusion weights of each modal feature are adaptively adjusted according to the extracted environmental context information.

[0108] Furthermore, the feature fusion weights are adjusted adaptively to the environment based on road morphology, meteorological conditions and signal status information. Spatial salience weights are enhanced under terrain conditions, location salience weights are enhanced on roads or curved slopes, and semantic salience weights are enhanced in densely landscaped areas. Real-time adaptive weights are achieved through a dynamic weight update mechanism.

[0109] The weighted fusion operation is S(x,y)=w1×F spatial (x,y)+w2×F position (x,y)+w3×F semantic (x,y) merges spatial saliency features, positional saliency features, and semantic saliency features into a unified continuous saliency feature map, and then uses weighted threshold segmentation and morphological processing to discretize the continuous saliency map into a multi-level obstacle saliency fusion feature map;

[0110] Where S(x,y) is the significance value of the position, F spatial For spatial saliency feature maps, F position For the location saliency feature map, F semantic is the semantic saliency feature map, and w1, w2, and w3 are the fusion weights of the spatial saliency feature map, the positional saliency feature map, and the semantic saliency feature map, respectively.

[0111] S5. Based on the preset obstacle type priority level, detect and process the spatial overlap recognition results in the initial obstacle salience map, retaining high-priority targets and suppressing low-priority targets to obtain the obstacle recognition results after conflict resolution. Note that the following should be noted in this step:

[0112] S5.1 The preset priority levels for obstacle types include: setting pedestrians as the first priority and assigning a weight coefficient of 1, setting vehicles as the second priority and assigning a weight coefficient of 0.8, setting park maintenance equipment as the third priority and assigning an equipment weight coefficient of 0.6, and setting fixed landscape facilities as the fourth priority and assigning a weight coefficient of 0.4.

[0113] S5.2 Perform spatial overlap detection on multiple obstacle recognition results from step S4. Calculate the overlap degree of the bounding boxes using the intersection-union ratio (IoU) = Area(A∩B) / Area(A∪B). When IoU > 0.3, it is considered spatial intersection overlap. Establish the overlap relationship of the comprehensive collision detection matrix.

[0114] Where Area(·) is the area function, A is the bounding box of the candidate obstacle, and B is the bounding box of the adjacent obstacle;

[0115] S5.3 When multiple obstacles of different types are detected in the same space, an improved priority non-maximum suppression algorithm is used for conflict resolution, and the results are calculated based on the comprehensive score. combined =confidence × priority weight The detection results are sorted, retaining complete information for high-priority targets, and correcting low-priority targets using the formula NewBBox. high =WeightedAverage(BBox) high BBox low overlap ratio Update the location information of high-priority targets;

[0116] Where confidence is the confidence level, and priority is the priority level. weight As a priority weight, NewBBox high The updated high-priority target bounding box, WeightedAverage(·) is the weighted average function, BBox high BBox represents the original bounding box of high-priority obstacles (such as first and second priority obstacles). low For low-priority obstacles (such as fourth priority), the original bounding box overlaps. ratio This represents the overlap weight ratio;

[0117] S5.4 The hierarchical conflict resolution strategy implements a three-level conflict resolution process. For the same overlapping targets, the detection result with the highest confidence is retained and feature information is fused. For different overlapping targets, high-priority targets are completely retained and low-priority targets are overlapped according to priority weights. For some overlapping targets, the region allocation weight is allocated according to the overlap share and the bounding box is adjusted to avoid conflict.

[0118] S5.5 Finally, generate obstacle recognition results after conflict resolution, including complete attribute information such as the obstacle's unique identifier, category information, priority level, confidence level, bounding box coordinates, segmentation mask, centroid coordinates, and time, forming obstacle recognition results that include location, type, and confidence assessment.

[0119] S6. Perform spatial consistency verification between the obstacle recognition results and the road path segmentation map, and eliminate inter-frame jumps based on time series smoothing processing. The final output includes the recognition results containing obstacle type, location, segmentation mask, and confidence score. Note that the following points should be noted in this step:

[0120] S6.1. The DeepLabV3+ deep learning segmentation network is used to segment the scenic roads into functional areas. The roads are divided into four functional areas: passable road area, tourist walking area, green landscape area and building area. Each area is assigned a unique label category and weight parameters. A pixel-level road path segmentation map is generated through a decoder structure that integrates multiple high-precision correction modules and low- and high-level features.

[0121] S6.2 Establish a matching rule base between obstacle types and road functional areas, and perform spatial consistency verification on the obstacle identification results from step S5, using spatial consistency scoring. The scoring of obstacles and functional areas is calculated, and a hierarchical verification strategy is adopted, including coarse-grained centroid verification, fine-grained overlap calculation and boundary cross-checking, with 0.6 defined as the verification threshold.

[0122] in, The overlap ratio, For regional weights, The target weight;

[0123] S6.3. Based on the consistency score, inconsistent results are classified into three levels of deviation: minor, moderate, and severe. Minor deviations (0.4 ≤ score < 0.6) are subject to location bias and confidence decay. Moderate deviations (0.2 ≤ score < 0.4) are reclassified and reassigned to different regions. Severe deviations (score < 0.2) are directly marked as outliers and removed using the location deletion formula New. Position =Original Position+α×(Nearest_Valid_Position-Original Position ) and confidence level calibration formula New Confidence =Original Confidence ×(1-β×Consistency Score Perform precise proofreading;

[0124] Among them, New Position The original text refers to the obstacle position coordinates after spatial consistency calibration. Position To calibrate the original detected position of the obstacle, α is the calibration coefficient, Nearest_Valid_Position is the center position of the nearest reasonable functional area to the original position, and New Confidence The obstacle detection confidence level after consistency calibration. Confidence To calibrate the initial detection confidence of the obstacle, β is the confidence attenuation coefficient, 1-β×Consistency Score The proportion to retain for confidence level;

[0125] S6.4 Establish a Kalman filter system for obstacle tracking. Define the state information of the obstacle as a six-dimensional state including the horizontal and vertical coordinates of the center position, the horizontal and vertical movement speed, and the width and height of the bounding box. Describe the state relationship changes of the obstacle between adjacent frames through a uniform motion model. Preset the state change matrix to reflect the linear change law of position with speed and velocity.

[0126] The Hungarian algorithm is used to achieve optimal association matching of obstacles between frames. The correspondence is established by calculating the comprehensive similarity between the detection result of the current frame and the tracked target of the previous frame. The calculation of the comprehensive similarity takes into account the matching degree of position distance, size similarity and appearance features. The association distance threshold is set to 50 pixels. When the detection result is more than 50 pixels away from the tracked target, it is identified as a newly appeared obstacle target. Tracked targets that are successfully associated for 3 consecutive steps are determined to be obstacles that have disappeared and are deleted from the tracking list.

[0127] In the processing of each frame, the prediction step is first performed to predict the obstacle state of the current frame based on the state estimation and state transition model of the previous frame. Then, the update step is performed to calculate the Kalman gain matrix to balance the weights of the predicted value and the initial value of the current frame. The optimal state estimate of the current frame is obtained by weighting the predicted state and the initial residual.

[0128] The obstacle recognition result of the current frame is fed back by the estimation result for instantaneous localization. The filter smooths abnormal position jumps based on the historical movement trajectory of the obstacle. When the detected position change exceeds the reasonable movement range, the weight of the current start will be reduced. At the same time, the position reliability score of obstacle detection is dynamically adjusted according to the state estimation in the simulation process.

[0129] S6.5 Establish a standardized data structure containing complete obstacle information, specifically including the obstacle's unique identifier, category information, spatial information, motion information, quality information, and processing information. For example, the category information includes the obstacle's category number, category name, priority level, and detection confidence; the spatial information includes the bounding box coordinates, center position, partition code array, and pixel area; the motion information includes velocity, predicted position, and tracking identifier; the quality information includes spatial consistency score, correction smoothness score, and overall quality score; and the processing information includes Boolean flags indicating the conflict resolution solution status, spatial correction status, and smoothing status.

[0130] Furthermore, a minimum confidence threshold of 0.3 is set, and recognition results with confidence levels below this threshold are filtered to ensure output quality. The maximum output per frame is 1 to balance the limitation of defect detection and computational efficiency. A comprehensive quality scoring mechanism is established, which weights and fuses spatial consistency score, smoothness score, and detection confidence with weights of 0.4, 0.3, and 0.3, respectively. The recognition results are sorted according to the comprehensive score, and high-quality detection targets are prioritized.

[0131] It should be noted that the output results support multiple standard data formats to ensure that the interface meets the integration needs of different application scenarios, such as JSON and XML formats. All output formats maintain the consistency and integrity of the data structure to ensure that downstream systems can correctly parse and use the recognition results.

[0132] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for obstacle recognition in autonomous driving in complex scenic road scenarios, characterized in that, include: Original images of scenic area roads were acquired and noise suppression, brightness equalization, and contrast enhancement were performed to obtain enhanced images with uniform resolution. The enhanced image is input into a multi-scale feature extraction network to extract texture features, shape features, and semantic features, which are then fused through a feature pyramid network to generate a multi-dimensional feature map. The multi-dimensional feature map is input into the multi-branch decoder to obtain the obstacle region segmentation result, the obstacle location bounding box coordinates, and the obstacle type classification probability, respectively. Combining road geometry, weather and traffic light status information, an attention mechanism is used to perform feature weighting on the obstacle region segmentation results, obstacle location bounding box coordinates, and obstacle type classification probabilities to generate an obstacle saliency feature map. Based on the preset obstacle type priority level, the recognition results of spatial overlap in the obstacle salience feature map are detected and processed, high-priority targets are retained and low-priority targets are suppressed to obtain the obstacle recognition results after conflict resolution. The obstacle recognition results are spatially consistent with the road path segmentation map, and inter-frame jumps are eliminated based on time series smoothing. The final output includes the recognition results containing obstacle type, location, segmentation mask and confidence level. Obtaining the obstacle saliency feature map includes: extracting surrounding environmental information of scenic roads, including road geometry information, surrounding road landscape layout information, signal condition information, and weather condition information; constructing a multimodal fusion framework to convert obstacle region segmentation results into spatial saliency features, obstacle location results into location saliency features, and obstacle type classification results into semantic saliency features; calculating the correlation weights between different modal features using an attention mechanism, and adaptively adjusting the weights of each modal feature fusion according to the regional path context information; and merging spatial saliency features, location saliency features, and semantic saliency features into a unified discrete obstacle saliency feature map through a weighted fusion operation. The obstacle location and type classification results in the obstacle saliency feature map are corrected using an obstacle type priority strategy. This includes: establishing an obstacle type priority level for the road scene, where pedestrians are set as the first priority, vehicles as the second priority, park maintenance equipment as the third priority, and fixed landscape facilities as the fourth priority; performing conflict detection on multiple obstacle identification results with spatially overlapping obstacle saliency features, and filtering the results according to the priority strategy when multiple different types of obstacles are detected in the same space; and transcribing the identification results of the fourth priority obstacles using a non-maximum transcription factor, while retaining the location and type classification results of the first priority obstacles, to obtain the obstacle identification results after conflict resolution. Combining the scenic area road path segmentation information, the corrected recognition results are checked for consistency, and the final obstacle recognition results are output. This includes: segmenting the scenic area roads into road path regions using a segmentation network, including passable road areas, pedestrian areas, green landscape areas, and building areas, generating a road path segmentation map; verifying the spatial consistency between the modified obstacle recognition results and the road path segmentation map, checking whether the recognized location of the obstacle matches the road functional area that should appear; correcting recognition results that do not meet the spatial consistency verification; and eliminating inter-frame jumps based on time series smoothing processing, finally outputting a complete obstacle recognition result containing obstacle type labels, location coordinates, region segmentation masks, and confidence scores.

2. The obstacle recognition method for autonomous driving in complex scenic road scenarios according to claim 1, characterized in that, The enhanced road image includes: Bilateral filtering is performed on the original image of the scenic road to suppress random noise while preserving edge details; An adaptive histogram equalization method is used to locally enhance the contrast in areas with uneven brightness distribution. The gamma coefficient is dynamically adjusted based on global brightness statistics to make the overall brightness of the enhanced image more balanced.

3. The obstacle recognition method for autonomous driving in complex scenic road scenarios according to claim 1 or 2, characterized in that, The acquisition of the original images of the scenic area roads meets the following environmental requirements, including: Images of scenic area roads were collected under sunny, cloudy, rainy, foggy, and nighttime conditions. For each batch of data collected under each weather condition, there should be at least three lighting orientations: facing the sun directly, facing away from the sun, and side lighting. The scenic area road images include at least images of main roads within the scenic area, images of secondary roads within the scenic area, images of parking lots within the scenic area, and images of scenic area roads under different weather conditions, wherein: The scenic road images under different meteorological conditions include, but are not limited to, strong light images under sunny high illumination conditions, weak light images under cloudy low illumination conditions, reflective images under rainy and dry conditions, blurred images under foggy conditions with reduced visibility, as well as clear images under daytime illumination conditions, supplementary lighting images under twilight light changes, and low illumination images under nighttime artificial lighting conditions.

4. The obstacle recognition method for autonomous driving in complex scenic road scenarios according to claim 1, characterized in that, Generating the multi-dimensional feature map includes: The enhanced road image is used as input data, and the shallow layer group, middle layer group and deep layer group in the multi-dimensional feature extraction network are used to extract features from the input image layer by layer. The shallow layer group extracts image edge texture features, the middle layer group extracts image shape and structure features, and the deep layer group extracts image semantic content features. The feature pyramid network fuses features from different layers and unifies the feature map size through upsampling and downsampling operations, ultimately outputting a multi-dimensional feature map containing multi-dimensional feature information.

5. The obstacle recognition method for autonomous driving in complex scenic road scenarios according to claim 4, characterized in that, The multi-dimensional feature map is input into a multi-branch decoder, and the obstacle region segmentation result, obstacle location result, and obstacle type classification result are obtained through three decoding branches, respectively: The first decoding branch is the obstacle region segmentation branch, which analyzes the image resolution on the multi-dimensional feature map through gradient inverse matrix operation, and determines the obstacle region attribution of each pixel through a pixel-level classifier to generate a binary obstacle region segmentation mask. The second decoding branch is the obstacle location branch, which converts the multi-dimensional feature map into fixed-length features through global average pooling operation for analysis, and then uses a multi-layer connection network to regress and predict the bounding box coordinate information of the obstacle. The third decoding branch is the obstacle classification type branch, which extracts global signal features through feature aggregation operations and then outputs the probability distribution of each obstacle through the classifier network.

6. The obstacle recognition method for autonomous driving in complex scenic road scenarios according to claim 5, characterized in that, The obstacle region segmentation result includes: describing the boundary information of the obstacle region with pixel-level precision, with each pixel corresponding to a binary label to indicate whether the pixel belongs to the obstacle region, forming a complete obstacle segmentation mask map; The obstacle location results include: obstacle spatial location information represented by bounding boxes, each bounding box being determined by the coordinates of the upper left corner and the lower right corner, indicating the accurate location range of the obstacle in the image, and also including the confidence score of the bounding boxes, which is used to indicate the reliability of the location results; The obstacle type classification results include: classification labels for common obstacle types in scenic area roads, including at least pedestrian categories, traffic vehicle categories, park maintenance equipment categories, and fixed landscape facility categories. Each category label corresponds to a probability value, which is used to indicate the likelihood of the obstacle belonging to a certain category.

7. The obstacle recognition method for autonomous driving in complex scenic road scenarios according to claim 1, characterized in that, The time series smoothing process specifically includes: Based on the Kalman filter algorithm, the position and velocity of obstacles in adjacent frames are estimated; The estimation results are fed back into the current frame recognition results to reduce instantaneous localization jitter and update the confidence level; Output the final obstacle recognition result after time smoothing.

Images