Automatic driving obstacle recognition method for complex scenic spot road scene

Abstract

The invention discloses an automatic driving obstacle recognition method for a complex scenic spot road scene. The method comprises the following steps: acquiring an original image of a scenic spot road; inputting a multi-scale feature extraction network to generate a multi-scale feature map; inputting the image into a multi-branch decoder to respectively obtain an obstacle region segmentation result, an obstacle position bounding box coordinate and an obstacle type classification probability; in combination with surrounding environment information of road geometrical morphology, weather and signal lamp states, feature weighting is carried out on the surrounding environment information by adopting an attention mechanism, and an initial obstacle saliency map is generated; according to a preset obstacle type priority level, detecting and processing an identification result of spatial overlapping in the image to obtain an obstacle identification result after conflict resolution; and performing spatial consistency proofreading on the road path segmentation map and the road path segmentation map, eliminating inter-frame jump based on time sequence smoothing, and finally outputting an identification result. According to the invention, the problem of high-precision and real-time obstacle recognition for complex scenic spot road scenes is effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and intelligent traffic in scenic areas, and particularly relates to an automatic driving obstacle recognition method for complex scenic road scenes. BACKGROUND

[0002] With the rapid development of artificial intelligence and computer vision technology, the environmental perception and obstacle recognition technology in automatic driving systems has become the core research direction in the field of intelligent transportation. Traditional obstacle recognition methods mainly rely on single sensor data processing, obtaining environmental information through radar, lidar or camera devices, and using basic image processing algorithms for target detection and classification. However, with the rise of deep learning technology, visual perception methods based on convolutional neural networks have gradually become mainstream, especially in multi-target recognition tasks in complex road environments, showing significant advantages. Currently, the obstacle recognition framework usually adopts an end-to-end deep learning architecture, combining feature extraction, target detection and semantic segmentation, etc. multiple sub-tasks to achieve accurate recognition of vehicles, pedestrians, traffic signs and other multi-class targets in road scenes. At the same time, the application of multi-modal fusion technology further improves the robustness of the system in adverse weather conditions, and the introduction of time sequence information effectively alleviates the misrecognition problem in single-frame detection.

[0003] CN115230694A discloses an automatic driving vehicle obstacle recognition method, which realizes obstacle screening through Frenet coordinate system conversion. However, this method is mainly designed for structured roads and lacks adaptability in complex terrains, diverse obstacle types and dynamic environmental changes in scenic road environments, especially when dealing with non-standard road geometries, it is prone to misjudgment.

[0004] CN118587683A discloses an obstacle recognition method based on a generative adversarial network, which improves the model generalization ability through data enhancement. However, its feature extraction process lacks specific consideration of special environmental factors in scenic road environments, such as complex lighting conditions, variable weather states and mobile obstacles specific to scenic areas (tourists, sightseeing vehicles, etc.), resulting in insufficient recognition accuracy and poor real-time performance in actual applications.

[0005] In addition, the prior art generally lacks an effective fusion mechanism for multi-scale features, making it difficult to simultaneously consider accurate detection of small targets at a long distance and complete segmentation of large targets at a short distance, and lacking an effective conflict resolution strategy when processing spatially overlapping targets, which is prone to false detection and missed detection phenomena; therefore, the existing automatic driving obstacle recognition technology has the problems of insufficient adaptability to complex scenic road environments, limited multi-scale feature fusion capability, lack of use of environmental context information, and imperfect spatially overlapping target processing mechanism, and the present application mainly solves the problem of high-precision, real-time obstacle recognition for complex scenic road scenes. SUMMARY

[0006] This section is intended to summarize some aspects of the embodiments of the present application and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of the specification to avoid obscuring the purpose of this section, the abstract and the title, and such simplifications or omissions cannot be used to limit the scope of the present application.

[0007] In view of the above-mentioned existing problems, the present application is proposed.

[0008] To solve the above technical problems, the present application provides the following technical solutions: collecting scenic road original images and performing noise suppression, brightness equalization and contrast enhancement to obtain enhanced images with uniform resolution;

[0009] The enhanced images are input into a multi-scale feature extraction network to extract texture features, shape features and semantic features respectively, and the features are fused through a feature pyramid network to generate multi-scale feature maps;

[0010] The multi-scale feature maps are input into a multi-branch decoder to obtain obstacle region segmentation results, obstacle position bounding box coordinates and obstacle type classification probabilities respectively;

[0011] The surrounding environment information of the road geometry, weather and signal light state is combined, and the attention mechanism is used to weight the feature of the obstacle region segmentation results, the obstacle position bounding box coordinates and the obstacle type classification probabilities to generate an initial obstacle saliency map;

[0012] According to the preset obstacle type priority level, the spatially overlapping recognition results in the initial obstacle saliency map are detected and processed, the high-priority targets are retained, and the low-priority targets are suppressed to obtain the obstacle recognition results after conflict resolution;

[0013] The obstacle recognition results and the road path segmentation map are spatially consistent, and based on time series smoothing processing to eliminate inter-frame jumps, finally output the recognition results containing obstacle type, position, segmentation mask and confidence.

[0014] As a preferred scheme of the automatic driving obstacle recognition method for a complex scenic spot road scene, the obtained enhanced road image comprises:

[0015] The bilateral filtering is performed on the scenic spot road original image to suppress random noise while retaining edge details.

[0016] The adaptive histogram equalization method is used to locally improve the contrast of the region with uneven brightness distribution.

[0017] The global brightness statistics are used to dynamically adjust the gamma coefficient, so that the overall brightness of the enhanced image tends to be balanced.

[0018] As a preferred scheme of the automatic driving obstacle recognition method for a complex scenic spot road scene, the collection of the scenic spot road original image meets the following environmental requirements, comprising:

[0019] The scenic spot road images are collected under sunny, cloudy, rainy, foggy and night conditions.

[0020] Each batch of collection under each weather condition includes at least three light directions: facing the sun, away from the sun and side light.

[0021] The scenic spot road images include at least the main road images in the scenic spot, the branch path images in the scenic spot, the scenic spot parking lot and the scenic spot road images under different weather conditions, wherein:

[0022] The scenic spot road images under different weather conditions include but are not limited to strong light images under sunny high irradiation conditions, weak light images under cloudy low light conditions, reflective images under rainy dry conditions, blurred images under foggy visibility reduction conditions, and obvious images under daytime light conditions, supplementary light images under the changing light of dusk, and low-illumination images under the condition of night artificial lighting.

[0023] As a preferred scheme of the automatic driving obstacle recognition method for a complex scenic spot road scene, the generation of the multi-dimensional feature map comprises:

[0024] The enhanced road image is used as input data, and the input image is extracted layer by layer through the shallow layer group, the middle layer group and the deep layer group in the multi-dimensional feature extraction network.

[0025] Among them, the shallow layer group extracts image edge texture features, the middle layer group extracts image shape structure features, and the deep layer group extracts image semantic content features.

[0026] The features of different layer groups are fused through a feature pyramid network, and the feature map sizes are unified through upsampling and downsampling operations, and finally a multi-dimensional feature map containing multi-dimensional feature information is output.

[0027] As a preferred scheme of the automatic driving obstacle recognition method for a complex scenic spot road scene, the multi-dimensional feature map is input into a multi-branch decoder, and obstacle region segmentation results, obstacle position positioning results and obstacle type classification results are obtained through three decoding branches, wherein:

[0028] The first decoding branch is an obstacle region segmentation branch, which analyzes the image resolution on the multi-dimensional feature map through a gradient inverse matrix operation, and determines the obstacle region attribution of each pixel point through a pixel-level classifier to generate a binary obstacle region segmentation mask;

[0029] The second decoding branch is an obstacle position positioning branch, which converts the multi-dimensional feature map into a fixed-length feature for analysis through a global average pooling operation, and then predicts the boundary box coordinate information of the obstacle through a multi-layer connection network regression;

[0030] The third decoding branch is an obstacle classification type branch, which extracts global signal features through a feature aggregation operation, and then outputs the probability distribution of each obstacle through a classifier network.

[0031] As a preferred scheme of the automatic driving obstacle recognition method for a complex scenic spot road scene, the obstacle region segmentation result includes: describing the obstacle region boundary information at the pixel level, each pixel point corresponds to a binary label, which is used to indicate whether the pixel point belongs to the obstacle region, forming a complete obstacle segmentation mask map;

[0032] The obstacle position positioning result includes: obstacle spatial position information in the form of a boundary box, each boundary box is determined by the upper left corner coordinate and the lower right corner coordinate, and the accurate position range of the obstacle in the image is calibrated, and the confidence score of the boundary box is included, which is used to indicate the reliability of the positioning result;

[0033] The obstacle type classification result includes: classification labels of common obstacle types in the scenic spot road, at least including a tourist pedestrian class, a traffic vehicle class, a park maintenance device class and a fixed landscape facility class, each class label corresponds to a probability value, which is used to indicate the possibility size of the obstacle belonging to the class.

[0034] As a preferred scheme of the automatic driving obstacle recognition method for a complex scenic spot road scene, the obtained obstacle saliency feature map includes:

[0035] Extract the surrounding environment information of the scenic spot road, including road geometric structure information, surrounding road landscape layout information, signal condition information and weather condition information;

[0036] A multi-modal fusion framework is constructed to convert the obstacle region segmentation result into a spatial saliency feature, convert the obstacle position positioning result into a position saliency feature, and convert the obstacle type classification result into a semantic saliency feature;

[0037] The correlation weight between different modal features is calculated by using an attention mechanism, and the weight fusion of each modal feature is adaptively adjusted according to the regional path context environment information;

[0038] The spatial saliency feature, the position saliency feature and the semantic saliency feature are combined into a unified discrete obstacle saliency feature map through a weighted fusion operation.

[0039] As a preferred scheme of the automatic driving obstacle recognition method for the complex scenic spot road scene, the obstacle position positioning result and the obstacle type classification result in the obstacle saliency feature map are corrected by using an obstacle type priority level strategy, including:

[0040] The obstacle type priority level of the road scene is established for the obstacle type, wherein the tourist pedestrian is set as the first priority, the traffic vehicle is set as the second priority, the park maintenance equipment is set as the third priority, and the fixed landscape facility is set as the fourth priority;

[0041] The conflict detection is performed on multiple obstacle recognition results with spatial overlap of the obstacle saliency feature, and when multiple obstacles of different types are detected in the same space, the result is screened according to the priority level strategy;

[0042] The recognition result of the fourth priority obstacle is transcribed by using a non-maximum transcription factor, the position positioning result and the type classification result of the first priority obstacle are retained, and the conflict-resolved obstacle recognition result is obtained.

[0043] As a preferred scheme of the automatic driving obstacle recognition method for the complex scenic spot road scene, the corrected recognition result is subjected to consistency checking in combination with the scenic spot road path segmentation information, and the scenic spot obstacle recognition result is finally output, including:

[0044] The road path region segmentation is performed on the scenic spot road by using a segmentation network, a passable road region, a tourist walking region, a green landscape region and a building region are generated, and a road path segmentation map is generated;

[0045] The modified obstacle recognition result is subjected to spatial consistency verification with the road path segmentation graph, to check whether the recognition position of the obstacle matches the road function area that should appear;

[0046] The recognition result that does not conform to the spatial consistency verification is corrected;

[0047] And based on time series smoothing processing, eliminate inter-frame jump, finally output complete obstacle recognition result containing obstacle type label, position coordinates, region segmentation mask and confidence score.

[0048] As a preferred scheme of the complex scenic road scene-oriented autonomous driving obstacle recognition method of the application, the time series smoothing processing specifically includes:

[0049] Based on the Kalman filtering algorithm, the obstacle position and speed of adjacent frames are estimated;

[0050] The estimation result is fed back to the current frame recognition result to weaken the instantaneous positioning jitter and update the confidence;

[0051] Output the final obstacle recognition result after time smoothing.

[0052] The application has the following beneficial effects:

[0053] 1. The stability and accuracy of subsequent feature extraction are improved, consistent image quality input can be maintained under various complex environmental conditions, and the influence of environmental changes on recognition accuracy is reduced;

[0054] 2. The limitation of insufficient single feature representation ability is overcome, accurate detection of long-distance small targets and short-distance large targets is realized, the accuracy of multi-scale target recognition is improved, and a unified feature representation framework is provided for crowd obstacles with large size difference in scenic roads;

[0055] 3. The dual optimization of detection accuracy and calculation efficiency is realized, which not only ensures the rapid accuracy of obstacle boundary, but also provides positioning ability under real-time requirement, and provides more rich and accurate spatial information for subsequent path planning and obstacle avoidance decision;

[0056] 4. The false detection results of unreasonable positions are effectively suppressed, and the recognition robustness under statistical network conditions is improved;

[0057] 5. The problem of repeated recognition caused by multiple detection boxes overlapping in traditional methods is effectively solved, the waste of computing resources and the confusion of decision are avoided, and each real obstacle has a unique and optimal recognition result, which is especially suitable for complex scenes such as tourist gathering and vehicle mixed driving in scenic roads, and improves the reliability and consistency of the recognition result. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments description. Obviously, the drawings in the following description only show some of the embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings. Among them:

[0059] Figure 1 The flowchart of the automatic driving obstacle recognition method for complex scenic road scene shown in the present application;

[0060] Figure 2 The application program display interface of the scenic automatic driving vehicle in the scenic road map shown in the present application. DETAILED DESCRIPTION

[0061] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings in the specification. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments.

[0062] Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort should belong to the protection scope of the present application.

[0063] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited by the specific embodiments disclosed below.

[0064] According to the embodiments of the present application, combined with the flowchart shown in Figure 1 and the scenic map application interface shown in Figure 2 , an automatic driving obstacle recognition method for complex scenic road scene specifically includes the following steps:

[0065] S1, collecting scenic road original images and performing noise suppression, brightness equalization and contrast enhancement to obtain enhanced images with uniform resolution. It should be noted that in this step:

[0066] S1.1, systematically collecting images of scenic roads under various weather conditions, which at least include sunny day, cloudy day, rainy day, foggy day and night condition, and the image collection under each weather condition is sequentially directional collection according to the direction of directly facing the sunlight, facing away from the sunlight and side light point, and the collected scenic road images at least cover the main road, branch path, parking lot and special functional area;

[0067] In optional embodiments, the image collection adopts a hierarchical collection strategy, such as hierarchical collection according to weather conditions, wherein:

[0068] Clear day condition collection selects cloudless or few-cloud weather, ensuring that the natural light source is sufficient and stable;

[0069] Overcast day condition collection selects weather with cloud cover but no precipitation, the light is relatively uniform but the intensity is low;

[0070] Rainy day condition collection selects under light rain to moderate rain conditions, there is water reflection on the water surface;

[0071] Foggy day condition collection selects under light fog to moderate fog conditions, the visibility is within the range of 50-200 meters;

[0072] Night condition collection selects under artificial lighting conditions, including street lights, car lights and other light sources;

[0073] Further, under the aforementioned weather conditions, the three-dimensional display orientation collection is adopted:

[0074] Direct sunlight collection (e.g. 0°±15°): the camera is located in the direction of the sun, simulating the backlight driving scene;

[0075] Backlight collection (e.g. 180°±15°): the camera is away from the direction of the sun, simulating the frontlight driving scene;

[0076] Side light collection (e.g. 90°±15° and 270°±15°): the camera is perpendicular to the direction of the sun, simulating the sidelight driving scene;

[0077] It should be noted that the driving scene collection adopts a spatial hierarchical strategy, wherein:

[0078] Roads with a main road width of ≥6 meters, including straight sections, curved sections, uphill and downhill sections;

[0079] Residential branch paths: one-way or pedestrian roads with a width of 3-6 meters, including circular roads and branch roads;

[0080] Open parking lots and semi-closed parking lots;

[0081] Special functional areas: high pedestrian density areas such as tourist gathering areas, parking lot entrances, and surrounding service facilities;

[0082] For example, the light intensity is defined as: clear day >50000 lux, overcast day 10000-50000 lux, rainy day <10000 lux; and foggy day 50-200 meters, rainy day >200 meters, clear / overcast day >1000 meters; The standard for collecting images includes: resolution ≥720p, frame rate ≥20 fps, dynamic range ≥8 bits;

[0083] It should be noted that the scenic road images under different weather conditions include but are not limited to strong light images under sunny high irradiation conditions, weak light images under cloudy low light conditions, reflective images under rainy dry conditions, blurred images under foggy reduced visibility conditions, and bright images under daytime light conditions, supplementary light images under changing light at dusk, and low-illumination images under artificial lighting conditions at night.

[0084] S1.2, performing indirect feedback processing on the collected original image, setting the standard deviation σ of the spatial distance weight function d 5-9 pixel range, pixel value variance weight function standard deviation σ r 20-80 pixel value range, using a 9x9 indirect feedback kernel, while preserving edge details, effectively increasing random noise;

[0085] In an optional embodiment, performing indirect feedback processing includes: calculating the neighborhood spatial distance weight of each pixel point, then calculating the pixel value similarity weight, and finally obtaining the adsorption result by weighted average;

[0086] S1.3, using an adaptive histogram equalization method, dividing the input image into 8x8 uniform grid blocks, calculating the variance histogram for each sub-block independently and setting a clipping threshold of 2.0-4.0, eliminating block effects by bilinear interpolation, and realizing local-global optimization of sparse fusion to unevenly distributed mean areas;

[0087] S1.4, dynamically calculating the gamma correction coefficient γ = 1 + α × (128-μ) / 128 according to the global brightness statistical information of the image, where μ is the average brightness of the image and α is the adjustment factor, which can convert the pixel output value I out = 255 × (I in / 255) γ , so that the enhanced image as a whole gradually tends to be balanced, where I in is the input pixel value;

[0088] S1.5, using a bicubic interpolation method to uniformly adjust the processed image to a preset standard resolution (such as 1280x720), avoiding image distortion and boundary effects through cropping strategy and mirror padding method, and finally outputting an enhanced image of uniform resolution as input data for subsequent feature extraction.

[0089] S2, input the enhanced image into a multi-scale feature extraction network, respectively extract texture features, shape features and semantic features, and fuse through a feature pyramid network to generate a multi-scale feature map. It should be noted that this step is:

[0090] S2.1, construct a plurality of feature extraction networks comprising a shallow layer group, a middle layer group and a deep layer group, wherein the shallow layer group comprises 16 layers of layers for extracting edge image texture features, the middle layer group comprises 16 layers of layers for extracting shape image structure features, and the deep layer group comprises 18 layers of layers for extracting image reconstruction feature content;

[0091] S2.2, layered feature coding is performed on the input enhanced image, the shallow layer group adopts Gabor filter set and local binary operator to extract multi-directional and multi-frequency texture features, and generates a 256-channel texture feature map; the middle layer group adopts Canny edge detection, Sobel operator and Laplace operator to extract curvature and shape contour, and generates a 512-channel shape feature map; and the deep layer group adopts a hollow structure and a spatial pooling module to extract multi-scale semantic information, and generates a 1024-channel landmark feature map;

[0092] S2.3, a top-down feature pyramid network is used to fuse different features, a 1×1 shape is used for vertical matching of features through side connection, a nearest neighbor interpolation is used for up-sampling operation, and layer-by-feature fusion is performed according to the formula P i =Conv 1×1 (C i )+Upsampie(P i+1 ), wherein P i is the i-th layer of pyramid feature, C i is the i-th layer of extracted feature, Conv 1×1 is a 1×1 operation, and Upsampie is an up-sampling operation;

[0093] S2.4, up-sampling and down-sampling operations are performed by using transposed error and maximum pooling, features of different scales are uniformly adjusted to a resolution of 320×180, and spatial correspondence and feature accuracy of the feature map are ensured by a bilinear interpolation method;

[0094] S2.5, end texture, shape features and contour features are spliced according to channel dimensions to form a multi-dimensional feature map with a channel number of 1792, principal component analysis is used to reduce the feature dimension to 512, and shape feature compression is performed through a channel attention mechanism, and a standardized multi-dimensional feature map containing multi-dimensional feature information is output as input data of a subsequent multi-branch decoder.

[0095] S3, the multi-scale feature map is input into a multi-branch decoder to obtain an obstacle region segmentation result, an obstacle position bounding box coordinate and an obstacle type classification probability. It should be noted that in this step:

[0096] The multi-dimensional feature map is input into a multi-branch decoder, and obstacle region segmentation results, obstacle position positioning results and obstacle type classification results are obtained through three decoding branches, wherein:

[0097] The first decoding branch is an obstacle region segmentation branch, which analyzes the image resolution on the multi-dimensional feature map through a gradient inverse matrix operation, and determines the attribution of each pixel point to an obstacle region through a pixel-level classifier to generate a binary obstacle region segmentation mask;

[0098] The second decoding branch is an obstacle position positioning branch, which converts the multi-dimensional feature map into a fixed-length feature for analysis through a global average pooling operation, and then predicts the boundary box coordinate information of the obstacle through a multi-layer connection network regression;

[0099] The third decoding branch is an obstacle classification type branch, which extracts global signal features through a feature aggregation operation, and then outputs the probability distribution of each obstacle through a classifier network.

[0100] As an example, the obstacle region segmentation result includes: describing the obstacle region boundary information at the pixel level, each pixel point corresponds to a binary label, which is used to indicate whether the pixel point belongs to the obstacle region, forming a complete obstacle segmentation mask graph.

[0101] As an example, the obstacle position positioning result includes: obstacle spatial position information in the form of a boundary box, each boundary box is determined by the upper left corner coordinate and the lower right corner coordinate, which marks the accurate position range of the obstacle in the image, and the confidence score of the boundary box is included, which is used to indicate the reliability of the positioning result;

[0102] As an example, the obstacle type classification result includes: classification labels of common obstacle types in the scenic road, including at least tourist pedestrian class, traffic vehicle class, park maintenance equipment class and fixed landscape facility class, each class label corresponds to a probability value, which is used to indicate the possibility of the obstacle belonging to the class.

[0103] It should be noted that the multi-branch decoder in the embodiment is composed of three decoding branches, which are respectively used for obstacle region segmentation, obstacle position positioning and obstacle type classification.

[0104] S4, combined with the road geometry, weather and signal light state of the surrounding environment information, the attention mechanism is used to weight the obstacle region segmentation result, the obstacle position boundary box coordinate and the obstacle type classification probability to generate an initial obstacle saliency map. It should be noted that this step is:

[0105] The lane lines are detected and the road geometric structure information is calculated through the Hough transform and the RANSAC algorithm, the green area, the building and the water landscape layout information are identified through the imaging segmentation network, the signal condition information such as the signal lamp state and the traffic sign is identified through the deep learning target detection algorithm, and the condition information such as the signal strength, the visibility and the sky state is obtained through the image perception analysis, the enhancement evaluation and the sky segmentation network;

[0106] A multi-modal fusion framework is constructed to convert the obstacle recognition results from the S3 step, wherein the obstacle region segmentation results are converted into spatial saliency features through pixel-level weight calculation and spatial convolution coding, the obstacle position bounding box is converted into position saliency features through distance weighting function and angle offset evaluation, and the obstacle type classification probability is converted through class mapping and semantic context enhancement;

[0107] The correlation weight between different modal features is calculated by using the multi-head attention mechanism of the Transformer calculation architecture, the spatial, position and semantic features are respectively taken as key-value triplets, 8 attention heads are used to process different feature subspaces, and the fusion weight of each modal feature is adaptively adjusted according to the extracted environmental context information;

[0108] Further, the feature fusion weight is adaptively adjusted according to the road form, meteorological condition and signal state information, the spatial saliency weight is enhanced in the terrain condition, the position saliency weight is enhanced in the road or curved ramp section, and the semantic saliency weight is enhanced in the landscape dense area, and the real-time environmental adaptive weight is realized through a dynamic weight updating mechanism;

[0109] The spatial saliency feature, the position saliency feature and the semantic saliency feature are combined into a unified continuous saliency feature map through a weighted fusion operation S(x, y) = w1xF spatial (x, y) + w2xF position (x, y) + w3xF semantic (x, y), and then the continuous saliency map is discretized into a multi-level obstacle saliency fusion feature map through weighted threshold segmentation and morphological processing;

[0110] wherein S(x, y) is the saliency value of the position, F spatial is the spatial saliency feature map, F position is the position saliency feature map, F semantic is the semantic saliency feature map, and w1, w2 and w3 are the fusion weights of the spatial saliency feature map, the position saliency feature map and the semantic saliency feature map respectively.

[0111] S5, according to the preset obstacle type priority level, the detection and processing of the initial obstacle saliency map spatial overlap recognition result, retain high priority target, inhibit low priority target, get conflict resolution after the obstacle recognition result. Among them, the need to explain this step is:

[0112] S5.1, the preset of obstacle type priority level includes: set the tourist pedestrian as the first priority and assign the weight coefficient as 1, set the traffic vehicle as the second priority and assign the weight coefficient as 0.8, set the park maintenance equipment as the third priority and assign the weight coefficient as 0.6, set the fixed landscape facilities as the fourth priority and assign the weight coefficient as 0.4;

[0113] S5.2, the spatial overlap detection is carried out on the multiple obstacle recognition results from S4 step, the boundary box degree overlap is calculated through the boundary box intersection ratio IoU=Area(A∩B) / Area(A∪B), when IoU>0.3, it is spatial intersection overlap, the comprehensive conflict detection matrix overlap relationship is established;

[0114] Among them, Area(·) is the area function, A is the boundary box of the candidate obstacle, B is the boundary box of the adjacent obstacle;

[0115] S5.3, when it is detected that multiple different types of obstacles appear in the same space, the improved priority non-maximum suppression algorithm is used for conflict resolution, according to the comprehensive score score combined =confidence×priority weight The detection result is sorted, the complete information of high priority target is retained, the low priority target is corrected, and the position information of high priority target is updated through the formula NewBBox high =WeightedAverage(BBox high ,BBox low ,overlap ratio );

[0116] Among them, confidence is the confidence, priority weight is the priority weight, NewBBox high is the updated high priority target boundary box, WeightedAverage(·) is the weighted average function, BBox high is the original boundary box of high priority obstacle (such as first, second priority), BBox low is the original boundary box of low priority obstacle (such as fourth priority), overlap ratio is the overlap weight ratio;

[0117] S5.4, execute a three-level conflict resolution process, retain the detection result with the highest confidence for the same overlapping target and fuse feature information, completely retain the high-priority target and overlap the low-priority target according to the priority weight for different overlapping targets, and allocate the region allocation right according to the overlapping share for partially overlapping targets and adjust the bounding box to avoid conflict;

[0118] S5.5, finally generate the obstacle recognition result after conflict resolution, including the unique identification, category information, priority level, confidence, bounding box coordinates, segmentation mask, centroid coordinates and time of the obstacle, and form the obstacle recognition result containing position positioning, type and confidence evaluation.

[0119] S6, the obstacle recognition result is consistent with the road path segmentation map in space, and the time sequence smoothing processing is carried out to eliminate the frame jump, and finally the recognition result containing obstacle type, position, segmentation mask and confidence is output. Among them, it needs to be explained that:

[0120] S6.1, a DeepLabV3+ deep learning segmentation network is used to segment the scenic road into four functional areas, i.e. passable road area, tourist walking area, green landscape area and building area, each area is assigned a unique identification category and weight parameter, and a multi-high-precision correction module and a low-high-level feature fusion decoder structure are used to generate a pixel-level road path segmentation map;

[0121] S6.2, a matching rule library of obstacle type and road function area is established, and the obstacle recognition result from S5 step is verified for spatial consistency, and the spatial consistency score is calculated The score of the obstacle and the functional area is calculated, a hierarchical verification strategy is adopted, including coarse-grained centroid verification, fine-grained overlap degree calculation and boundary intersection check, and 0.6 is defined as the correction threshold;

[0122] wherein, is the overlap ratio, is the area weight, is the target weight;

[0123] S6.3, according to the consistency score, the inconsistent results are divided into three deviation levels of slight, medium and severe, the position deviation and confidence attenuation are carried out for slight deviation (0.4≤score<0.6), the reclassification and area reallocation are carried out for medium deviation (0.2≤score<0.4), and the abnormal mark is directly carried out for severe deviation (score<0.2), through the position deletion formula New Position =Original Position+ a x (Nearest_Valid_Position - Original Position ) and confidence correction formula New Confidence = Original Confidence x (1 - b x Consistency Score ) for accurate correction;

[0124] wherein, New Position is the obstacle position coordinate after spatial consistency correction, Original Position is the original detection position of the obstacle before correction, a is the correction coefficient, Nearest_Valid_Position is the center position of the nearest reasonable functional area from the original position, New Confidence is the obstacle detection confidence after consistency correction, Original Confidence is the original detection confidence of the obstacle before correction, b is the confidence decay coefficient, and 1 - b x Consistency Score is the confidence retention ratio;

[0125] S6.4, a Kalman filter system for obstacle tracking is established, the state information of the obstacle is defined as a six-dimensional state including the horizontal and vertical coordinates of the center position, the horizontal and vertical direction motion speeds, and the width and height of the bounding box, the state relationship change of the obstacle between adjacent frames is described by a uniform motion model, and a preset state change matrix is used to reflect the linear change law of the position with the speed and the speed;

[0126] The Hungarian algorithm is used to realize the optimal association matching of the obstacles between frames, the corresponding relationship is established by calculating the comprehensive similarity between the detection results of the current frame and the tracked targets of the previous frame, the calculation of the comprehensive similarity comprehensively considers the position distance, the size similarity and the matching degree of the appearance features, the association distance threshold is set to 50 pixels, when the distance between the detection result and the tracked target exceeds the threshold, it is identified as a newly appeared obstacle target, and the tracked target that is successfully associated for three steps in succession is determined as a disappeared obstacle and is deleted from the tracking list;

[0127] In the processing process of each frame, a prediction step is first performed, the obstacle state of the current frame is predicted according to the state estimation of the previous frame and the state transition model, then an update step is performed, the Kalman gain matrix is calculated to balance the weights of the predicted value and the initial value of the current frame, and the optimal state estimation of the current frame is obtained by weighted prediction state and initial residual error;

[0128] The obstacle recognition result of the current frame is fed back through the estimation result to perform instantaneous positioning, the filter smoothes the abnormal position jump according to the historical motion track of the obstacle, and when the detected position change exceeds the reasonable motion range, the weight of the current start will be reduced, and the position confidence score of the obstacle detection is dynamically adjusted according to the state in the simulation process;

[0129] S6.5, a standardized data structure containing complete information of the obstacle is established, specifically including a unique identifier of the obstacle, category information, spatial information, motion information, quality information and processing information, and exemplarily, the category information includes a category number, a category name, a priority level and a detection confidence of the obstacle, the spatial information includes a bounding box coordinate, a center position, a partition code array and a pixel area, the motion information includes a speed, a predicted position and a tracking identifier, the quality information includes a spatial consistency score, a correction smoothness score and a comprehensive quality score, and the processing information includes a Boolean flag indicating a state of conflict elimination, a state of spatial correction and a state of smoothing;

[0130] Further, a minimum confidence threshold of 0.3 is set, and the recognition result with a confidence lower than the threshold is filtered to ensure the output quality, and the maximum output of a single frame is 1 to balance the detection defect limitation and the calculation efficiency, a comprehensive quality score mechanism is established, the spatial consistency score, the smoothness score and the detection confidence are weighted and fused according to the weights of 0.4, 0.3 and 0.3, and the recognition result is sorted according to the comprehensive score and the detection target with high quality is given priority;

[0131] It should be noted that the output result supports output of multiple standard data formats to meet the integration needs of different application scenarios, such as JSON format and XML format, and all output formats maintain the consistency and integrity of the data structure to ensure that the downstream system can correctly parse and use the recognition result.

[0132] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, and they should be covered in the scope of the claims of the present application.

Claims

1. A method for automatic driving obstacle identification for a complex scenic road scene, characterized in that, The method comprises the following steps: Collecting original images of scenic area roads and performing noise suppression, brightness equalization and contrast enhancement to obtain enhanced images with uniform resolution; Inputting the enhanced images into a multi-scale feature extraction network to extract texture features, shape features and semantic features respectively, and fusing the features through a feature pyramid network to generate multi-scale feature maps; Inputting the multi-scale feature maps into a multi-branch decoder to obtain obstacle region segmentation results, obstacle position bounding box coordinates and obstacle type classification probabilities respectively; Combining the road geometry, weather and signal light state of the surrounding environment information, and using an attention mechanism to weight the obstacle region segmentation results, obstacle position bounding box coordinates and obstacle type classification probabilities to generate an initial obstacle saliency map; According to the preset obstacle type priority level, detecting and processing the spatially overlapping recognition results in the initial obstacle saliency map, retaining high-priority targets and suppressing low-priority targets to obtain obstacle recognition results after conflict resolution; Conducting spatial consistency correction on the obstacle recognition results and the road path segmentation map, and eliminating inter-frame jumps based on time sequence smoothing processing to finally output the recognition results containing obstacle type, position, segmentation mask and confidence. 2.The method of claim 1, wherein, The enhanced road images are obtained by: Performing bilateral filtering on the original scenic area road images to suppress random noise while preserving edge details; Using an adaptive histogram equalization method to locally enhance the contrast of areas with uneven brightness distribution; Adjusting the gamma coefficient dynamically according to the global brightness statistics to make the overall brightness of the enhanced image tend to be uniform.

3. The method of claim 1 or 2, wherein, The collection of the original scenic area road images meets the following environmental requirements, including: Collecting scenic area road images under sunny, cloudy, rainy, foggy and night conditions respectively; Each batch of images collected under each weather condition includes at least three illumination directions: directly facing the sun, facing away from the sun and side light; The scenic area road images include at least main road images within the scenic area, branch path images within the scenic area, scenic area parking lot images and scenic area road images under different weather conditions, wherein: The scenic area road images under different weather conditions include but are not limited to strong light images under sunny high irradiation conditions, weak light images under cloudy low light conditions, reflective images under dry rainy conditions, blurred images under foggy conditions, and bright images under daytime light conditions, supplementary light images under changing light conditions at dusk, and low-illumination images under artificial lighting conditions at night.

4. The automatic driving obstacle recognition method for a complex scenic road scene according to claim 1 or 2, characterized in that, The method for generating multi-dimensional feature maps comprises the following steps: Taking the enhanced road images as input data, the input images are extracted layer by layer through the shallow layer group, the middle layer group and the deep layer group in the multi-dimensional feature extraction network; The shallow layer group extracts image edge texture features, the middle layer group extracts image shape structure features, and the deep layer group extracts image semantic content features; The features of different layer groups are fused through a feature pyramid network, and the feature map size is unified through upsampling and downsampling operations, and finally a multi-dimensional feature map containing multi-dimensional feature information is output.

5. The method of claim 4, wherein the method comprises: The multi-dimensional feature map is input into a multi-branch decoder, and obstacle region segmentation results, obstacle position positioning results and obstacle type classification results are obtained through three decoding branches respectively, wherein: The first decoding branch is an obstacle region segmentation branch, which analyzes the image resolution on the multi-dimensional feature map through a gradient inverse matrix operation, and determines the attribution of each pixel point to an obstacle region through a pixel-level classifier to generate a binary obstacle region segmentation mask; The second decoding branch is an obstacle position positioning branch, which converts the multi-dimensional feature map into a fixed-length feature for analysis through a global average pooling operation, and then predicts the boundary box coordinate information of the obstacle through a multi-layer connection network regression; The third decoding branch is an obstacle classification type branch, which extracts global signal features through a feature aggregation operation, and then outputs the probability distribution of each obstacle through a classifier network.

6. The method of claim 5, wherein the method comprises: The obstacle region segmentation result includes: describing the obstacle region boundary information at the pixel level, each pixel point corresponds to a binary label, which indicates whether the pixel point belongs to the obstacle region, forming a complete obstacle segmentation mask map; The obstacle position positioning result includes: obstacle spatial position information in the form of a boundary box, each boundary box is determined by the upper left corner coordinate and the lower right corner coordinate, which marks the accurate position range of the obstacle in the image, and also includes the confidence score of the boundary box, which indicates the reliability of the positioning result; The obstacle type classification result includes: classification labels of common obstacle types in the scenic road, including at least tourist pedestrian class, traffic vehicle class, park maintenance equipment class and fixed landscape facility class, each class label corresponds to a probability value, which indicates the possibility of the obstacle belonging to the class.

7. The method of claim 6, wherein the method comprises: The obstacle saliency feature map is obtained, including: Extracting the surrounding environment information of the scenic road, including road geometry information, surrounding road landscape layout information, signal condition information and weather condition information; Constructing a multi-modal fusion framework to convert the obstacle region segmentation result into spatial saliency features, the obstacle position positioning result into position saliency features, and the obstacle type classification result into semantic saliency features; Using attention mechanism to calculate the correlation weight between different modal features, and adaptively adjusting the weight fusion of each modal feature according to the regional path context environment information; The spatial saliency features, position saliency features and semantic saliency features are merged into a unified discrete obstacle saliency feature map through weighted fusion operation. 8.The method of claim 7, wherein, The obstacle position positioning result and the obstacle type classification result in the obstacle saliency feature map are corrected using the obstacle type priority level strategy, including: Establishing an obstacle type priority level for the road scene according to the obstacle type, wherein the tourist pedestrian is set as the first priority, the traffic vehicle is set as the second priority, the park maintenance equipment is set as the third priority, and the fixed landscape facility is set as the fourth priority; The multiple obstacle recognition results with spatial overlap of obstacle salient features are subjected to conflict detection, when multiple different types of obstacles are detected in the same space, the result is screened according to the priority level strategy; The recognition result of the fourth priority obstacle is subjected to transcription processing by a non-maximum transcription factor, the position positioning result and type classification result of the first priority obstacle are retained, and the obstacle recognition result after conflict resolution is obtained.

9. The method of claim 8, wherein the method comprises: The modified recognition result is subjected to consistency correction in combination with the scenic road path segmentation information, and finally the scenic obstacle recognition result is output, including: The road path region of the scenic road is segmented by a segmentation network, and a road path segmentation map is generated, including a passable road region, a tourist walking region, a green landscape region and a building region; The modified obstacle recognition result is subjected to spatial consistency verification with the road path segmentation map, and whether the recognition position of the obstacle matches the road function region that should appear is checked; The recognition result that does not meet the spatial consistency verification is corrected; And based on time series smoothing processing, inter-frame jumping is eliminated, and finally complete obstacle recognition results including obstacle type label, position coordinates, region segmentation mask and confidence score are output.

10. The method of claim 9, wherein the method is a method of automatic driving obstacle recognition for a complex scenic road scene. The time series smoothing processing specifically includes: Based on the Kalman filtering algorithm, the position and speed of the obstacle in adjacent frames are estimated; The estimation result is fed back to the current frame recognition result to weaken the instantaneous positioning jitter and update the confidence; The final obstacle recognition result after time smoothing is output.

Images