Vehicle edge driving scene identification method, vehicle and computer readable storage medium

Abstract

The embodiment of the invention provides a vehicle edge driving scene recognition method, a vehicle and a computer readable storage medium. The method comprises the steps that a scene image of a to-be-recognized driving scene of the vehicle is acquired; calling a semantic segmentation model, and segmenting the scene image into a plurality of sub-image areas; environment feature vectors corresponding to the multiple sub-image areas are determined, and the environment feature vectors are used for representing features of the sub-image areas for the global environment; based on the environment feature vector, determining isolation degree evaluation results corresponding to the plurality of sub-image regions; determining a marginal evaluation index corresponding to the scene image based on the isolation degree evaluation results corresponding to the plurality of sub-image regions; and in response to the edge evaluation index, indicating that the edge degree of the to-be-identified driving scene is greater than or equal to an edge degree threshold, determining that the to-be-identified driving scene is an edge driving scene. The technical problem that the identification of the edge driving scene of the vehicle is not comprehensive is solved.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and more specifically, to a method for recognizing edge driving scenes of a vehicle, a vehicle, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of autonomous driving technology, a vehicle's autonomous driving capability has become an important indicator of its advancement and safety. The reliable operation of autonomous vehicles on the road highly depends on the perception system's accurate understanding and rapid response to driving scenarios. However, edge-of-the-road or extreme driving scenarios (such as rain, snow, fog, nighttime, strong light / backlight, traffic congestion, and obstructions) are the main reasons for the degradation or even failure of perception model performance.

[0003] In related technologies, a large amount of labeled extreme scene data is mainly used to train the classification model, and then the classification model is used to identify the vehicle driving scene. However, since the labeled extreme scene data is difficult to cover all edge cases, the classification model is limited to recognizing driving scenes that have appeared in the labeled extreme scene dataset, and it is difficult to identify and deal with new extreme cases. There is a technical problem of incomplete recognition of the vehicle's edge driving scene.

[0004] There is currently no good solution to the above problems. Summary of the Invention

[0005] This application provides a method for recognizing edge driving scenes of a vehicle, a vehicle, and a computer-readable storage medium, so as to at least solve the technical problem of incomplete recognition of edge driving scenes of a vehicle.

[0006] According to one aspect of the embodiments of this application, a method for recognizing edge driving scenes of a vehicle is provided, comprising: acquiring a scene image of a driving scene to be recognized; invoking a semantic segmentation model to segment the scene image into multiple sub-image regions, wherein any sub-image region includes a semantic feature in the scene image; determining an environmental feature vector corresponding to each of the multiple sub-image regions, wherein the environmental feature vector is used to characterize the features of the sub-image region relative to the global environment; determining an isolation degree evaluation result corresponding to each of the multiple sub-image regions based on the environmental feature vector, wherein the isolation degree evaluation result is used to characterize the degree of isolation of the sub-image region; determining an edge evaluation index corresponding to the scene image based on the isolation degree evaluation result corresponding to each of the multiple sub-image regions, wherein the edge evaluation index is used to characterize the edge degree of the driving scene to be recognized; and determining the driving scene to be recognized as an edge driving scene in response to the edge evaluation index indicating that the edge degree of the driving scene to be recognized is greater than or equal to an edge degree threshold.

[0007] Furthermore, the semantic segmentation model is invoked to segment the scene image into multiple sub-image regions, including: invoking the semantic segmentation model; inputting the scene image into the semantic segmentation model to extract multiple semantic regions in the scene image, wherein any semantic region includes a semantic element in the scene image; and performing connected component analysis on the multiple semantic regions to obtain multiple sub-image regions.

[0008] Furthermore, connected component analysis is performed on multiple semantic regions to obtain multiple sub-image regions, including connecting semantic regions that contain the same semantic elements to obtain multiple sub-image regions.

[0009] Further, the environmental feature vectors corresponding to each of the multiple sub-image regions are determined, including: extracting the visual features, illumination features, and environmental features corresponding to each of the multiple sub-image regions, wherein the visual features are used to represent the visual attributes of the sub-image region, the illumination features are used to characterize the illumination level of the sub-image region, and the environmental features are used to characterize the environmental conditions corresponding to the sub-image region; based on the visual features, illumination features, and environmental features, the environmental feature vectors corresponding to each of the multiple sub-image regions are determined.

[0010] Furthermore, based on visual features, illumination features, and environmental features, the environmental feature vectors corresponding to each of the multiple sub-image regions are determined, including: concatenating the visual features, illumination features, and environmental features into vectors to obtain the environmental feature vectors corresponding to each of the multiple sub-image regions.

[0011] Furthermore, based on the environmental feature vector, the isolation degree evaluation results corresponding to each of the multiple sub-image regions are determined, including: determining the Euclidean distance between the environmental feature vector of the target sub-image region and the environmental feature vectors of the sub-image regions other than the target sub-image region in the multiple sub-image regions, and obtaining multiple Euclidean distances, wherein the target sub-image region is any sub-image region among the multiple sub-image regions; and determining the isolation degree evaluation result corresponding to the target sub-image region based on the average Euclidean distance corresponding to the multiple Euclidean distances.

[0012] Furthermore, based on the isolation degree evaluation results corresponding to each of the multiple sub-image regions, the marginality evaluation index corresponding to the scene image is determined, including: sorting the multiple sub-image regions in descending order according to their isolation degree to obtain a sorted queue; selecting the top target number of sub-image regions from the sorted queue; determining the average isolation degree corresponding to the target number of sub-image regions; and determining the marginality evaluation index corresponding to the scene image based on the average value.

[0013] Furthermore, the edge severity of the driving scene to be identified is positively correlated with the degree of isolation of each of the multiple sub-image regions.

[0014] According to another aspect of the embodiments of this application, a vehicle edge driving scene recognition device is also provided, comprising: an acquisition unit, configured to acquire a scene image of a driving scene to be recognized; a segmentation unit, configured to invoke a semantic segmentation model to segment the scene image into multiple sub-image regions, wherein any sub-image region includes a semantic feature in the scene image; a first determination unit, configured to determine an environmental feature vector corresponding to each of the multiple sub-image regions, wherein the environmental feature vector is used to characterize the features of the sub-image region relative to the global environment; a second determination unit, configured to determine an isolation degree evaluation result corresponding to each of the multiple sub-image regions based on the environmental feature vector, wherein the isolation degree evaluation result is used to characterize the isolation degree of the sub-image region; a third determination unit, configured to determine an edge evaluation index corresponding to the scene image based on the isolation degree evaluation result corresponding to each of the multiple sub-image regions, wherein the edge evaluation index is used to characterize the edge degree of the driving scene to be recognized; and a fourth determination unit, configured to determine the driving scene to be recognized as an edge driving scene in response to the edge evaluation index indicating that the edge degree of the driving scene to be recognized is greater than or equal to an edge degree threshold.

[0015] According to another aspect of the embodiments of this application, a vehicle is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0016] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0017] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0018] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0019] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0020] In this embodiment, semantic segmentation technology is introduced to decompose complex scene images into multiple sub-image regions containing specific semantic features (e.g., vehicles, pedestrians, road markings). This fine-grained segmentation strategy greatly enriches the understanding of scene details. Subsequently, an environmental feature vector is constructed for each sub-image region, revealing its uniqueness within the global environment. The isolation degree evaluation results of each sub-image region, calculated through an unsupervised learning process, quantify the edge degree of each sub-region. Based on the isolation degree evaluation results corresponding to each of the multiple sub-image regions, an edge evaluation index for the entire scene image can be determined. This edge evaluation index quantifies the edge degree of the driving scene to be identified, accurately determining whether the scene to be identified is an edge driving scene. In other words, this application, through an automated and unsupervised learning process, can not only identify known edge scenes but also discover unknown edge situations, providing stronger support for autonomous driving systems facing complex and ever-changing real-world driving environments. This solves the technical problem of incomplete identification of edge driving scenes for vehicles, laying a solid foundation for improving the safety and reliability of autonomous driving. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0022] Figure 1 This is a flowchart of a vehicle edge driving scene recognition method according to an embodiment of this application;

[0023] Figure 2 This is a flowchart of a method for recognizing edge driving scenarios according to an embodiment of this application;

[0024] Figure 3 This is a flowchart of an edge scene data construction method according to an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of a vehicle edge driving scene recognition device according to an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] According to an embodiment of this application, an embodiment of a vehicle edge driving scene recognition method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0029] This embodiment provides a method for recognizing vehicle edge driving scenes. Figure 1 This is a flowchart of a vehicle edge driving scene recognition method according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps:

[0030] Step S101: Obtain scene images of the driving scene to be identified for the vehicle.

[0031] In the technical solution provided in step S101 of this application, the aforementioned scene image is an image of a vehicle's driving scene captured in real time or historically by an in-vehicle camera or data recorder. This scene image will serve as the data basis for subsequent analysis and identification. The scene image contains specific information about the surrounding environment when the vehicle is driving on the road, including but not limited to road conditions, weather conditions, light levels, and the positions of traffic participants.

[0032] In this embodiment, the acquisition of scene images relies on various sensors equipped on the vehicle, particularly cameras. These cameras can be monocular, binocular, or multi-view cameras, mounted in different parts of the vehicle (e.g., front, sides) to capture a 360-degree view of the driving environment. Furthermore, the scene image data can originate from real-time acquisition or from previously recorded datasets, the latter often used for offline analysis and model training.

[0033] Optionally, after acquiring the scene image, some preprocessing operations can be performed to ensure that the image quality and format meet the requirements of subsequent analysis. Preprocessing steps may include image resizing, color correction, noise suppression, etc., to improve the accuracy of subsequent semantic segmentation and feature extraction.

[0034] Optionally, scene images of the driving scenario serve as the primary source of information for the perception and decision-making of the autonomous driving system, and the image quality of these scene images directly affects the system's performance. For example, in complex traffic environments, high-resolution, low-noise scene images can help the autonomous driving system more accurately identify road markings, the positions of other vehicles and pedestrians, thereby making safer driving decisions. In adverse weather conditions, such as rain, snow, or fog, appropriate image preprocessing and enhancement can make scene images clearer, improving the perception performance of autonomous vehicles under these conditions.

[0035] Step S102: Call the semantic segmentation model to segment the scene image into multiple sub-image regions.

[0036] In the technical solution provided in step S102 of this application, after obtaining the scene image through step S101, a semantic segmentation model can be invoked to segment the scene image into multiple sub-image regions. Each sub-image region includes a semantic feature in the scene image, which characterizes the semantic category to which the scene elements in the scene image belong, such as vehicles, pedestrians, or roads. The semantic segmentation model is a model pre-trained with a large number of driving scene image samples, capable of accurately decomposing the input image into multiple sub-regions, each corresponding to a specific semantic category, such as vehicles, pedestrians, or roads. This semantic segmentation model can be a Mask Region-based Convolutional Neural Network (Mask R-CNN), a U-Net, or a more advanced pixel-level classification mask model (Mask2Former), without specific limitations here.

[0037] In this embodiment, semantic segmentation is a key technology in the field of computer vision, capable of classifying each pixel in a scene image into a predefined semantic category, such as roads, vehicles, pedestrians, and trees. Compared to traditional image classification or object detection, semantic segmentation provides a more refined, pixel-level understanding, facilitating in-depth analysis of the spatial distribution characteristics of various components in an image.

[0038] For example, the acquired scene image can be preprocessed to meet the input requirements of the semantic segmentation model. This preprocessing includes adjusting image size, standardizing pixel values, etc., which are not specifically limited here. After preprocessing, the preprocessed scene image can be input into the semantic segmentation model. After receiving the scene image, the semantic segmentation model can output a semantic segmentation map with the same size as the scene image. This semantic segmentation map divides the scene image into multiple sub-image regions, each corresponding to a semantic feature, that is, each sub-image region corresponds to a semantic category label.

[0039] In this step, pixel-level segmentation of the scene image allows for focusing on the specificity and complexity of each sub-image region, something that whole-image analysis cannot achieve. This fine-grained segmentation method can more accurately capture local anomalies in vehicle driving scenes.

[0040] Step S103: Determine the environmental feature vectors corresponding to each of the multiple sub-image regions.

[0041] In the technical solution provided by step S103 of this application, the environmental feature vector is used to characterize the features of the corresponding sub-image region in relation to the global environment. The environmental feature vector is obtained by fusing the visual features, illumination features and environmental features of the corresponding sub-image region.

[0042] In this embodiment, when determining the environmental feature vectors corresponding to each of the multiple sub-image regions, the visual features, illumination features, and environmental features corresponding to each sub-image region can be determined first. For visual features, a visual semantic feature vector can be extracted for each sub-image region using a visual language model (e.g., the CLIP model). This visual semantic feature vector reflects the visual characteristics of the sub-image region, such as visual attribute information like the category, shape, and texture of objects within the sub-image region. For illumination features, based on the brightness information of each sub-image region, statistical measures such as the mean brightness and standard deviation of brightness can be calculated to form the illumination feature vector for each sub-image region. This illumination feature vector is used to characterize the illumination features of the sub-image region. For environmental features, based on global environment-related prompts (e.g., "rainy day," "night") corresponding to the scene image, a global environmental feature vector can be generated using the text encoder of a visual language model (e.g., the CLIP model). This vector is then assigned to each sub-image region, providing environmental context information for the local features of the scene image. The global environmental feature is a key bridge connecting the local and the global context.

[0043] Optionally, after obtaining the visual features, illumination features, and environmental features corresponding to each of the multiple sub-image regions, the feature vectors corresponding to these three features can be concatenated to form a comprehensive environmental feature vector that includes visual representation, illumination statistics, and environmental context information. This environmental feature vector not only includes the internal features of the sub-image region but also incorporates its position and state within the global environment, thus providing a more comprehensive description of the environmental state of the sub-image region. In this way, the correlation between the sub-image region and the global environment can be quantified, providing rich multimodal information for subsequent isolation scoring.

[0044] In this step, by constructing environmental feature vectors, the characteristics of each sub-image region and its relationship with the overall environment can be understood from a multi-dimensional and fine-grained perspective. This process not only deepens the understanding of local features but also makes the feature representation more comprehensive by fusing global environmental information, providing solid data support and theoretical basis for the subsequent evaluation and construction of edge scenes (or extreme scenes).

[0045] Step S104: Based on the environmental feature vector, determine the isolation degree evaluation results corresponding to each of the multiple sub-image regions.

[0046] In the technical solution provided in step S104 of this application, after obtaining the environmental feature vector, the isolation degree evaluation results corresponding to each of the multiple sub-image regions can be determined based on the environmental feature vector. The isolation degree evaluation results are used to characterize the degree of isolation of the sub-image regions. As shown in step S103, the environmental feature vector includes the visual semantic features, illumination features, and global environmental features of the sub-image region, and is the result of multimodal feature fusion. These feature vectors represent the complexity and uniqueness of the sub-image regions in a high-dimensional space. For example, the visual features of a specific sub-image region are used to describe the shape, color, and texture of the sub-image region, while the illumination features capture the brightness and shadow changes of the sub-image region, and the global environmental features reflect the overall environmental state related to the sub-image region, such as "rainy day" or "night." The isolation degree evaluation result is obtained by calculating the degree of isolation of the environmental feature vector in the high-dimensional feature space. In the high-dimensional space, if a feature vector is far from other vectors in the dataset, then this vector is considered more isolated. This isolation can reflect the differences between sub-image regions and regular scenes in terms of visual semantics, lighting conditions, or environmental background, thereby indirectly indicating whether they belong to the edge scene.

[0047] In this embodiment, the environmental feature vectors of multiple sub-image regions can be constructed into a set or matrix, with each row representing the feature vector of a sub-region. Then, a proximity algorithm from unsupervised learning, such as the K-Nearest Neighbors (K-NN) algorithm, can be used to calculate the isolation degree of each sub-image region's feature vector. For example, for any sub-image region's corresponding environmental feature vector, the K nearest neighbor environmental feature vectors can be found in the feature space, and then the average Euclidean distance between the environmental feature vector and these nearest neighbor vectors can be calculated. This average Euclidean distance can be considered as the isolation degree score of the sub-image region, that is, the isolation degree evaluation result of the sub-image region.

[0048] Optionally, to ensure the comparability of isolation scores, the calculated isolation scores can be standardized, such as by L2 norm standardization, to ensure that the score values ​​are within a certain range, which facilitates subsequent edge scene evaluation and screening.

[0049] Optionally, the isolation score directly reflects whether the features of a sub-image region are "edge" or "abnormal" in the dataset. A sub-image region with a high isolation score means that it is significantly different from other sub-image regions in the dataset in terms of visual semantics, lighting conditions, or environmental features. This may be because the sub-image region contains anomalies specific to edge driving scenes, such as pedestrians under abnormal lighting or road signs in rainy nights. Therefore, by calculating the isolation score, the sub-image regions most likely to belong to edge driving scenes can be quantified and identified.

[0050] In this step, the anomalies and complexity of each sub-image region are effectively assessed by calculating its isolation score. This step not only leverages the advantages of modern computer vision models (such as Mask2Former and CLIP) but also combines traditional machine learning methods, such as the K-Nearest Neighbors (K-NN) algorithm, to achieve fine-grained quantization of multimodal features and automated edge scene recognition. The isolation evaluation results provide crucial basis for subsequent image-level edge judgment and the construction of edge scene datasets.

[0051] Step S105: Based on the isolation degree evaluation results corresponding to each of the multiple sub-image regions, determine the edge evaluation index corresponding to the scene image.

[0052] In the technical solution provided by step S105 of this application, after obtaining the isolation degree evaluation results corresponding to each of the multiple sub-image regions through step S104, the edge evaluation index corresponding to the scene image can be determined based on the isolation degree evaluation results corresponding to each of the multiple sub-image regions. The edge evaluation index is used to characterize the edge degree or extreme degree of the driving scene to be identified.

[0053] In this embodiment, the isolation degree evaluation results of multiple sub-image regions can be aggregated to construct an edge evaluation index that can comprehensively reflect the edge degree of the driving scene to be identified. As can be seen from the aforementioned step S104, the isolation degree evaluation results reflect the uniqueness of each sub-graphic region in the multimodal feature space, and by aggregating these results, a quantitative index about the overall edge degree of the scene can be obtained.

[0054] For example, the top few sub-image regions with higher isolation scores (e.g., the top 5 or top 10) can be selected from the isolation scores of multiple sub-image regions, and then the isolation scores of the selected sub-image regions can be calculated using a weighted average or a simple average. The purpose of this is to focus on the most peripheral, extreme, or abnormal parts of the scene, because seemingly normal scenes often hide local edge conditions, which pose a greater challenge to autonomous driving systems.

[0055] Optionally, after obtaining the average isolation scores of the selected sub-image regions, an edge evaluation index for the scene image can be calculated according to a preset score aggregation strategy. This edge evaluation index comprehensively considers the isolation scores of all sub-image regions in the scene, reflecting the diverse anomaly characteristics of the entire scene. A higher score means that there are more or more obvious edge conditions in the scene.

[0056] In this step, by aggregating the isolation scores of sub-image regions, an edge evaluation index for the driving scene to be identified can be constructed, providing a quantitative indicator for the edge assessment of the driving scene to be identified.

[0057] Step S106: In response to the edge evaluation index indicating that the edge degree of the driving scene to be identified is greater than or equal to the edge degree threshold, the driving scene to be identified is determined to be an edge driving scene.

[0058] In the technical solution provided by step S106 of this application, after obtaining the edge evaluation index of the scene image through step S105, the edge evaluation index is used to indicate the edge degree of the driving scene to be identified. The edge degree threshold is a numerical standard pre-set according to system requirements and the diversity of target scenes, used to filter out truly complex edge driving scenes or extreme driving scenes that pose a challenge to the perception system of the autonomous driving system.

[0059] In this embodiment, after obtaining the edge evaluation index of the scene image, the edge degree of the driving scene to be identified indicated by the edge evaluation index can be compared with the edge degree threshold to obtain a comparison result. If the comparison result indicates that the edge degree of the driving scene to be identified is greater than or equal to the edge degree threshold, the driving scene to be identified is determined to be an edge driving scene.

[0060] In this step, by quantifying the edge degree of the scene image and comparing it with a preset edge degree threshold, the goal of automatically recognizing edge driving scenes is achieved. This not only reduces the reliance on manual intervention but also improves recognition efficiency and accuracy.

[0061] In steps S101 to S106 above, semantic segmentation technology is introduced to decompose complex scene images into multiple sub-image regions containing specific semantic features (e.g., vehicles, pedestrians, road markings). This fine-grained segmentation strategy greatly enriches the understanding of scene details. The environmental feature vector of each sub-image region reveals the uniqueness of the corresponding sub-image region in the global environment. By calculating the isolation evaluation results of each sub-image region, the degree of anomaly of each sub-image region relative to the entire dataset can be quantitatively evaluated. This proximity analysis based on high-dimensional feature space does not artificially define edge scenes, but allows the data itself to reveal which local conditions deviate from the norm, thus more realistically reflecting edge situations in complex driving environments. Finally, by aggregating the edge evaluation results of all sub-image regions, a comprehensive edge evaluation index is generated for the scene image. This edge evaluation index can accurately determine whether the driving scene to be identified is an edge driving scene, thereby improving the recognition accuracy of edge driving scenes and solving the technical problem of low recognition accuracy of driving scenes.

[0062] The edge driving scene recognition method for the aforementioned vehicles described in this application will be further introduced below.

[0063] As an optional implementation, step S102 involves calling a semantic segmentation model to segment the scene image into multiple sub-image regions, including: calling the semantic segmentation model; inputting the scene image into the semantic segmentation model to extract multiple semantic regions from the scene image, wherein any semantic region includes a semantic element in the scene image; and performing connected component analysis on the multiple semantic regions to obtain multiple sub-image regions.

[0064] In this embodiment, the scene image is divided into multiple sub-image regions by invoking a semantic segmentation model. Each sub-image region focuses on specific semantic elements, such as vehicles, pedestrians, and roads. This decomposition method not only improves the depth of understanding of image content but also creates conditions for subsequent feature extraction and abnormal scene recognition.

[0065] Optionally, semantic segmentation models are powerful tools in the field of deep learning, capable of classifying each pixel in an image into a predefined category, thereby identifying different objects and background regions in the image. In step S102, a pre-trained semantic segmentation model, such as Mask2Former, is first loaded and invoked. This model is an advanced model based on the Transformer architecture, which performs better than traditional convolutional neural networks in complex scenes, and is particularly suitable for image parsing in autonomous driving scenarios. Next, the scene image data obtained in step S101 is input into the semantic segmentation model. The semantic segmentation model performs depth inference on the scene image and outputs a predicted label map that matches the size of the input image, where different label values ​​correspond to different semantic elements in the scene image. This means that the scene image is decomposed into a combination of various semantic regions such as roads, vehicles, pedestrians, trees, and buildings.

[0066] Optionally, the predicted label map output by the semantic segmentation model divides the entire scene image into multiple semantic regions. However, since objects of the same category may appear scattered in the image, such as multiple cars or multiple pedestrians, based on this, connected component analysis can be performed on multiple semantic regions to identify the connected parts under the same category label, and each connected part can be regarded as an independent sub-image region.

[0067] As an optional implementation, connected component analysis is performed on multiple semantic regions to obtain multiple sub-image regions, including: connecting semantic regions that include the same semantic elements in multiple semantic regions to obtain multiple sub-image regions.

[0068] In this embodiment, when performing connected component analysis on multiple semantic regions, regions containing the same semantic elements can be connected among the multiple semantic regions obtained from semantic segmentation, thereby dividing meaningful sub-image regions.

[0069] Optionally, during semantic segmentation, each pixel is assigned a category label, thereby decomposing the scene image into multiple regions with specific meanings. However, due to factors such as image noise and edge detection errors, the same object or semantic element may be segmented into multiple discontinuous fragments rather than a complete region. Such segmentation results may lead to information loss or inaccurate evaluation conclusions during feature analysis and evaluation. Therefore, the purpose of introducing connected component analysis is to merge these fragmented regions, restore the complete shape of the object, and facilitate subsequent multimodal feature learning and edge scene evaluation.

[0070] Optionally, connected component analysis algorithms, such as the 4-connectivity or 8-connectivity criteria, can be used to scan the entire semantic segmentation map. For a specific semantic category (e.g., "vehicle"), all pixels labeled as belonging to this category are traversed. By examining the connectivity between adjacent pixels, they are grouped together to form a connected component. Through connected component analysis, small regions that might otherwise be scattered but belong to the same semantic element are merged into a complete sub-image region. This process ensures that each sub-image region represents a complete object instance or environmental unit, avoiding information fragmentation problems that may be encountered in subsequent analysis.

[0071] Optionally, for each identified connected component, a corresponding sub-image region is generated. This typically involves using a mask to extract relevant pixels from the original image, ensuring that the sub-image region not only contains the pixels within the connected component but also preserves its contextual information within the original scene.

[0072] Optionally, metadata for each sub-image region is recorded, including but not limited to its semantic category, its exact location in the image (via mask or bounding box), size, and shape. This information will be used for subsequent feature extraction and edge detection.

[0073] In this step, connected component analysis serves as a bridge between semantic segmentation and multimodal feature learning. It addresses the region fragmentation problem that semantic segmentation may cause, ensuring the accuracy of subsequent feature extraction. Connected component analysis of multiple semantic regions aims to reconstruct and instantiate objects or regions in the scene, providing a more accurate and comprehensive data foundation for subsequent edge scene recognition and evaluation.

[0074] As an optional implementation, step S103, determining the environmental feature vectors corresponding to each of the multiple sub-image regions, includes: extracting visual features, illumination features, and environmental features corresponding to each of the multiple sub-image regions, wherein the visual features are used to represent the visual attributes of the sub-image regions, the illumination features are used to characterize the illumination level of the sub-image regions, and the environmental features are used to characterize the environmental conditions corresponding to the sub-image regions; and determining the environmental feature vectors corresponding to each of the multiple sub-image regions based on the visual features, illumination features, and environmental features.

[0075] In this embodiment, when determining the environmental feature vectors corresponding to each of the multiple sub-image regions, the visual features, illumination features, and environmental features corresponding to each sub-image region can be extracted first. The extraction of visual features mainly relies on a pre-trained deep learning model, such as the CLIP model. The CLIP model, trained on a large-scale image and text pairing dataset, is able to understand image content and map it into a shared embedding space. By inputting a cropped image of the sub-image region, the CLIP model can output a high-dimensional vector that encodes the visual semantic information within the region, including the type, shape, texture, and color of objects.

[0076] Optionally, for illumination features, which focus on describing the lighting conditions within a sub-image region, it is a quantification of the illumination conditions in a local area of ​​the image. These illumination features include brightness, contrast, and illumination direction. Statistics such as the mean brightness, standard deviation of brightness, and maximum brightness of each sub-image region can be calculated. Illumination features have a significant impact on the visual effect of an image, especially in edge-driven driving scenes, such as in rainy nights or under strong light. Illumination features can reveal abnormal states in brightness, shadows, and reflections in image regions, which is crucial for detecting local edge conditions.

[0077] Optionally, environmental features relate to the macroscopic environmental state of the sub-image region. These features are generated based on global environmental cues (e.g., "foggy" or "sunny"). CLIP's text encoder converts these cues into feature vectors, which are then matched with the image's global features to find the most relevant environmental description. The environmental feature vectors provide contextual information for each sub-region, helping the system understand the meaning of local features within a broader environmental context. This is particularly important when assessing whether a sub-region belongs to an edge scene, where environmental features provide crucial reference information.

[0078] Optionally, after extracting the visual features, illumination features, and environmental features corresponding to each of the multiple sub-image regions, these features can be integrated into a comprehensive environmental feature vector.

[0079] In this step, by extracting and integrating the visual features, illumination features, and environmental features of the sub-image regions, an environmental feature vector describing the complexity of the local environment is constructed. Subsequent region-level edge measurement and image-level edge evaluation provide key multimodal information.

[0080] As an optional implementation, environmental feature vectors corresponding to multiple sub-image regions are determined based on visual features, illumination features, and environmental features. This includes concatenating the visual features, illumination features, and environmental features into vectors to obtain environmental feature vectors corresponding to multiple sub-image regions.

[0081] In this embodiment, after obtaining the visual features, illumination features, and environmental features corresponding to multiple sub-image regions, the visual features, illumination features, and feature vectors related to the global environment extracted from the sub-image regions can be concatenated to obtain the environmental feature vectors corresponding to the multiple sub-image region molecules.

[0082] For example, the visual feature vector corresponding to visual features, the illumination feature vector corresponding to illumination features, and the environment feature vector corresponding to environmental features can be concatenated in a certain order to form a representation in a high-dimensional vector space. For instance, assuming the visual feature vector is 768-dimensional, the illumination feature is 5-dimensional, and the environment feature vector is also 768-dimensional, then the final environment feature vector will be 768 + 5 + 768 = 1541-dimensional.

[0083] Optionally, before concatenating the feature vectors, it is usually necessary to preprocess each feature, such as normalization, to ensure that different features are compared on the same scale and to avoid the numerical range of certain features having a dominant influence on the final result.

[0084] Optionally, vector concatenation allows the environmental feature vector to contain comprehensive information about the sub-image region, including not only the visual features of the sub-image region itself, but also its lighting conditions and the state of the global environment. This fusion of multimodal information can assess the degree of anomaly in the sub-image region across different dimensions, thereby more accurately identifying which regions or scenes are marginal.

[0085] In this step, a multi-dimensional feature representation is generated for each sub-image region by stitching together visual, lighting, and environmental features. This representation not only captures the characteristics of the local region but also considers its position and state in the global environment, providing a solid foundation for subsequent isolation degree calculation and marginality evaluation.

[0086] As an optional implementation, step S104, based on the environmental feature vector, determines the isolation degree evaluation result corresponding to each of the multiple sub-image regions, including: determining the Euclidean distance between the environmental feature vector of the target sub-image region and the environmental feature vectors of the sub-image regions other than the target sub-image region in the multiple sub-image regions, obtaining multiple Euclidean distances, wherein the target sub-image region is any sub-image region among the multiple sub-image regions; and determining the isolation degree evaluation result corresponding to the target sub-image region based on the average Euclidean distance corresponding to the multiple Euclidean distances.

[0087] In this embodiment, after determining the environmental feature vectors corresponding to each of the multiple sub-image regions, the isolation evaluation results for each sub-image region can be determined based on these environmental feature vectors. Here, we take any one of the multiple sub-image regions as an example for illustration; for ease of explanation, this any one sub-image region is referred to as the target sub-image region. For the target sub-image region, the Euclidean distance between its environmental feature vector and the environmental feature vectors of the other sub-image regions (excluding the target sub-image region) is calculated. This Euclidean distance is a mathematical expression of the straight-line distance between two vectors and is suitable for distance metrics in high-dimensional spaces.

[0088] Optionally, after obtaining the Euclidean distance between the environmental feature vector of the target sub-image region and the environmental feature vectors of other sub-image regions besides the target sub-image region, the average of these multiple Euclidean distances can be further calculated to obtain the average Euclidean distance between the environmental feature vector of the target sub-image region and the environmental feature vectors of other sub-image regions. This average Euclidean distance reflects the average degree of difference between the target sub-image region and other sub-image regions in multimodal features and is an important indicator for measuring isolation.

[0089] Optionally, the average Euclidean distance can be used as the isolation score of the target sub-image region, i.e., the isolation assessment result. A higher isolation score indicates a greater difference in multimodal features between the target sub-image region and other sub-image regions, meaning a higher degree of isolation for the target sub-image region. Following this method, the isolation assessment result for each sub-image region can be determined.

[0090] In this step, the isolation degree assessment result of each sub-image region is determined by calculating the average Euclidean distance, which provides a key quantitative indicator for the edge assessment of the scene image in subsequent steps.

[0091] As an optional implementation, step S105, based on the isolation degree evaluation results corresponding to each of the multiple sub-image regions, determines the edge evaluation index corresponding to the scene image, including: arranging the multiple sub-image regions in descending order according to their isolation degree to obtain a sorting queue; selecting the top target number of sub-image regions from the sorting queue; determining the average isolation degree corresponding to the target number of sub-image regions; and determining the edge evaluation index corresponding to the scene image based on the average value.

[0092] In this embodiment, when determining the marginality evaluation index of the scene image based on the isolation degree evaluation results corresponding to each of the multiple sub-image regions, the multiple sub-image regions can be arranged in descending order according to their respective isolation degree (or isolation degree score) to obtain a sorting queue. That is, the isolation degree of the sub-image regions ranked earlier in the sorting queue is higher than that of the sub-image regions ranked later.

[0093] Optionally, after obtaining the sorted queue, a top-target number of sub-image regions can be selected from the queue. The selection of the target number should be based on experimental verification and business needs, determining the degree of understanding of the overall anomalies of the scene. For example, selecting the top 5% or the top 10 sub-regions can ensure that the most challenging local conditions in the scene are focused on, even if these conditions account for a small percentage of the whole image, they may still have a significant impact on the autonomous driving system.

[0094] Optionally, after selecting the target number of sub-image regions, the average isolation level of the selected sub-image regions (i.e., the average isolation score) can be calculated. This average not only quantifies the anomalous level of the selected sub-image regions but also reflects the overall edge strength of the scene image by combining the scores of multiple sub-image regions. A higher average isolation score indicates more or more significant "edge" or "anomalous" factors in the scene.

[0095] Optionally, the edge evaluation index of the scene image can be determined based on the average value of the calculated isolation level. This edge evaluation index is a quantitative standard for measuring whether a scene belongs to an edge driving scenario. The higher the value of the edge evaluation index of the scene image, the more the scene deviates from the norm in terms of visual semantics, lighting conditions, or environmental conditions, which places higher demands on the perception capabilities of the autonomous driving system.

[0096] This step transforms the assessment from local anomaly detection to a global evaluation of the edge characteristics of the entire scene image. This evaluation method effectively overcomes the limitations of edge scene recognition based on the entire image, capturing the complexity and details within the scene, especially critical local areas that may cause performance degradation or safety threats to autonomous driving systems. Furthermore, by setting flexible target numbers and thresholds, the edge scene selection criteria can be automatically adjusted according to different testing and optimization needs.

[0097] As an optional implementation, the edge degree of the driving scene to be identified is positively correlated with the isolation degree of each of the multiple sub-image regions.

[0098] In this embodiment, the degree of isolation (or "isolation score") refers to the uniqueness and anomalousness of a sub-image region in the multimodal feature space. The higher the degree of isolation of a sub-region, the less common its features are in the dataset, or the greater the difference from the normal scene.

[0099] Optionally, the edge severity of the driving scene to be identified is proportional to the isolation level of the sub-image regions. This means that if multiple sub-regions in the scene exhibit high isolation scores, the entire scene is more likely to be classified as an edge driving scene. This is because edge driving scenes often contain multiple local anomalies that are significantly different from normal driving conditions, such as road signs in inclement weather, pedestrians in low light conditions, and vehicle reflections in rainy nights. These anomalies are not only visual (e.g., color, shape, texture) but may also appear in lighting conditions or environmental conditions (e.g., day or night, sunny or rainy / snowy).

[0100] Optionally, by establishing a positive correlation between the isolation degree of the sub-image region and the edge degree of the driving scene to be identified, edge scene evaluation from local to global is realized. Without the need for manually predefined rules or a large amount of labeled data, the system can automatically identify and evaluate various unknown or complex edge conditions. This not only overcomes the limitations of traditional methods, but also provides richer and more realistic data support for autonomous driving systems, which helps to improve their perception capabilities and safety in complex environments.

[0101] The above technical solutions of the embodiments of this application will be further illustrated below with reference to preferred embodiments.

[0102] In related technologies, identifying edge cases relies on the knowledge and experience of experts to manually select edge cases from massive datasets, or to filter data using predefined rules (such as weather and lighting conditions). The advantage of this method is that experts can identify potentially dangerous scenarios based on intuition and experience. However, manual screening is time-consuming and labor-intensive, cannot handle large-scale datasets, is extremely inefficient, and the identification process is influenced by personal experience and biases, potentially leading to the omission of some edge cases or novel extreme scenarios.

[0103] Alternatively, supervised learning methods are also employed in related technologies to automatically identify scene types by training a classification model. This method requires a large amount of labeled data, with each image labeled as either "normal" or "edge-shaped," especially covering various edge conditions. Although supervised learning can provide high recognition accuracy, the cost of acquiring labeled data is high, and the trained model can only identify scene types that have appeared in the training set. For unseen edge conditions, the recognition ability is greatly reduced, easily leading to false negatives.

[0104] Optionally, related technologies also utilize simulation techniques to generate edge scene data to address the technical problem of data scarcity. However, simulation data differs from real-world data in terms of texture, lighting, and environment distribution, which may lead to performance degradation of the model in real-world scenarios. Moreover, simulation data cannot fully reproduce all the complexities and uncertainties of the real world, especially since rare events may be ignored in the simulation environment.

[0105] To address the technical challenges in identifying edge driving scenes in related technologies, this application provides a method for identifying edge driving scenes based on unsupervised learning and multimodal feature fusion. Through semantic segmentation, the scene image is divided into multiple sub-image regions containing different entities (e.g., vehicles, pedestrians, road markings), achieving a deep understanding of scene details. Furthermore, the features of each sub-image region are captured from multiple angles, including its visual semantics, lighting conditions, and global environmental context, forming a comprehensive multimodal feature representation. Using unsupervised algorithms, the "isolation degree" of each sub-image region's features in the high-dimensional feature space—that is, the degree of anomaly relative to other data—can be automatically calculated without any predefined labels or manual intervention. Finally, by aggregating the isolation degree scores of the sub-image regions, an edge evaluation index for the scene image is derived. This index accurately identifies whether the driving scene to be identified is an edge driving scene, improving the accuracy of scene identification, enhancing the adaptability and safety of the autonomous driving system to complex driving environments, reducing reliance on expensive labeled data, and strengthening the system's generalization ability.

[0106] Figure 2 This is a flowchart of a method for recognizing edge driving scenes according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps.

[0107] Step S201: Environment initialization and input data preparation.

[0108] This embodiment primarily involves setting up the computing environment and loading pre-trained models. This includes, but is not limited to, configuring computing devices such as Graphics Processing Units (GPUs), Compute Unified Device Architecture (CUDA), importing necessary Python libraries, and preloading semantic segmentation models (e.g., Mask2Former) and visual language models (e.g., CLIP). Simultaneously, it defines the paths to input image folders and output folders, as well as a predefined list of text prompts describing the environment, providing initial preparation for subsequent data processing.

[0109] Step S202, semantic segmentation and region instantiation.

[0110] In this embodiment, all images in the input folder are traversed, and semantic segmentation is performed on each image, parsing it into multiple structured semantic regions, such as vehicles, pedestrians, and roads. Then, through connected component analysis, different instances within the same category can be further distinguished, such as multiple cars or multiple pedestrians. To enhance processing accuracy, regions with excessively small areas can be filtered out to eliminate noise, retaining only the effective regions and recording their key metadata: the filename of the image to which it belongs, the category identifier (ID), the instance ID, and a pixel-level binary mask, providing fine-grained localization for subsequent feature extraction.

[0111] For example, for each image, a segmentation preprocessor is used for preprocessing. This step may include resizing the image, normalizing pixel values, and converting color channels, with the aim of converting the image into a standard format suitable for model input. Subsequently, the image is fed into a pre-trained semantic segmentation model for inference. This model can identify different objects and environmental elements in the image, such as vehicles, pedestrians, and roads, and annotate the segmented image with masks of different colors or values.

[0112] Optionally, since an image may contain multiple cars or pedestrians, these objects, although belonging to the same category, should be treated as different instances in the actual scene. Therefore, the `skimage.measure.label` function is applied for connected component analysis, which can identify and label continuous object regions based on the category IDs of adjacent pixels. By setting a certain area threshold (e.g., 300 pixels), the system can filter out regions that are too small, may be noise, or artifacts, ensuring that only meaningful object instances remain. For each valid region that passes the area filtering, the system records detailed metadata, including the original image filename to which the region belongs, the category ID (indicating the object type), the instance ID (distinguishing different objects within the same category), and a binary mask covering the region. The latter is used in subsequent processes to accurately extract region features from the original image.

[0113] Step S203: Multimodal feature extraction and fusion.

[0114] In this embodiment, the multimodal features include visual features, illumination features, and global environment features. For visual features, specific objects, such as vehicles or pedestrians, are cropped from the original image using region bounding boxes. Then, region masking is used to set non-target areas to black, thus generating clean sub-images containing only the target region. Inputting these sub-images into CLIP's visual encoder yields a 768-dimensional feature vector that highly summarizes the visual semantic information of the region, encoding visual attributes such as the object's shape, color, and texture.

[0115] Optionally, for illumination features, the sub-image is converted to grayscale, and then a series of illumination statistics are calculated within the defined range of the region mask, including local brightness, brightness standard deviation, maximum brightness, surrounding average brightness, and backlight score. These statistics together constitute a 5-dimensional handcrafted feature vector, which, after L2 normalization, can standardize the illumination characteristics of the region, helping to identify scenes with abnormal illumination.

[0116] Optionally, for global environment features, the entire original image is fed into the CLIP model for encoding, obtaining a 768-dimensional global image feature. Then, using predefined environment cues, corresponding feature vectors are generated. Cosine similarity is calculated to find the environment description that best matches the global image feature; this description's feature vector is selected as the global environment feature. The introduction of this feature not only enriches the understanding of the scene environment but also provides additional contextual information for the features of each sub-region, enhancing the model's ability to recognize complex edge conditions.

[0117] Optionally, after obtaining the visual features, illumination features, and global environment features of each sub-image, the visual semantic features, illumination statistical features, and global environment feature vectors can be concatenated and fused to form a 1541-dimensional comprehensive feature representation. This process ensures that the features of each semantic region are not only isolated visual information but also contain rich details of illumination conditions and environmental context. The 1541-dimensional feature vector provides a comprehensive description for subsequent unsupervised learning algorithms, enabling the system to more accurately identify and evaluate edge driving scenarios and effectively capture anomalies even in complex and changing environments, thereby improving the robustness and safety of autonomous driving.

[0118] Step S204, Regional-level marginality measurement.

[0119] In this embodiment, after the extraction and fusion of multimodal features, the next stage is the construction of the feature matrix. The task of this stage is to integrate the 1541-dimensional feature vectors corresponding to all semantic regions in all images to form a large two-dimensional feature matrix. Each row of the matrix represents the features of a region, and the columns correspond to the various dimensions of the feature vectors. This matrix form not only facilitates storage and processing, but more importantly, it provides a unified feature space for the subsequent unsupervised K-nearest neighbor algorithm, enabling the algorithm to find neighboring regions based on multimodal features and thus calculate the isolation degree of each region.

[0120] Optionally, the K-Nearest Neighbors (K-NN) algorithm from unsupervised learning was used to calculate isolation degree, a metric that quantifies the degree to which a region is isolated from other regions in the feature space. For example, for each region in the feature matrix, the K-NN algorithm finds its 2nd to K+1th nearest neighbor regions and calculates the average Euclidean distance from that region to these neighbors. This average distance reflects the "proximity" of the current region to other regions in the feature space; in other words, a lack of "nearest neighbors" indicates that the feature combination of that region is relatively rare in the dataset, possessing a certain degree of "abnormality" or "isolation." The average distance calculated above is used as the isolation degree score for that region. The higher the isolation degree score, the more isolated the region is in the feature space, and the greater the feature difference from other regions; therefore, it is more likely to belong to the edge driving scene. The calculation of the isolation degree score provides an important basis for subsequent image-level edge assessment.

[0121] Step S205: Image-level score aggregation and sorting.

[0122] In this embodiment, after scoring the isolation degree of each semantic region in the image, these local scores can be aggregated into a comprehensive edge assessment value for each image. Considering that edge scenes often contain multiple anomalous regions, a strategy is adopted: selecting the top 5 regions with the highest isolation degree scores in each image, calculating the average score of these regions, and using this average as the edge assessment value for that image. This strategy of taking the highest average ensures that even if only a few regions in the image are anomalous, they can still be identified as potential edge scenes.

[0123] For example, after obtaining the extreme scores of all images, they are sorted in descending order based on these scores, meaning images with higher scores are considered more marginal or extreme. This sorting visually identifies which images are at the forefront of anomalous states in the grading process. Then, following a predetermined strategy, such as selecting the Top-60 images, these top-ranked images are identified as the most marginal scenes discovered. This selection method ensures a focus on images that statistically deviate significantly from the normal distribution, ensuring that the dataset contains the most challenging driving situations, which is crucial for improving the safety and robustness of autonomous driving systems.

[0124] Step S206: Output and visualization of results.

[0125] In this embodiment, after feature extraction, edge assessment, and ranking are completed, the most "edge" images are saved to a designated output directory and renamed in the format "rank_score.jpg". This naming convention concisely reflects the relative importance and degree of anomalousness of the image within its category. Through this intuitive filename identification, researchers can quickly locate and review these edge scenarios, assess their impact on the perception capabilities of the autonomous driving system, and thus optimize the model accordingly to enhance its performance in complex environments.

[0126] In steps S201 to S206 above, the invention employs a semantic segmentation model to achieve pixel-level image parsing, decomposing complex scenes into multiple sub-regions focusing on specific entities (vehicles, pedestrians, road surfaces, etc.). Next, the CLIP model extracts the visual semantic features and global environmental features of each sub-region, while simultaneously manually calculating illumination statistical features, forming a comprehensive multimodal feature representation system. The K-NN algorithm is used to quantify the isolation degree of each region, i.e., its degree of anomaly relative to the overall dataset. This metric is not constrained by prior knowledge and can identify new or complex edge conditions. Finally, by aggregating and ranking the isolation degree scores of regions, the Top-N edge images are selected to construct a high-quality edge scene dataset. This not only significantly reduces the reliance on expensive labeled data and expert intervention, improving data processing efficiency and generalization ability, but also ensures that the autonomous driving model can maintain stable performance when facing unknown and edge environments.

[0127] Figure 3 This is a flowchart of an edge scene data construction method according to an embodiment of this application, such as... Figure 3As shown, a filtered image set is obtained, covering various driving environments of vehicles. After obtaining the image set, it can be input into a semantic segmentation model for segmentation. For the mask of each category in the segmentation image, connected component analysis is performed using `skimage.measure.label` to identify different instances within the same category (e.g., multiple cars, multiple pedestrians). An area threshold (e.g., 300 pixels) is set to filter out excessively small regions to eliminate noise interference. Meta-information is recorded for each valid region: the image file name, category ID, instance ID, and its pixel-level binary mask. Next, region feature extraction is performed, including semantic features, illumination features, and environmental perception features. Multimodal features are constructed based on these features to obtain the multimodal features of each region. Then, based on these multimodal features, the edge score of the region is calculated, and image-level score aggregation is performed. Finally, based on the aggregation results, edge image filtering is performed to obtain edge image data.

[0128] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0129] According to an embodiment of this application, an embodiment of a vehicle edge driving scene recognition device is provided. It should be noted that the device can be used to perform the above-described vehicle edge driving scene recognition method.

[0130] Figure 4 This is a schematic diagram of a vehicle edge driving scene recognition device according to an embodiment of this application. Figure 4 As shown, the vehicle edge driving scene recognition device 400 may include: an acquisition unit 401, a segmentation unit 402, a first determination unit 403, a second determination unit 404, a third determination unit 405, and a fourth determination unit 406.

[0131] The acquisition unit 401 is used to acquire scene images of the driving scene to be identified for the vehicle.

[0132] The segmentation unit 402 is used to call the semantic segmentation model to segment the scene image into multiple sub-image regions, wherein any sub-image region includes a semantic feature in the scene image.

[0133] The first determining unit 403 is used to determine the environmental feature vectors corresponding to each of the multiple sub-image regions, wherein the environmental feature vectors are used to characterize the features of the sub-image regions in relation to the global environment.

[0134] The second determining unit 404 is used to determine the isolation degree evaluation results corresponding to each of the multiple sub-image regions based on the environmental feature vector, wherein the isolation degree evaluation results are used to characterize the degree of isolation of the sub-image regions.

[0135] The third determining unit 405 is used to determine the edge evaluation index corresponding to the scene image based on the isolation degree evaluation results corresponding to each of the multiple sub-image regions. The edge evaluation index is used to characterize the edge degree of the driving scene to be identified.

[0136] The fourth determining unit 406 is used to determine the driving scene to be identified as an edge driving scene in response to the edge evaluation index indicating that the edge degree of the driving scene to be identified is greater than or equal to the edge degree threshold.

[0137] Optionally, the segmentation unit 402 is further configured to: invoke a semantic segmentation model; input the scene image into the semantic segmentation model to extract multiple semantic regions in the scene image, wherein any semantic region includes a semantic element in the scene image; and perform connected component analysis on the multiple semantic regions to obtain multiple sub-image regions.

[0138] Optionally, the segmentation unit 402 is also used to: connect semantic regions that include the same semantic elements in multiple semantic regions to obtain multiple sub-image regions.

[0139] Optionally, the first determining unit 403 is further configured to: extract visual features, illumination features, and environmental features corresponding to each of the multiple sub-image regions, wherein the visual features are used to represent the visual attributes of the sub-image regions, the illumination features are used to characterize the illumination level of the sub-image regions, and the environmental features are used to characterize the environmental conditions corresponding to the sub-image regions; and determine the environmental feature vectors corresponding to each of the multiple sub-image regions based on the visual features, illumination features, and environmental features.

[0140] Optionally, the first determining unit 403 is further configured to: perform vector concatenation of visual features, illumination features and environmental features to obtain environmental feature vectors corresponding to each of the multiple sub-image regions.

[0141] Optionally, the second determining unit 404 is further configured to: determine the Euclidean distance between the environmental feature vector of the target sub-image region and the environmental feature vectors of the sub-image regions other than the target sub-image region in the plurality of sub-image regions, and obtain a plurality of Euclidean distances, wherein the target sub-image region is any sub-image region among the plurality of sub-image regions; and determine the isolation degree evaluation result corresponding to the target sub-image region based on the average Euclidean distance corresponding to the plurality of Euclidean distances.

[0142] Optionally, the third determining unit 405 is further configured to: sort multiple sub-image regions in descending order according to their degree of isolation to obtain a sorting queue; select the top target number of sub-image regions from the sorting queue; determine the average degree of isolation corresponding to the target number of sub-image regions; and determine the edge evaluation index corresponding to the scene image based on the average value.

[0143] Optionally, the edge severity of the driving scene to be identified is positively correlated with the degree of isolation of each of the multiple sub-image regions.

[0144] In the edge driving scene recognition device for vehicles described in this application, semantic segmentation technology is introduced to decompose complex scene images into multiple sub-image regions containing specific semantic features (e.g., vehicles, pedestrians, road markings). This fine-grained segmentation strategy greatly enriches the understanding of scene details. Subsequently, an environmental feature vector is constructed for each sub-image region, revealing the uniqueness of the sub-image region in the global environment. The isolation degree evaluation results of each sub-image region, calculated through an unsupervised learning process, quantify the edge degree of each sub-region. Based on the isolation degree evaluation results corresponding to each of the multiple sub-image regions, an edge evaluation index for the entire scene image can be determined. This edge evaluation index can quantify the edge degree of the driving scene to be recognized, and accurately determine whether the scene to be recognized is an edge driving scene. In other words, this application, through an automated and unsupervised learning process, can not only recognize known edge scenes but also discover unknown edge situations, providing stronger support for autonomous driving systems facing complex and ever-changing real driving environments. This solves the technical problem of incomplete recognition of vehicle edge driving scenes and lays a solid foundation for improving the safety and reliability of autonomous driving.

[0145] Embodiments of this application also provide a vehicle, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods described in various embodiments of this application when it runs.

[0146] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0147] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0148] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0149] Embodiments of this application also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of this application.

[0150] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0152] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0153] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0154] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0155] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for recognizing the edge driving scene of a vehicle, characterized in that, include: Acquire scene images of the vehicle's driving scenario to be identified; A semantic segmentation model is invoked to segment the scene image into multiple sub-image regions, wherein any one of the sub-image regions includes a semantic feature in the scene image; Determine the environmental feature vector corresponding to each of the plurality of sub-image regions, wherein the environmental feature vector is used to characterize the features of the sub-image region in relation to the global environment; Based on the environmental feature vector, the isolation degree evaluation result corresponding to each of the plurality of sub-image regions is determined, wherein the isolation degree evaluation result is used to characterize the degree of isolation of the sub-image region; Based on the isolation degree evaluation results corresponding to each of the multiple sub-image regions, the edge evaluation index corresponding to the scene image is determined, wherein the edge evaluation index is used to characterize the edge degree of the driving scene to be identified. In response to the edge evaluation index indicating that the edge degree of the driving scene to be identified is greater than or equal to the edge degree threshold, the driving scene to be identified is determined to be an edge driving scene.

2. The method according to claim 1, characterized in that, The semantic segmentation model is invoked to segment the scene image into multiple sub-image regions, including: Invoke the semantic segmentation model; The scene image is input into the semantic segmentation model to extract multiple semantic regions from the scene image, wherein any one of the semantic regions includes a semantic element in the scene image; Connectivity analysis is performed on the multiple semantic regions to obtain the multiple sub-image regions.

3. The method according to claim 2, characterized in that, Connectivity analysis is performed on the multiple semantic regions to obtain the multiple sub-image regions, including: By connecting the semantic regions that include the same semantic element, the plurality of sub-image regions are obtained.

4. The method according to claim 1, characterized in that, Determining the environmental feature vectors corresponding to each of the plurality of sub-image regions includes: Extract the visual features, illumination features, and environmental features corresponding to each of the multiple sub-image regions, wherein the visual features are used to represent the visual attributes of the sub-image region, the illumination features are used to characterize the illumination level of the sub-image region, and the environmental features are used to characterize the environmental conditions corresponding to the sub-image region. Based on the visual features, the illumination features, and the environmental features, the environmental feature vector corresponding to each of the plurality of sub-image regions is determined.

5. The method according to claim 4, characterized in that, Based on the visual features, the illumination features, and the environmental features, the environmental feature vector corresponding to each of the plurality of sub-image regions is determined, including: The visual features, the illumination features, and the environmental features are concatenated into vectors to obtain the environmental feature vectors corresponding to each of the multiple sub-image regions.

6. The method according to claim 1, characterized in that, Based on the environmental feature vector, the isolation degree evaluation results corresponding to each of the multiple sub-image regions are determined, including: The environmental feature vector of the target sub-image region is determined, and the Euclidean distance between it and the environmental feature vector of the sub-image regions other than the target sub-image region in the plurality of sub-image regions is obtained, wherein the target sub-image region is any sub-image region among the plurality of sub-image regions; Based on the average Euclidean distance corresponding to multiple Euclidean distances, the isolation degree evaluation result corresponding to the target sub-image region is determined.

7. The method according to claim 1, characterized in that, Based on the isolation degree evaluation results corresponding to each of the multiple sub-image regions, the edge evaluation index corresponding to the scene image is determined, including: According to the degree of isolation, the multiple sub-image regions are arranged in descending order to obtain a sorting queue; The first target number of the sub-image regions are selected from the sorting queue; Determine the average degree of isolation corresponding to the target number of the sub-image regions; Based on the average value, the edge evaluation index corresponding to the scene image is determined.

8. The method according to any one of claims 1 to 7, characterized in that, The edge degree of the driving scene to be identified is positively correlated with the isolation degree of each of the multiple sub-image regions.

9. A vehicle, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the storage medium is located to perform the method according to any one of claims 1 to 8.

Images