An adaptive spatial layered image enhancement method and system for autonomous driving

Abstract

The application discloses an adaptive spatial hierarchical image enhancement method and system for automatic driving, and belongs to the technical field of image data processing. The method comprises the following steps: acquiring a front view image sample; extracting a shallow feature map retaining a spatial position structure by using a shallow feature extraction network; performing spatial longitudinal aggregation processing on the shallow feature map to obtain a normalized longitudinal response vector and calculate a cumulative distribution function; determining the position of a hierarchical boundary in a shallow feature map coordinate system according to the cumulative distribution function based on a preset hierarchical threshold and mapping the position to an original image coordinate system; dividing the image into multiple spatial hierarchical regions along the vertical direction according to the mapped hierarchical boundary; applying a differential enhancement operation to each region; and outputting the enhanced sub-image after splicing and fusing. Based on the method, an adaptive spatial hierarchical image enhancement system for automatic driving is further provided. The application improves the learning ability of an automatic driving visual perception model for different scale targets and complex scenes.

Description

Technical Field

[0001] This invention belongs to the field of image data processing technology, and in particular relates to an adaptive spatial layering image enhancement method and system for autonomous driving. Background Technology

[0002] Autonomous vehicles rely on various sensors for environmental perception, among which camera-based visual perception systems play a crucial role in tasks such as object detection, lane line recognition, and drivable area segmentation. In recent years, deep learning methods have been widely applied to training visual perception models for autonomous driving; however, model performance largely depends on the diversity and distribution quality of the training data. To improve model generalization ability, researchers often employ data augmentation techniques to randomly transform training images, such as random cropping, color perturbation, and occlusion simulation. However, existing data augmentation methods typically apply a uniform augmentation operation to the entire image, without considering the differences in image spatial structure within autonomous driving scenarios.

[0003] In images from forward-looking cameras used in autonomous driving, different vertical regions typically correspond to different physical distances and semantic information due to perspective projection. For example, the upper region of the image usually corresponds to the distant background and sky; the middle region mainly contains vehicles and traffic participants at medium to long distances; and the lower region typically contains roads, vehicles, and pedestrians at close range. Different regions exhibit significant differences in their sensitivity to data augmentation: small targets in medium to long distance regions are more sensitive to scale changes, while close-range regions are more susceptible to occlusion. Therefore, adopting a uniform augmentation strategy may not fully utilize the spatial structure information of the image, thus limiting model performance improvement. Furthermore, while some existing methods attempt to divide the image into regions using a fixed ratio, this approach cannot adapt to perspective changes in different scenes, such as ramps, changes in camera pitch angle, or changes in traffic density, thereby affecting the effectiveness of the augmentation strategy.

[0004] Therefore, there is an urgent need for a data augmentation method that can adaptively determine the augmentation region based on the image content in order to improve the training effect of autonomous driving vision models. Summary of the Invention

[0005] To address the aforementioned technical challenges, this invention proposes an adaptive spatially layered image enhancement method and system for autonomous driving. By analyzing the spatial structure of the image to dynamically generate enhancement region boundaries and employing differentiated enhancement strategies for different regions, the model's learning ability for targets at different scales and complex scenes is improved.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, this invention proposes an adaptive spatial layered image enhancement method for autonomous driving, comprising the following steps: Obtain a front view image sample of the target vehicle in an autonomous driving scenario; use a shallow feature extraction network to extract features from the front view image sample to obtain a shallow feature map; the shallow feature map is a feature map that retains the spatial position structure and is output by the network layer near the input end in the shallow feature extraction network. The shallow feature map is subjected to spatial vertical aggregation to obtain a normalized vertical response vector that reflects the structural response intensity at different height positions of the image, and the cumulative distribution function is calculated based on the normalized vertical response vector. Based on multiple preset layer thresholds, the positions of multiple layer boundaries in the shallow feature map coordinate system are determined according to the cumulative distribution function, and the multiple layer boundaries are mapped to the front view image sample coordinate system. Based on the mapped layer boundaries, the front view image sample is divided into multiple spatial layer regions along the vertical direction; a differential enhancement operation matching the region is applied to each spatial layer region to obtain an enhanced sub-image of each region; the enhanced sub-images of each region are stitched and fused in the original spatial order to output the final enhanced image.

[0007] Furthermore, the shallow feature extraction network is: the first two convolutional stages of ResNet, the shallow backbone stage of CSPDarkNet, or the first stage of Swin Transformer.

[0008] Furthermore, the process of performing spatial vertical aggregation on the shallow feature map to obtain a normalized vertical response vector reflecting the structural response intensity at different height positions of the image includes: Let the shallow feature map be represented as: ;in, Indicates the number of channels; Indicates altitude; Indicates width; Spatial vertical aggregation processing is as follows: for the shallow feature map In the channel dimension and width dimension Pooling is performed on the top layer to obtain the initial longitudinal response vector: , ; For the initial longitudinal response vector The normalized longitudinal response vector is obtained by performing normalization: , ; in, This represents the normalized longitudinal response vector.

[0009] Furthermore, the cumulative distribution function is calculated based on the normalized longitudinal response vector, specifically as follows: , ; in, This indicates the sequence from the top row 1 to the top row 2 of the feature map. The cumulative response value of the row.

[0010] Furthermore, based on multiple preset layer thresholds, the positions of multiple layer boundaries in the shallow feature map coordinate system are determined according to the cumulative distribution function, and the multiple layer boundaries are mapped to the front view image sample coordinate system, specifically: The preset multiple stratification thresholds include a first preset threshold. Second preset threshold ,and ; The vertical position of the multiple layer boundaries in the shallow feature map coordinate system and , is represented as: ; ; The multiple layer boundaries are then mapped to the front view image sample coordinate system, and represented as follows: ; ; in, Indicates the height of the front view image sample; This indicates the longitudinal position of the first layer boundary in the coordinate system of the front view image sample; This indicates the longitudinal position of the second layer boundary in the coordinate system of the front view image sample.

[0011] Furthermore, based on the mapped multiple hierarchical boundaries, the front view image sample is divided into multiple spatial hierarchical regions along the vertical direction; specifically: Based on the mapped first layer boundary Second layer boundary The front view image sample is divided into an upper region along the vertical direction. Central region and lower area , is represented as: ; in, Indicates the range from row 0 of the front view image sample to... upper area of ​​the row ; Indicates the first from the front view image sample Arriving Central region of the line ; Indicates the image sample from the front view Arriving lower area of ​​the row .

[0012] Furthermore, a differentiated enhancement operation matching the region is applied to each of the aforementioned spatial hierarchical regions, specifically as follows: A weak enhancement strategy is applied to the upper region to obtain the enhanced upper region. The weak enhancement strategy includes: mild noise, brightness perturbation, brightness adjustment, mild contrast change, and slight blurring; The enhanced central region is obtained by applying a scale transformation enhancement operation to the central region. The scaling enhancement operation involves locally magnifying the central region based on its center and then cropping it back to the original size of the central region. The magnification factor is [missing value]. ,and ; An occlusion simulation enhancement operation is applied to the lower region, wherein the occlusion simulation enhancement operation is to overlay a random binary occlusion mask on the lower region. Generate the enhanced lower region : ; in, This represents a random binary occlusion mask.

[0013] Furthermore, the enhanced sub-images of each region are stitched and fused together in their original spatial order to output the final enhanced image. Specifically, the enhanced upper region is then processed... Enhanced central region and the enhanced lower region The pieces are vertically spliced ​​according to the original spatial order, and the splicing boundaries are smoothed.

[0014] Furthermore, the smoothing process includes: at the first layer boundary Second layer boundary Within the width of the neighboring pixels, the pixel values ​​are transitionally blended using linear interpolation or Gaussian weighted averaging.

[0015] Secondly, this invention also proposes an adaptive spatial layered image enhancement system for autonomous driving, comprising: The feature extraction module is used to acquire front view image samples of the target vehicle in an autonomous driving scenario; and to extract features from the front view image samples using a shallow feature extraction network to obtain a shallow feature map; the shallow feature map is a feature map that retains the spatial position structure and is output by the network layer near the input end of the shallow feature extraction network. The aggregation calculation module is used to perform spatial vertical aggregation processing on the shallow feature map to obtain a normalized vertical response vector that reflects the structural response intensity at different height positions of the image, and to calculate the cumulative distribution function based on the normalized vertical response vector. The boundary generation module is used to determine the positions of multiple layer boundaries in the shallow feature map coordinate system based on multiple preset layer thresholds and the cumulative distribution function, and to map the multiple layer boundaries to the front view image sample coordinate system. The fusion output module is used to divide the front view image sample into multiple spatial hierarchical regions along the vertical direction according to the multiple mapped hierarchical boundaries; apply a differential enhancement operation matching the region to each spatial hierarchical region to obtain the enhanced sub-image of each region; and stitch and fuse the enhanced sub-images of each region in the original spatial order to output the final enhanced image.

[0016] Compared with the prior art, the present invention has the following beneficial effects: (i) Adaptive spatial layering to adapt to changing scenarios Existing data augmentation methods typically use fixed proportions or fixed positions to divide images into regions, which cannot adapt to perspective differences caused by changes in slopes, camera pitch angles, or traffic density in autonomous driving scenarios. This invention uses a shallow feature extraction network to obtain shallow feature maps that preserve spatial location structure, and adaptively generates layered boundaries based on the image's own vertical structure response. This allows the region division to dynamically adjust with the image content, making the division results more consistent with the near-far layer distribution trend in forward-view images under different driving scenarios.

[0017] (ii) Differentiation enhancement strategy to improve model generalization ability Existing methods typically apply a uniform enhancement operation to the entire image, without considering the differentiated sensitivity of different spatial regions to the enhancement operation. This invention employs a weak enhancement strategy to avoid introducing meaningless perturbations, addressing the low information density of the upper far-background region. It also utilizes a scale transformation enhancement strategy to improve the model's perception of small targets in the middle and mid-to-far distance regions, which are sensitive to scale changes. Furthermore, it employs an occlusion simulation enhancement strategy to improve the model's robustness in complex environments, addressing the susceptibility of the lower near-field region to occlusion. Through region-differentiated enhancement, the enhancement operation for each region is matched to its spatial hierarchy characteristics, effectively improving the quality of training data and the enhancement effect on the visual perception model.

[0018] (iii) No additional annotation is required, and the computational cost is low. Existing adaptive methods based on semantic or depth information typically rely on object detection annotations or depth estimation results, requiring additional annotation costs or computational resources. This invention's adaptive hierarchical boundary generation relies solely on shallow image feature responses, requiring no additional annotation information (such as object bounding boxes, semantic segmentation labels, or depth maps), and can be directly integrated into existing visual perception model training processes. Furthermore, shallow feature extraction only requires the outputs of the first few layers of the backbone network, resulting in low computational cost and no increase inference latency.

[0019] (iv) Solve the problem of small target detection and improve perception performance In autonomous driving scenarios, small targets such as vehicles and pedestrians at medium to long distances are difficult to detect due to their small pixel count in images. This invention magnifies the central region (by 1.2 to 1.6 times), allowing small targets to appear in training images at a larger local scale. This effectively enhances the model's feature learning ability for small-scale targets, thereby improving the performance of autonomous driving visual perception systems in target detection tasks.

[0020] (v) Enhance near-field robustness to cope with occlusion interference In close-range autonomous driving scenarios, vehicles and pedestrians are easily obstructed by other objects or interfered with by shadows, localized strong light, etc. This invention enhances occlusion simulation by applying a random binary occlusion mask to the lower region, enabling the model to learn robust representations of local occlusion during the training phase. This allows the model to better cope with occlusion and localized interference in actual deployment, thereby improving the stability and safety of the perception system. Attached Figure Description

[0021] Figure 1 This is a flowchart of an adaptive spatial layering image enhancement method for autonomous driving proposed in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the front view image sample proposed in Embodiment 1 of the present invention; Figure 3 This is a shallow feature map of the front view image sample proposed in Embodiment 1 of the present invention after being extracted by a shallow feature extraction network; Figure 4 This is a schematic diagram of data augmentation on a shallow feature map as proposed in Embodiment 1 of the present invention; Figure 5 This is a schematic diagram illustrating data augmentation implemented using the method proposed in Embodiment 1 of the present invention; Figure 6 This is a schematic diagram of an adaptive spatial layering image enhancement system for autonomous driving proposed in Embodiment 2 of the present invention; Figure 7 This is a schematic diagram of an adaptive spatial layering image enhancement device for autonomous driving proposed in Embodiment 3 of the present invention. Detailed Implementation

[0022] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific embodiments and in conjunction with the accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the invention. To simplify the disclosure of the invention, components and arrangements of specific examples are described below. Furthermore, reference numerals and / or letters may be repeated in different examples. This repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed. It should be noted that the components illustrated in the drawings are not necessarily drawn to scale. Descriptions of well-known components, processing techniques, and processes are omitted in this invention to avoid unnecessarily limiting the invention.

[0023] Example 1 Embodiment 1 of this invention proposes an adaptive spatial layering image enhancement method for autonomous driving. This method addresses the problem that existing image data enhancement methods apply a uniform enhancement operation to the entire image, which fails to adaptively divide and differentiate the image spatial structure due to perspective projection differences in autonomous driving scenarios, thus limiting the training effect of visual perception models.

[0024] Figure 1 This is a flowchart of an adaptive spatial layering image enhancement method for autonomous driving proposed in Embodiment 1 of the present invention; In step S1, a front view image sample of the target vehicle in the autonomous driving scenario is obtained; Figure 2 This is a schematic diagram of the front view image sample proposed in Embodiment 1 of the present invention; In Embodiment 1 of this invention, the acquisition of forward-looking image samples includes: real-time acquisition by an image acquisition device installed on the target vehicle, and acquisition from publicly available autonomous driving datasets, simulation platforms, or offline data acquisition. The image acquisition device includes, but is not limited to, an in-vehicle forward-looking monocular camera, a left or right eye camera in a binocular camera system, and a forward-facing road camera in a multi-camera surround-view system. This invention preferably uses images acquired by a forward-looking monocular camera because forward-looking images most clearly demonstrate the spatial hierarchy caused by perspective projection in autonomous driving scenarios. That is, different positions in the vertical direction of the image correspond to different physical distance ranges in the scene; the upper part of the image corresponds to the distant background, the middle part corresponds to mid-to-long-distance traffic participants, and the lower part corresponds to the near-distance road and vehicles. The forward-looking image sample can be an RGB three-channel color image or a grayscale image, and its resolution is determined by the parameters of the image acquisition device, typically 1920×1080, 1280×720, or similar sizes. Let the acquired forward-looking image sample be denoted as I, and its data form can be represented as: ; in, This indicates the number of channels in the front view image sample; typically, C=3 in an RGB image. This indicates the height of the front view image sample, i.e., the number of pixels in the vertical direction; This represents the width of the front view image sample, i.e., the number of pixels in the horizontal direction.

[0025] In step S2, a shallow feature extraction network is used to extract features from the front view image sample to obtain a shallow feature map; the shallow feature map is the feature map output by the network layer near the input end of the shallow feature extraction network, which retains the spatial position structure. Figure 3 This is a shallow feature map of the front view image sample proposed in Embodiment 1 of the present invention after being extracted by a shallow feature extraction network; After acquiring the front view image sample, a shallow feature extraction network is used to extract features from the front view image sample to obtain a shallow feature map.

[0026] The shallow feature extraction network refers to the early network layer near the input in a convolutional neural network or visual Transformer, which has not yet undergone multiple downsampling and semantic abstraction. Shallow feature extraction networks include: the first two convolutional stages of ResNet, the shallow backbone stage of CSPDarkNet, or the first stage of the Swin Transformer.

[0027] The reasons for choosing shallow features in this invention are as follows: Shallow features preserve spatial relationships more Deep features tend to be more abstract semantics, while shallow features still retain the spatial structure of the original image better.

[0028] (2) Shallow features are more sensitive to local structures For example, road edges, lane lines, vehicle outlines, and pedestrian boundaries can all help determine which vertical areas are more "active".

[0029] (3) Shallow features have low computational cost If features are output directly from the first few backbone layers, there is no need to add a complex network structure.

[0030] In specific embodiments of the present invention, the shallow feature extraction network can be flexibly selected according to task requirements and computational resources. For example, for a convolutional neural network-based architecture, the outputs of the first two convolutional stages of ResNet (i.e., the first and second convolutional blocks) can be used as the shallow feature map, where the downsampling factor of the feature map is typically 4 times (i.e., ...). =H / 4, =W / 4); Alternatively, the shallow backbone stage output of CSPDarkNet can be used. This network structure is widely used in object detection tasks, and its shallow features also retain good spatial location information. For Transformer-based architectures, the first-stage window attention output of the Swin Transformer can be used as the shallow feature map. The feature map output at this stage also maintains relatively high spatial resolution and can reflect the local structure distribution of the image. The specific selection of the above feature extraction networks are all known techniques in the art. Those skilled in the art can select suitable shallow feature extraction networks to implement this invention without creative effort, based on the guidance of this invention.

[0031] Let the shallow feature map be represented as: ;in, Indicates the number of channels; Indicates the height of the shallow feature map; This indicates the width of the shallow feature map.

[0032] Height of shallow feature map and width Typically smaller than the height H and width W of the original image, but with a lower downsampling ratio (usually 2 to 8 times), it still retains rich spatial location and structural information. Compared with deep semantic features, shallow feature maps do not involve the discrimination of high-level semantic categories such as "vehicles," "pedestrians," and "traffic signs," but only represent low-level visual structures in the image such as edges, textures, and local gradients. This makes subsequent region boundary determination based on feature responses unaffected by the distribution of semantic categories, resulting in better scene generalization ability.

[0033] The obtained shallow feature map F is used as input data for subsequent spatial vertical aggregation processing to calculate the structural response intensity at different height positions of the image, thereby generating adaptive layer boundaries.

[0034] In step S3, the shallow feature map is subjected to spatial vertical aggregation processing to obtain a normalized vertical response vector that reflects the structural response intensity at different height positions of the image. Figure 4 This is a schematic diagram of data augmentation on a shallow feature map as proposed in Embodiment 1 of the present invention; The purpose of spatial vertical aggregation is to compress information from a three-dimensional shallow feature map (channel dimension, height dimension, and width dimension) along the channel and width dimensions, retaining only the spatial distribution information of the height dimension, thereby obtaining a one-dimensional vertical response vector. Each element in this response vector corresponds to the overall structural response intensity at a certain height position (i.e., a certain row) in the feature map, reflecting the richness and activity of the visual structure within the corresponding horizontal strip region in the original image. By analyzing the cumulative distribution of this response vector in the vertical direction, the layer boundaries of different spatial levels in the image can be adaptively determined.

[0035] Spatial vertical aggregation processing is as follows: for the shallow feature map In the channel dimension and width dimension Pooling is performed on the top layer to obtain the initial longitudinal response vector: , ; s(y) is a one-dimensional vector of length H′, representing the average feature response at each height position from the top to the bottom of the image.

[0036] After obtaining the initial longitudinal response vector s(y), it is normalized to obtain the normalized longitudinal response vector. The purpose of normalization is to eliminate the differences in the absolute value of the response between different images caused by global factors such as illumination, exposure, and contrast, so that the subsequent thresholding has consistency across images.

[0037] For the initial longitudinal response vector The normalized longitudinal response vector is obtained by performing normalization: , ; in, This represents the normalized longitudinal response vector. In this step, besides simple average pooling, methods such as maximum response, L2 energy, and weighted averaging can also be used.

[0038] Normalized longitudinal response This vector serves as the foundation for subsequent calculations of the cumulative distribution function. It reflects the distribution of structural response intensity at various heights from the top to the bottom of the image. When the proportion of structural response changes significantly within a certain height range, it indicates the existence of a spatial hierarchy transition boundary, thus providing a reliable basis for adaptive stratification.

[0039] In step S4, the cumulative distribution function is calculated based on the normalized longitudinal response vector. The cumulative distribution function is used to characterize the cumulative structural response intensity from the top of the shallow feature map to a certain height position, and its value reflects the proportion of the region above that height in the entire image's longitudinal structural response.

[0040] Specifically: , ; in, This indicates the sequence from the top row 1 to the top row 2 of the feature map. The cumulative response value of the row. It is a monotonically non-decreasing function with a range of [0,1], and (The cumulative response count for line 0 is 0) (The total sum of responses for all rows is 1).

[0041] The cumulative distribution function (CDF) transforms the normalized vertical response vector from a probability density form to a probability distribution form. Its physical meaning is as follows: when the CDF value of a row increases rapidly, it indicates that the normalized response value near that row is large, meaning that the horizontal position contains rich visual structural information; when the CDF value of a row changes gradually, it indicates that the response value near that row is small, meaning that the visual structure at that horizontal position is relatively sparse. By analyzing the variation of the CDF with height y, the transition positions of different spatial levels in the image can be determined.

[0042] In this invention, the calculation of the cumulative distribution function provides a direct basis for determining the stratification boundary based on a preset threshold. Because... The longitudinal response distribution is mapped to a monotonically increasing curve between 0 and 1, with multiple preset stratification thresholds (such as...). =0.3 and =0.7) can directly correspond to a specific percentile on the curve, thereby adaptively determining the precise boundary position of different spatial level regions in the longitudinal direction of the image. This method based on cumulative distribution threshold does not rely on a fixed ratio or absolute coordinates, but dynamically determines the boundary position according to the structural distribution of each image itself, thus adapting to the differences in perspective relationships in different scenes (such as ramps, changes in camera pitch angle, etc.).

[0043] In step S5, based on multiple preset layer thresholds, the positions of multiple layer boundaries in the shallow feature map coordinate system are determined according to the cumulative distribution function, and the multiple layer boundaries are mapped to the front view image sample coordinate system. Layer boundaries are the vertical dividing points used to delineate different spatial hierarchical regions. In this invention, the number of layer boundaries determines the final number of regions. Taking the division into three spatial hierarchical regions as an example, two layer boundaries need to be determined: the first layer boundary is used to distinguish the upper region from the middle region, and the second layer boundary is used to distinguish the middle region from the lower region.

[0044] The preset multiple tiered thresholds include a first preset threshold. Second preset threshold ,and In a preferred embodiment of the present invention, The value range is from 0.2 to 0.4. The value range is from 0.6 to 0.8. =0.3、 Taking a value of 0.7 as an example, the first layer boundary corresponds to the position where the cumulative distribution value reaches 30%, and the second layer boundary corresponds to the position where the cumulative distribution value reaches 70%. The specific values ​​of the above thresholds can be adjusted according to the actual application scenario and the needs of the training task.

[0045] Vertical positions of multiple layer boundaries in the shallow feature map coordinate system and , is represented as: ; ; The multiple layer boundaries are then mapped to the front view image sample coordinate system, and represented as follows: ; ; in, Indicates the height of the front view image sample; This indicates the longitudinal position of the first layer boundary in the coordinate system of the front view image sample; This indicates the vertical position of the second layer boundary in the coordinate system of the front view image sample. Since the shallow feature map and the front view image sample maintain a spatial correspondence in the vertical direction (i.e., the y-th row in the feature map corresponds to the region in the original image that is roughly at the same vertical position), the layer boundary can be accurately restored to the original image space through the above linear mapping.

[0046] Mapped and For integers (or rounded to the nearest integer) in the range of 0 to H, the front view image samples are divided into different spatial hierarchical regions in the vertical direction. The boundary determination method described above, based on the cumulative distribution function and preset thresholds, is dynamically calculated entirely according to the spatial structure of each input image, without relying on fixed ratio divisions or predefined templates. Therefore, it can adaptively adapt to perspective changes in different driving scenarios, such as horizon shift caused by driving on slopes, changes in the proportion of near and far regions caused by changes in camera pitch angle, and changes in target distribution caused by changes in traffic density. This step can automatically adjust the position of the layer boundary to ensure the accuracy and adaptability of the region division.

[0047] In step S6, the front view image sample is divided into multiple spatial hierarchical regions along the vertical direction according to the multiple mapped hierarchical boundaries. Based on the mapped first layer boundary Second layer boundary The front view image sample is divided into an upper region along the vertical direction. Central region and lower area , is represented as: ; in, From row 0 of the front view image sample to upper area of ​​the row ; Indicates the first from the front view image sample Arriving Central region of the line ; Indicates the image sample from the front view Arriving lower area of ​​the row .

[0048] In step S7, a differential enhancement operation matching the region is applied to each of the spatial hierarchy regions to obtain an enhanced sub-image of each region; Figure 5 This is a schematic diagram illustrating data augmentation implemented using the method proposed in Embodiment 1 of the present invention; In this invention, Figure 4 and Figure 5 The black rectangular area in the middle represents a random occlusion mask M applied to the lower region R3, used to simulate close-range occlusion or local visual interference, and is not a license plate masking or image anomaly.

[0049] The upper region, i.e., the far background region R1, mainly consists of the distant background and has a low information density. Therefore, it is not advisable to use excessive enhancement in this region, otherwise it is easy to introduce meaningless perturbations. Therefore, a weak enhancement strategy is adopted for this region, including mild noise, brightness perturbation, brightness adjustment, mild contrast changes, and slight blurring.

[0050] The central region, i.e., the mid-to-long-range target region R2, typically includes small targets and mid-to-long-range targets. Therefore, the focus is on enhancing scale representation capabilities, employing scale enhancement strategies:

[0051] in:

[0052] Essentially, it allows targets in the middle layer region to appear at a larger local scale while keeping the input size constant, thereby helping the model to better learn the features of small targets.

[0053] The lower region, i.e., the near-field region R3, corresponds to near-view traffic participants and roads, and is prone to complex situations such as occlusion, shadows, and localized glare. Therefore, the focus is on improving robustness, using occlusion simulation enhancement:

[0054] Where M is a random occlusion mask.

[0055] In step S8, the enhanced sub-images of each region are stitched and fused together in the original spatial order to output the final enhanced image.

[0056] First, the enhanced central region Cut to the original size.

[0057] The enhanced upper region Cut to the middle area of ​​the original size and the enhanced lower region The pieces are vertically spliced ​​according to the original spatial order, and the splicing boundaries are smoothed.

[0058] The enhanced upper region Place at the top of the image (row 0 to row 1) (Line), cut to the middle area of ​​the original size. Placed in the center of the image (the first) Arriving at the (Line), will enhance the lower region Placed at the bottom of the image (the first) (Up to row H) The three remain unchanged in the horizontal direction and are connected end to end in the vertical direction.

[0059] Because the three regions have undergone different types of enhancement operations, pixel value discontinuities or unnatural visual transitions (such as abrupt brightness changes or texture jumps) may occur at the region boundaries. These abrupt changes may adversely affect the training of the visual perception model. Therefore, this invention performs a smoothing process on the region boundaries after stitching. This smoothing process involves using linear interpolation or a Gaussian weighted average to seamlessly blend pixel values ​​within the neighborhood pixel width of the stitching boundary.

[0060] The smoothing process includes: at the first layer boundary Second layer boundary Within the width of the neighboring pixels, the pixel values ​​are transitionally blended using linear interpolation or Gaussian weighted averaging.

[0061] Through the above-described stitching, fusion, and smoothing processes, the final enhanced image retains the global spatial structure of the original image while applying differentiated enhancements to different spatial levels, avoiding pixel value abrupt changes and visual discontinuities at region boundaries. This final enhanced image can be directly input into visual perception models such as object detection networks or semantic segmentation networks for training. Due to the adaptive hierarchical and differentiated enhancement processing of the training data, the model's learning ability for targets of different scales, scenes at different distances, and under different occlusion conditions is effectively improved, thereby enhancing the perception performance and generalization ability of autonomous driving visual perception systems in complex environments.

[0062] The present invention proposes an adaptive spatial layered image enhancement method for autonomous driving in Embodiment 1. This method dynamically generates enhancement region boundaries by analyzing the spatial structure of the image and adopts differentiated enhancement strategies for different regions, thereby improving the model's learning ability for targets of different scales and complex scenes.

[0063] The adaptive spatial layering image enhancement method for autonomous driving proposed in Embodiment 1 of this invention can be integrated into the training process of an autonomous driving visual perception model. During the training phase, an adaptive layering algorithm is first executed on the input front-view image samples, followed by corresponding enhancement operations on different regions. The enhanced images are then input into an object detection network or a semantic segmentation network for training.

[0064] Example 2 Based on the adaptive spatial layering image enhancement method for autonomous driving proposed in Embodiment 1 of this invention, Embodiment 2 of this invention also proposes an adaptive spatial layering image enhancement system for autonomous driving. Figure 6 This is a schematic diagram of an adaptive spatial layered image enhancement system for autonomous driving proposed in Embodiment 2 of the present invention; the system includes: The feature extraction module is used to acquire front view image samples of the target vehicle in an autonomous driving scenario; and to extract features from the front view image samples using a shallow feature extraction network to obtain a shallow feature map; the shallow feature map is a feature map that retains the spatial position structure and is output by the network layer near the input end of the shallow feature extraction network. The aggregation calculation module is used to perform spatial vertical aggregation processing on the shallow feature map to obtain a normalized vertical response vector that reflects the structural response intensity at different height positions of the image, and to calculate the cumulative distribution function based on the normalized vertical response vector. The boundary generation module is used to determine the positions of multiple layer boundaries in the shallow feature map coordinate system based on multiple preset layer thresholds and the cumulative distribution function, and to map the multiple layer boundaries to the front view image sample coordinate system. The fusion output module is used to divide the front view image sample into multiple spatial hierarchical regions along the vertical direction according to the multiple mapped hierarchical boundaries; apply a differential enhancement operation matching the region to each spatial hierarchical region to obtain the enhanced sub-image of each region; and stitch and fuse the enhanced sub-images of each region in the original spatial order to output the final enhanced image.

[0065] The description of the relevant parts of the adaptive spatial layered image enhancement system for autonomous driving provided in Embodiment 2 of this application can be found in the detailed description of the corresponding parts of the adaptive spatial layered image enhancement method for autonomous driving provided in Embodiment 1 of this application, and will not be repeated here.

[0066] The adaptive spatial layered image enhancement system for autonomous driving provided in Embodiment 2 of this application modularizes the process of the method disclosed in Embodiment 1 of this invention, and can achieve the same technical effect as the adaptive spatial layered image enhancement method for autonomous driving disclosed in Embodiment 1 of this invention.

[0067] Example 3 The present invention also proposes a device, Figure 7 This is a schematic diagram of an adaptive spatial layering image enhancement device for autonomous driving proposed in Embodiment 3 of the present invention; At the hardware level, the electronic device 700 includes a processor 710, and optionally, an internal bus 720, a network interface 730, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or it may also include non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations. The processor 710, network interface 730, and memory can be interconnected via an internal bus 720. This internal bus 720 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be categorized as an address bus, data bus, control bus, etc. For ease of illustration, only a single bidirectional arrow is used in this diagram, but this does not imply that there is only one bus or one type of bus. The memory is used to store programs. Specifically, the program can include program code, which includes computer operation instructions. The memory can include main memory 740 and non-volatile memory 750, and provides instructions and data to the processor 710. The processor 710 reads the corresponding computer program from the non-volatile memory 750 into the main memory 740 and then runs it, which is an adaptive spatial layering image enhancement device for autonomous driving. The processor 710 executes the program stored in the memory and specifically performs the following: In step S1, a front view image sample of the target vehicle in the autonomous driving scenario is obtained; In step S2, a shallow feature extraction network is used to extract features from the front view image sample to obtain a shallow feature map; the shallow feature map is the feature map output by the network layer near the input end of the shallow feature extraction network, which retains the spatial position structure. In step S3, the shallow feature map is subjected to spatial vertical aggregation processing to obtain a normalized vertical response vector that reflects the structural response intensity at different height positions of the image. In step S4, the cumulative distribution function is calculated based on the normalized longitudinal response vector; In step S5, based on multiple preset layer thresholds, the positions of multiple layer boundaries in the shallow feature map coordinate system are determined according to the cumulative distribution function, and the multiple layer boundaries are mapped to the front view image sample coordinate system. In step S6, the front view image sample is divided into multiple spatial hierarchical regions along the vertical direction according to the multiple mapped hierarchical boundaries. In step S7, a differential enhancement operation matching the region is applied to each of the spatial hierarchy regions to obtain an enhanced sub-image of each region; In step S8, the enhanced sub-images of each region are stitched and fused together in the original spatial order to output the final enhanced image.

[0068] Figure 1 It can be applied to processor 710, or implemented by processor 710. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the processor or by instructions in the form of software. The processor mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0069] The description of the relevant parts of the adaptive spatial layering image enhancement device for autonomous driving provided in Embodiment 3 of this application can be found in the detailed description of the corresponding parts of the adaptive spatial layering image enhancement method for autonomous driving provided in Embodiment 1 of this application, and will not be repeated here.

[0070] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that the elements inherent in a process, method, article, or apparatus that includes a list of elements are included. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Additionally, portions of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0071] While specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art can make other modifications or variations based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. An adaptive spatial layered image enhancement method for autonomous driving, characterized in that, Includes the following steps: Obtain a front view image sample of the target vehicle in an autonomous driving scenario; use a shallow feature extraction network to extract features from the front view image sample to obtain a shallow feature map; the shallow feature map is a feature map that retains the spatial position structure and is output by the network layer near the input end in the shallow feature extraction network. The shallow feature map is subjected to spatial vertical aggregation to obtain a normalized vertical response vector that reflects the structural response intensity at different height positions of the image, and the cumulative distribution function is calculated based on the normalized vertical response vector. The process of performing spatial vertical aggregation on the shallow feature map to obtain a normalized vertical response vector reflecting the structural response intensity at different height positions in the image includes: Let the shallow feature map be represented as: ;in, Indicates the number of channels; Indicates altitude; Indicates width; Spatial vertical aggregation processing is as follows: for the shallow feature map In the channel dimension and width dimension Pooling is performed on the top layer to obtain the initial longitudinal response vector: ， ； For the initial longitudinal response vector The normalized longitudinal response vector is obtained by performing normalization: ， ； in, This represents the normalized longitudinal response vector; Based on multiple preset layer thresholds, the positions of multiple layer boundaries in the shallow feature map coordinate system are determined according to the cumulative distribution function, and the multiple layer boundaries are mapped to the front view image sample coordinate system; specifically: The preset multiple stratification thresholds include a first preset threshold. Second preset threshold ,and ; The vertical position of the multiple layer boundaries in the shallow feature map coordinate system and , is represented as: ； ； This indicates the sequence from the top row 1 to the top row 2 of the feature map. The cumulative response value of the row; The multiple layer boundaries are then mapped to the front view image sample coordinate system, and represented as follows: ； ； in, Indicates the height of the front view image sample; This indicates the longitudinal position of the first layer boundary in the coordinate system of the front view image sample; This indicates the longitudinal position of the second layer boundary in the coordinate system of the front view image sample; Based on the mapped multi-layer boundaries, the front view image sample is divided into multiple spatial layer regions along the vertical direction; a differential enhancement operation matching the region is applied to each spatial layer region to obtain the enhanced sub-image of each region; the enhanced sub-images of each region are stitched and fused in the original spatial order to output the final enhanced image. Apply differentiated enhancement operations matching the spatial hierarchy region to each of the aforementioned regions, specifically as follows: The enhanced upper region is obtained by applying a weak enhancement strategy to the upper region. The weak enhancement strategy includes: mild noise, brightness perturbation, brightness adjustment, mild contrast change, and slight blurring; The enhanced central region is obtained by applying a scale transformation enhancement operation to the central region. The scaling enhancement operation involves locally magnifying the central region based on its center and then cropping it back to the original size of the central region. The magnification factor is [missing value]. ,and ; An occlusion simulation enhancement operation is applied to the lower region, wherein the occlusion simulation enhancement operation is to overlay a random binary occlusion mask on the lower region. Generate the enhanced lower region : ； in, Represents a random binary occlusion mask; Indicates the lower region.

2. The method according to claim 1, characterized in that, The shallow feature extraction network is: the first two convolutional stages of ResNet, the shallow backbone stage of CSPDarkNet, or the first stage of Swin Transformer.

3. The method according to claim 1, characterized in that, The cumulative distribution function is calculated based on the normalized longitudinal response vector, specifically as follows: ， ； in, This indicates the sequence from the top row 1 to the top row 2 of the feature map. The cumulative response value of the row.

4. The method according to claim 1, characterized in that, Based on the mapped hierarchical boundaries, the front view image sample is divided into multiple spatial hierarchical regions along the vertical direction; specifically: Based on the mapped first layer boundary Second layer boundary The front view image sample is divided into an upper region along the vertical direction. Central region and lower area , is represented as: ； in, Indicates the range from row 0 of the front view image sample to... upper area of ​​the row ; Indicates the first from the front view image sample Arriving Central region of the line ; Indicates the image sample from the front view Arriving lower area of ​​the row .

5. The method according to claim 1, characterized in that, The enhanced sub-images of each region are stitched and fused together in their original spatial order to output the final enhanced image. Specifically, the enhanced upper region is then processed. Enhanced central region and the enhanced lower region The pieces are vertically spliced ​​according to the original spatial order, and the splicing boundaries are smoothed.

6. The method according to claim 5, characterized in that, The smoothing process includes: at the first layer boundary Second layer boundary Within the width of the neighboring pixels, the pixel values ​​are transitionally blended using linear interpolation or Gaussian weighted averaging.

7. An adaptive spatial layered image enhancement system for autonomous driving, used to execute the adaptive spatial layered image enhancement method for autonomous driving as described in any one of claims 1 to 6, characterized in that, include: The feature extraction module is used to acquire front view image samples of the target vehicle in an autonomous driving scenario; and to extract features from the front view image samples using a shallow feature extraction network to obtain a shallow feature map; the shallow feature map is a feature map that retains the spatial position structure and is output by the network layer near the input end of the shallow feature extraction network. The aggregation calculation module is used to perform spatial vertical aggregation processing on the shallow feature map to obtain a normalized vertical response vector that reflects the structural response intensity at different height positions of the image, and to calculate the cumulative distribution function based on the normalized vertical response vector. The boundary generation module is used to determine the positions of multiple layer boundaries in the shallow feature map coordinate system based on multiple preset layer thresholds and the cumulative distribution function, and to map the multiple layer boundaries to the front view image sample coordinate system. The fusion output module is used to divide the front view image sample into multiple spatial hierarchical regions along the vertical direction according to the multiple mapped hierarchical boundaries; apply a differential enhancement operation matching the region to each spatial hierarchical region to obtain the enhanced sub-image of each region; and stitch and fuse the enhanced sub-images of each region in the original spatial order to output the final enhanced image.

Images