Image enhancement processing method and device, electronic rearview mirror system and storage medium

Abstract

The invention relates to the field of image processing, and provides an image enhancement processing method and device, an electronic rearview mirror system and a storage medium. The method comprises the following steps: acquiring an original image of a vehicle in a current driving scene, and segmenting the original image into at least two sub-image blocks; determining at least one simulation exposure degree based on the structural feature information of each sub-image block; and processing the original image by adopting a first image enhancement processing strategy or a second image enhancement processing strategy based on the scene type corresponding to the current driving scene to obtain a final enhanced image. Compared with the prior art, the image enhancement effect is good, the calculation efficiency and the robustness of the algorithm can be considered, the image can be processed in real time, and the stable image enhancement effect can be kept in different scenes.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image enhancement processing method, device, electronic rearview mirror system and storage medium. Background Art

[0002] Compared with traditional rearview mirrors, the electronic rearview mirror system can display the collected images of the vehicle's rear environment on the display screen in the cab in real time, providing the driver with a more comprehensive and clear field of view, which is conducive to improving driving safety.

[0003] In order to improve the quality of the collected images, the collected images are usually enhanced. Some image enhancement processing algorithms used in related technologies basically adopt a unified image enhancement strategy to enhance the collected images, but do not differentiate the images collected in different scenes, resulting in poor image enhancement effects. For example, in a night vision scene without light, the brightness of the image is not improved enough, resulting in serious loss of details; and in a night vision scene with light, the bright areas of the image are often over-enhanced, thereby affecting the naturalness and color saturation of the image. In addition, these image enhancement processing algorithms are highly complex and have poor robustness, making it difficult to achieve real-time image processing and maintain stable image enhancement effects in different scenes (for example, night vision scenes with light and night vision scenes without light). Summary of the Invention

[0004] In view of this, the embodiments of the present application provide an image enhancement processing method, device, electronic rearview mirror system and storage medium to solve the problem that some image enhancement algorithms applied in related technologies have poor image enhancement effects, high algorithm complexity and poor robustness, making it difficult to achieve real-time image processing and maintain stable image enhancement effects in different scenarios.

[0005] A first aspect of the embodiments of the present application provides an image enhancement processing method, comprising:

[0006] Obtaining an original image of the vehicle in the current driving scene, and dividing the original image into at least two sub-image blocks;

[0007] Determine the scene type corresponding to the current driving scene;

[0008] determining at least one simulated exposure level based on structural feature information of each sub-image block;

[0009] For each simulated exposure level, based on the scene type, a gamma correction value corresponding to each sub-image block is determined, and gamma correction processing is performed on the corresponding sub-image block based on the gamma correction value to obtain a simulated exposure sequence, where different gamma correction values correspond to different scene types;

[0010] The simulated exposure sequences corresponding to each simulated exposure level are weightedly fused to obtain the final enhanced image.

[0011] A second aspect of the embodiments of the present application provides an image enhancement processing device, comprising:

[0012] an acquisition module configured to acquire an original image of the vehicle in a current driving scene and segment the original image into at least two sub-image blocks;

[0013] A first determining module is configured to determine a scene type corresponding to the current driving scene;

[0014] a second determining module configured to determine at least one simulated exposure level based on structural feature information of each sub-image block;

[0015] a third determining module configured to determine, for each simulated exposure level, a gamma correction value corresponding to each sub-image block based on the scene type, and perform gamma correction processing on the corresponding sub-image block based on the gamma correction value to obtain a simulated exposure sequence, wherein different gamma correction values correspond to different scene types;

[0016] The fusion module is configured to perform weighted fusion on the simulated exposure sequences corresponding to the various simulated exposure levels to obtain a final enhanced image.

[0017] According to a third aspect of the present application, an electronic rearview mirror system is provided, comprising:

[0018] An image acquisition device, and an image processing device and an image display device connected to the image acquisition device; the image processing device includes the image enhancement processing device of the second aspect;

[0019] an image acquisition device configured to acquire an original image of the vehicle in a current driving scene and transmit the image to the image processing device;

[0020] an image processing device configured to process the original image to obtain a final enhanced image, and output the final enhanced image to the image display device;

[0021] The image display device is configured to display the final enhanced image.

[0022] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above method are implemented.

[0023] Compared with the prior art, the embodiments of the present application have at least the following beneficial effects: by first determining the scene type corresponding to the current driving scenario and then performing image enhancement processing on the collected original image using an image enhancement processing strategy corresponding to that scene type, the clarity, contrast, natural color, and saturation of the images collected in different scenarios are significantly improved, while retaining more image detail information. At the same time, this method also takes into account computational efficiency and performance, ensuring real-time image processing capabilities while the vehicle is in motion, enhancing the robustness of the algorithm, and maintaining stable image enhancement effects in different scenarios. This significantly improves the usability of the electronic rearview mirror system at night or in low-light conditions, thereby providing the driver with clearer and more reliable visual information. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0025] Figure 1 This is a technical diagram of an image enhancement processing method provided in an embodiment of the present application;

[0026] Figure 2 This is a flow chart of an image enhancement processing method provided in an embodiment of the present application;

[0027] Figure 3 This is a schematic diagram of a process for performing image enhancement processing on an original image according to an embodiment of the present application;

[0028] Figure 4 This is a flowchart of an image enhancement processing method provided in an application example of this application;

[0029] Figure 5 Schematic diagram of the structure of an image enhancement processing device provided in an embodiment of the present application;

[0030] Figure 6 Schematic diagram of the structure of an electronic rearview mirror system provided in an embodiment of the present application;

[0031] Figure 7 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0033] An image enhancement processing method, device, and electronic rearview mirror system according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0034] With the rapid development of automotive electronic technology, electronic rearview mirror systems have gradually become standard configurations in modern vehicles due to their advantages of providing a wider field of view and reducing blind spots.

[0035] The electronic rearview mirror system mainly includes an image acquisition device, an image processing device and an image display device. Among them, the image acquisition device can be a camera mounted on the vehicle (specifically, it can be a high-definition camera). The camera is mainly used to capture real-time images around the vehicle. The image processing device may include an image sensor, an imaging processor and a controller. The image sensor is mainly used to convert the light signal captured by the camera into an electrical signal, and further process it into a digital signal for subsequent processing. The imaging processor is mainly used to digitally process the signal obtained from the image sensor, including image enhancement, denoising, scaling and other operations to obtain high-quality images. The controller is mainly used for the operation and control of the entire electronic rearview mirror system, including signal processing, display driver, user interface, etc. The image display device can be a display screen installed in the cockpit (such as a central display screen, etc.). The display screen is mainly used to display the processed image to the driver in real time.

[0036] When driving at night or in low-light conditions like tunnels, the quality of images captured by the electronic rearview mirror system's camera is often poor, with issues such as insufficient brightness, poor contrast, and blurred details. However, some image enhancement algorithms used in related technologies have poor image enhancement effects, are complex, and lack robustness, making it difficult to achieve real-time image processing and maintain stable image enhancement results in different scenarios.

[0037] In view of this, an embodiment of the present application provides an image enhancement processing method. By performing differentiated processing on the original images collected in different driving scenarios (such as night vision scenarios with and without lights), the method significantly improves the clarity, contrast, natural color, and saturation of the images collected in different driving scenarios, and retains more image detail information. At the same time, the method also takes into account computational efficiency and performance, ensuring real-time image processing capabilities while the vehicle is in motion, enhancing the robustness of the algorithm, and being able to maintain a stable image enhancement effect in different scenarios. This significantly improves the usability of the electronic rearview mirror system at night or in low-light conditions, thereby providing the driver with clearer and more reliable visual information.

[0038] Figure 1 This is a schematic diagram of a technical route for an image enhancement processing method provided in an embodiment of the present application. This technical route mainly includes:

[0039] Step S101: input an original image of the vehicle in the current driving scene, wherein the original image can be acquired by an image acquisition device (such as a vehicle-mounted camera, etc.) of the vehicle's electronic rearview mirror system and transmitted to an image processing device.

[0040] Step S102: Determine the scene type corresponding to the current driving scene.

[0041] Step S103: If the scene type of the current driving scene is the first night vision scene (such as a night vision scene without lights), the first image enhancement processing strategy is used to process the original image to obtain a final enhanced image, and then jump to step S105.

[0042] Step S104: If the scene type of the current driving scene is the second night vision scene (such as a night vision scene with lights), the second image enhancement processing strategy is used to process the original image to obtain a final enhanced image, and then jump to step S105.

[0043] Step S105 : outputting the final enhanced image to the image display device of the electronic rearview mirror system.

[0044] Figure 2 This is a flow chart of an image enhancement processing method provided in an embodiment of the present application. Figure 2 The image enhancement processing method can be performed by an image processing device of an electronic rearview mirror system of a vehicle. Figure 2 As shown, the image enhancement processing method includes:

[0045] Step S201 : obtaining an original image of the vehicle in the current driving scene, and dividing the original image into at least two sub-image blocks.

[0046] The vehicles in the embodiments of the present application may be new energy vehicles. New energy vehicles refer to vehicles that use new energy sources (non-traditional oil and diesel energy) and have advanced technology. These vehicles use new power systems that can effectively reduce vehicle emissions, reduce environmental impact, and improve energy efficiency. The new energy vehicles in the embodiments of the present application include but are not limited to the following types of vehicles: electric vehicles (EVs), pure electric vehicles (BVs), fuel cell electric vehicles (FCVs), plug-in hybrid electric vehicles (PHVs), and hybrid electric vehicles (HVs).

[0047] As an example, the image acquisition device of the electronic rearview mirror system (for example, a rearview camera mounted on the vehicle) can be used to capture the rear environment image of the vehicle in the current driving scenario (i.e., the original image), and the captured original image can be transmitted to the image processing device of the electronic rearview mirror system for image enhancement processing.

[0048] In some embodiments, the original image may be first subjected to relative total variation processing to extract the main image structure of the original image; then, the original image may be clustered and segmented based on the extracted main image structure to obtain at least two sub-image blocks with different structural feature information.

[0049] The main structure of an image refers to the most core and critical part of the image that can attract the observer's main attention and carry key information. It shapes the image's meaning and visual center of gravity from multiple dimensions.

[0050] Generally speaking, the texture and structure of an original image exhibit completely different characteristics. The detailed texture of an original image often exhibits large gradient variations, while the main structure of the original image often changes little. Therefore, within a window, regions with significant texture detail produce a larger windowed total variational response D(·), while image edge regions produce a larger windowed intrinsic variational response L(·).

[0051] For example, the windowed total variation response D(·) can be calculated according to the following formula (1):

[0052]

[0053] In formula (1), D(·) represents the total windowed variation response. In the original input image, with point p as the center, the total variation D in the x and y directions is calculated within its neighborhood window R(p): x (p), D y (p), has a large value in both detail texture and main structure; q represents any point in the neighborhood window R(p); Represents the partial derivative of the original image in the x direction; Represents the partial derivative of the original image in the y direction; gp，q A weight function representing spatial association.

[0054] For example, the windowed intrinsic variational response L(·) can be calculated according to the following formula (2):

[0055]

[0056] In formula (2), L(·) represents the windowed intrinsic variational response, that is, the total absolute change in the window. In the input original image, with point p as the center, the absolute change L in the x and y directions is calculated in its neighborhood window R(p): x (p), L y (p); In the main area with uniform undulation, this value is almost zero, but at the edge where the pixel difference is large, this value is large; Represents the partial derivative of the original image in the x direction; Represents the partial derivative of the original image in the y direction; g p，q A weight function representing spatial association.

[0057] The g in the above equations (1) and (2) can be calculated by the following equation (3): p，q :

[0058]

[0059] In formula (3), σ represents the standard deviation of the Gaussian distribution, which is used to determine the weight of each pixel in the filter; x p 、y p Respectively represent the horizontal and vertical coordinates of the current pixel; x q 、y q Represent the horizontal and vertical coordinates of the neighborhood pixels respectively.

[0060] In order to further enhance the contrast between texture and structure, especially for visually prominent areas, the relative total variation model can be obtained based on D(·) and L(·) in the above equations (1) and (2). The expression of its objective function is shown in the following equation (4):

[0061]

[0062] In formula (4), λ is the weight factor for controlling the penalty term; ε is a small number (approaching zero) to prevent the denominator from being zero; I str Indicates the main image structure corresponding to the input original image; I in Represents the original input image.

[0063] Among them, the objective function f tar The first item in: Ensure that the main structure of the image is similar to the original input image as a whole; the second item: It can make the image texture details smaller, that is, ensure that the features of the main structure of the image are prominent.

[0064] By adopting the above relative total variation model, the main image structure of the original image can be better extracted, and then the original image can be clustered and segmented based on the extracted main image structure to obtain m sub-image blocks, where m is an integer ≥2.

[0065] Based on the extracted main structure of the image, the original image is clustered and segmented to obtain m sub-image blocks. Specifically, pixels with similar structural features (for example, grayscale values of pixels, color values (such as component values of color spaces such as RGB and HSV), texture features (such as energy, entropy, contrast extracted by gray-level co-occurrence matrix), shape features (such as contours, geometric shapes, etc.)) are grouped into one category, thereby achieving segmentation of the original image and obtaining m sub-image blocks.

[0066] The algorithm for clustering and segmenting the original image can adopt K-Mans clustering algorithm, hierarchical clustering algorithm, DBSCAN (density-based spatial clustering application) algorithm, etc.

[0067] Step S202: Determine the scene type corresponding to the current driving scene.

[0068] In some embodiments, an image acquisition device (such as an onboard camera) can be used to capture images of the vehicle's surrounding environment while it is driving in various driving scenarios. These surrounding environment images are then used to train an initial neural network model (which can be an existing deep learning network model such as a convolutional neural network model) until a preset convergence condition is reached (such as the model accuracy reaching a preset accuracy threshold, or the number of training rounds reaching a preset number of rounds, etc.), thereby obtaining a trained final neural network model that can distinguish the scene classifications (scene types) of various driving scenarios. Subsequently, the original images captured in the current driving scenario can be input into the above-mentioned final neural network model for detection, thereby determining the scene type corresponding to the current driving scenario.

[0069] The above-mentioned scene types include, but are not limited to, unlit night vision scenes and illuminated night vision scenes. Unlit night vision scenes generally refer to scenes where nighttime visual observation or imaging is performed without artificial light supplemental lighting or other obvious light illumination. Illuminated night vision scenes generally refer to scenes where nighttime visual observation or imaging is performed with artificial light supplemental lighting or other obvious light illumination.

[0070] Step S203: determining at least one simulated exposure level based on the structural feature information of each sub-image block.

[0071] The structural feature information includes at least one of the brightness feature, contrast feature, texture feature, shape feature, etc. of the sub-image block.

[0072] Simulated exposure simulates the effects of different exposure levels by adjusting image brightness, contrast, and other parameters based on the image's pixel information. For example, increasing the simulated exposure level is equivalent to increasing the amount of light entering the camera during shooting, brightening the image overall; decreasing the simulated exposure level darkens the image.

[0073] Different simulated exposure levels affect the image enhancement effect of a sub-image block. The simulated exposure level is related to the average brightness of the sub-image block. Generally speaking, for sub-image blocks with lower average brightness, increasing the simulated exposure level will result in a more dramatic image enhancement effect; for sub-image blocks with higher average brightness, decreasing the simulated exposure level will result in a more moderate image enhancement effect.

[0074] In some embodiments, at least one simulated exposure level can be determined based on the brightness characteristics (e.g., average brightness) of each sub-image block. For example, if the input original image is clustered and segmented to obtain four sub-image blocks (in this case, m=4), namely sub-image block 1, sub-image block 2, sub-image block 3, and sub-image block 4, then at least one simulated exposure level can be determined based on the average brightness of sub-image blocks 1-4.

[0075] In order to balance the efficiency and robustness of the image enhancement algorithm, 1 to 4 simulated exposure levels can generally be set according to the average brightness response factor of each sub-image block.

[0076] In step S204, for each simulated exposure level, a gamma correction value corresponding to each sub-image block is determined based on the scene type, and gamma correction processing is performed on the corresponding sub-image block based on the gamma correction value to obtain a simulated exposure sequence, where different scene types correspond to different gamma correction values.

[0077] For different scene types, the quality of the original images collected (such as image brightness, contrast, etc.) often varies greatly. Therefore, different gamma correction values are needed to perform gamma correction on the original images collected under different scene types to ensure the image enhancement effect.

[0078] Step S205 , performing weighted fusion on the simulated exposure sequences corresponding to the simulated exposure levels to obtain a final enhanced image.

[0079] The technical solution provided by the embodiments of this application first determines the scene type corresponding to the current driving scenario and then enhances the collected original image using an image enhancement processing strategy corresponding to that scene type. This significantly improves the clarity, contrast, natural color, and saturation of images collected in different scenarios, while retaining more image detail information. At the same time, this method also balances computational efficiency and performance, ensuring real-time image processing capabilities while the vehicle is in motion, enhancing the robustness of the algorithm, and maintaining stable image enhancement effects in different scenarios. This significantly improves the usability of the electronic rearview mirror system at night or in low-light conditions, thereby providing drivers with clearer and more reliable visual information.

[0080] In some embodiments, based on the scene type, determining a gamma correction value for each sub-image block, and performing gamma correction processing on the corresponding sub-image block based on the gamma correction value to obtain a simulated exposure sequence includes:

[0081] If the scene type is the first night vision scene, then for each sub-image block, based on the structural feature information corresponding to the sub-image block, determining a first average brightness response factor corresponding to the sub-image block;

[0082] For each sub-image block, determining a first gamma correction value corresponding to the sub-image block based on a first average brightness response factor corresponding to the sub-image block;

[0083] Gamma correction processing is performed on the corresponding sub-image block based on the first gamma correction value to obtain a simulated exposure sequence.

[0084] As an example, a captured original image is fed into a pre-trained final neural network model, which outputs the scene type corresponding to the current driving scenario. If the scene type is a first night vision scene (i.e., a dark night vision scene), the following process can be used to determine the first gamma correction value for each sub-image block in the original image.

[0085] In the first night vision scene, the original image collected is usually a low-light image with problems such as insufficient brightness, low contrast or blurred details. For example, in the first night vision scene, the original image is usually with a brightness of <1cd / m 2 , images with signal-to-noise ratio <20dB, contrast <0.5, exposure time >1 / 30 second, and ISO value >800.

[0086] First, we can draw an average brightness response curve based on the image transmitted by the electronic rearview mirror system. Since the distribution trend of the average brightness response curve is similar to a Gaussian function with u = 1 and σ = 0.4, especially the dark area that deserves more attention has a higher degree of overlap, the brightness response curve can be designed as a function G(x). The expression of function G(x) is shown in the following formula (5):

[0087]

[0088] In formula (5), the distribution trend of the function G(x) is approximated by a Gaussian-like function with u = 1 and σ = 0.4; x represents the input variable, which can be the intensity of the input light signal (such as different illuminance values, in lux), the grayscale value of the image signal (integer range from 0 to 255), or different exposure settings (such as aperture value, shutter speed, etc.); u represents the center position parameter of the distribution; σ represents the standard deviation of the Gaussian distribution.

[0089] Among them, the average brightness response curve is a graph used to intuitively display the average brightness response relationship of an imaging system, display device, etc. to different input signals or conditions.

[0090] A Gaussian-like function refers to a class of functions that are similar to the Gaussian function (normal distribution function) in form, properties or functions.

[0091] Based on the above formula (5), the mathematical expression for calculating the first average brightness response factor corresponding to each sub-image block can be determined as follows: As shown in the following formula (6).

[0092]

[0093] In formula (6), represents the first average brightness response factor of the mth sub-image block under the kth simulated exposure level; k represents the kth simulated exposure level; I subk_m_i represents the brightness intensity value of the i-th pixel in the m-th sub-image block, and n represents the total number of pixels in the m-th sub-image block.

[0094] In order to balance the computational efficiency and robustness of the image enhancement algorithm, the value of k is generally 1 to 4 in dark night vision scenes.

[0095] As an example, see Figure 3 Assuming that the current driving scene is a night vision scene without lights, the original image I is collected by the camera of the vehicle's electronic rearview mirror system. in_1 , for the original image I in_1Cluster segmentation is performed to obtain 4 sub-image blocks (m=4 at this time), which are respectively recorded as sub-image block ①, sub-image block ②, sub-image block ③ and sub-image block ④. According to the average brightness of sub-image blocks ① to ④, 4 simulated exposure levels are determined, which are respectively recorded as simulated exposure levels 1, 2, 3, and 4. Among them, the exposure levels of simulated exposure levels 1, 2, 3, and 4 are different, so the number of simulated exposure levels is 4, and the values of k are 1, 2, 3, and 4.

[0096] For the simulated exposure level 1, that is, when k=1, the sub-image block ① (which can be marked as I subk_1 , where m=1), sub-image block ② (which can be marked as I subk_2 , where m = 2), sub-image block ③ (which can be marked as I subk_3 , where m=3), sub-image block ④ (which can be marked as I subk_4 , where m=4) at the simulated exposure level 1.

[0097] The following description will be made by taking the calculation of the first average brightness response factor of the sub-image block ① under the simulated exposure level 1 as an example.

[0098] Substituting the brightness intensity value of each pixel and the total number of pixels of sub-image block ① under simulated exposure level 1 into the above formula (6), the first average brightness response factor of sub-image block ① under simulated exposure level 1 can be calculated as

[0099] Similarly, the first average brightness response factor of sub-image blocks ② to ④ under the simulated exposure level 1 can be calculated by referring to the above method. and I will not go into details here.

[0100] Next, the first average brightness response factor of the sub-image blocks ① to ④ under the simulated exposure level 1 can be used to to Determine the first gamma correction value of sub-image blocks ① to ④ at simulated exposure level 1 (corresponding to the first gamma correction value of sub-image block ①), (corresponding to the first gamma correction value of sub-image block ②), (corresponding to the first gamma correction value of sub-image block ③), (corresponding to the first gamma correction value of sub-image block ④).

[0101] Similarly, for the simulated exposure level 2, that is, when k=2, the first average brightness response factor of the sub-image blocks ① to ④ under the simulated exposure level 2 can be calculated by referring to the above method: to Then, according to the first average brightness response factor of sub-image blocks ① to ④ under the simulated exposure level 2 to Determine the first gamma correction value of sub-image blocks ① to ④ at simulated exposure level 2 and For the simulated exposure level 3, that is, when k=3, the first average brightness response factor of the sub-image blocks ① to ④ under the simulated exposure level 3 can be calculated by referring to the above method: to Then, according to the first average brightness response factor of sub-image blocks ① to ④ under the simulated exposure level 3 to Determine the first gamma correction value of sub-image blocks ① to ④ under simulated exposure level 3 to For the simulated exposure level 4, that is, when k=4, the first average brightness response factor of the sub-image blocks ① to ④ under the simulated exposure level 4 can be calculated by referring to the above method: to Then, according to the first average brightness response factor of sub-image blocks ① to ④ under the simulated exposure level 4 to Determine the first gamma correction value of sub-image blocks ① to ④ under simulated exposure level 4 to I will not go into details here.

[0102] In some embodiments, determining a first gamma correction value corresponding to the sub-image block based on a first average luminance response factor corresponding to the sub-image block includes:

[0103] Determining an image enhancement weight value corresponding to the sub-image block based on the first average brightness response factor;

[0104] Based on the block pixel intensity value and the image enhancement weight value of the sub-image block, a first gamma correction value corresponding to the sub-image block is determined.

[0105] The pixel intensity value of a block refers to the intensity value (such as the brightness intensity value) of all pixels in the sub-image block.

[0106] As an example, the image enhancement weight value corresponding to each sub-block can be calculated according to the following formula (7).

[0107]

[0108] In formula (7), represents the image enhancement weight value of the mth sub-block under the kth simulated exposure level, k represents the kth simulated exposure level, represents the first average brightness response factor of the mth sub-image block at the kth simulated exposure level.

[0109] When the average brightness of a sub-image block is lower, the image enhancement weight value corresponding to that sub-image block is closer to 1, which means that a greater degree of simulated exposure is applied to darker areas, resulting in a more drastic image enhancement effect. When the average brightness of a sub-image block is higher, the image enhancement weight value corresponding to that sub-image block is closer to 0, which means that a slightly smaller degree of simulated exposure is applied to brighter areas, resulting in a more moderate image enhancement effect.

[0110] In some embodiments, determining a first gamma correction value corresponding to the sub-image block based on a block pixel intensity value and an image enhancement weight value of the sub-image block includes:

[0111] Determining a first weighted probability density function value corresponding to the sub-image block based on a block pixel intensity value, a maximum block pixel intensity value, a minimum block pixel intensity value, and an image enhancement weight value of the sub-image block;

[0112] Determining a weighted cumulative distribution function value corresponding to the sub-image block based on the first weighted probability density function value;

[0113] Based on the weighted cumulative distribution function value, a first gamma correction value corresponding to the sub-image block is determined.

[0114] The maximum pixel intensity value of the image block refers to the intensity value of the pixel with the largest intensity among all pixels of the sub-image block.

[0115] The minimum pixel intensity value of the image block refers to the intensity value of the pixel with the smallest intensity among all pixels of the sub-image block.

[0116] As an example, the first weighted probability density function value corresponding to each sub-image block can be calculated according to the following formula (8).

[0117]

[0118] In formula (8), represents the first weighted probability density function, pdf() represents the pixel probability density function, represents the pixel intensity value of the mth sub-image block at the kth simulated exposure level; represents the first weighted probability density function value of the mth sub-image block at the kth simulated exposure level; represents the pixel probability density function value of the mth sub-image block under the kth simulated exposure level; represents the minimum pixel probability density function value of the mth sub-image block under the kth simulated exposure level; represents the maximum pixel probability density function value of the mth sub-image block under the kth simulated exposure level; Represents the image enhancement weight value of the mth sub-block under the kth simulated exposure level.

[0119] The weighted cumulative distribution function value corresponding to each sub-image block is calculated according to the following formula (9).

[0120]

[0121] In formula (9), cdf w () represents the weighted cumulative distribution function, pdf w () represents the weighted probability density function, represents the pixel intensity value of the mth sub-image block at the kth simulated exposure level, represents the weighted cumulative distribution function value of the mth sub-image block under the kth simulated exposure level; represents the first weighted probability density function value of the mth sub-image block at the kth simulated exposure level; l represents the intensity value of each pixel of the input sub-image block; l max Represents the maximum pixel intensity value of the input sub-image block.

[0122] The first gamma correction value corresponding to each sub-image block is calculated according to the following formula (10).

[0123]

[0124] In formula (10), represents the first gamma correction value of the m-th sub-image block at the k-th exposure level; represents the weighted cumulative distribution function value of the mth sub-image block under the kth simulated exposure level.

[0125] In some embodiments, in a first night vision scene, local adaptive gamma correction is performed on the corresponding sub-image blocks based on the first gamma correction value to obtain an image enhancement sequence, wherein the first image enhancement sequence includes enhanced image blocks corresponding to each sub-image block; then, guided filtering denoising and sharpening are performed on the image enhancement sequence to obtain a simulated exposure sequence, wherein the simulated exposure sequence includes simulated exposure image blocks corresponding to each enhanced image block.

[0126] The enhanced image block corresponding to each sub-image block can be calculated according to the following formula (11).

[0127]

[0128] In formula (11) represents the enhanced image block corresponding to the m-th sub-image block; represents the pixel intensity value of the mth sub-image block at the kth simulated exposure level; represents the maximum pixel intensity value of the mth sub-image block under the kth simulated exposure level; represents the first gamma correction value of the mth sub-image block at the kth exposure level.

[0129] In the first night vision scene, for each simulated exposure level, the first gamma correction value corresponding to each sub-image block is calculated using the above method, and local adaptive gamma correction is performed on the corresponding sub-image block based on each first gamma correction value. This can not only improve the visual quality of the image, but also improve the efficiency of generating the simulated exposure sequence, avoid manual intervention, thereby reducing computing costs, and alleviate the amplification of noise caused by directly increasing the image brightness.

[0130] In some embodiments, weighted fusion of simulated exposure sequences corresponding to respective simulated exposure levels to obtain a final enhanced image includes:

[0131] For each simulated exposure level, determining a weight map of a simulated exposure sequence corresponding to the simulated exposure level;

[0132] Based on the weight maps corresponding to the simulated exposure sequences, each simulated exposure sequence is weightedly fused to obtain a fused enhanced image;

[0133] The fused enhanced image is upsampled to obtain the final enhanced image.

[0134] A weight map refers to a map formed by assigning a weight value to each pixel or data point in the fields of image and data processing. The weight value represents the importance or influence of the pixel or data point position.

[0135] Before fusing the simulated exposure sequences corresponding to the various simulated exposure levels, it is necessary to determine the weight maps corresponding to the various simulated exposure sequences, that is, to determine the importance or influence of each pixel or data point of each simulated exposure sequence on image feature extraction and smoothing.

[0136] In some embodiments, a weight map can be determined based on the importance or impact of pixel characteristics (e.g., grayscale value, color value, or texture features) of each simulated exposure sequence on image feature extraction and smoothing in the image fusion task. For example, the grayscale value of each pixel in the simulated exposure sequence can be normalized and used as the weight value.

[0137] In some embodiments, based on the weight maps corresponding to the respective simulated exposure sequences, weighted fusion is performed on the respective simulated exposure sequences to obtain a fused enhanced image, including:

[0138] Decomposing the weight map corresponding to each simulated exposure sequence into a Gaussian pyramid, and decomposing each simulated exposure sequence into a Laplacian pyramid, wherein the Gaussian pyramid and the Laplacian pyramid have the same number of layers;

[0139] The Gaussian pyramids corresponding to the simulated exposure sequences are used to perform weighted fusion on the Laplacian pyramids corresponding to the simulated exposure sequences to obtain a fused enhanced image.

[0140] The Gaussian pyramid is constructed by performing multiple downsampling and Gaussian filtering operations on an image. The Laplacian pyramid is constructed based on the Gaussian pyramid. Each layer of the Laplacian pyramid is the difference between the two adjacent layers of the Gaussian pyramid.

[0141] The Laplacian pyramids corresponding to each simulated exposure sequence can be weighted fused according to the following formula (12) to obtain a fused enhanced image.

[0142]

[0143] In formula (12), represents the fused enhanced image; x and y represent the x and y directions of the fused enhanced image respectively; i represents the summation index, and the value of i is 1, 2, 3, ..., k; k represents the number of simulated exposure levels; A weight map i representing the simulated exposure sequence i at the i-th simulated exposure level; I represents the Gaussian pyramid of the weight map i of the simulated exposure sequence i under the i-th simulated exposure level; expk_i (x, y) represents the simulated exposure sequence i at the i-th simulated exposure level; represents the Laplacian pyramid of the simulated exposure sequence i at the i-th simulated exposure level; S represents the number of layers of the Laplacian pyramid and the Gaussian pyramid; h and w represent the height and width of the image, respectively.

[0144] In formula (12), each layer of the Laplacian pyramid is obtained by accumulating and averaging the weights of the Gaussian pyramid of the corresponding layer. In this way, the weight information of the simulated exposure sequence at each simulated exposure level can be assigned to the detail information to obtain a fused enhanced image.

[0145] Finally, the fused enhanced image can be upsampled according to the following formula (13) to obtain the final enhanced image.

[0146]

[0147] In formula (13), I out (x, y) represents the final enhanced image; U d() represents the upsampling operator; j represents the summation index, and the value of j is 1, 2, 3, ..., S; S represents the number of layers of the Laplacian pyramid and Gaussian pyramid; represents the fused enhanced image, and d represents the dynamic range of the image.

[0148] As an example, see Figure 3 , assuming that the input original image I in_1 After clustering and gamma correction, we can obtain simulated exposure sequences 1 to 4 corresponding to simulated exposure levels 1, 2, 3, and 4. In this example, the values of k are 1, 2, 3, and 4. We can first determine the weight of simulated exposure sequence 1 corresponding to simulated exposure level 1. Figure 1 (marked as ), the weight of simulated exposure sequence 2 corresponding to simulated exposure level 2 Figure 2 (marked as ), the weight of simulated exposure sequence 3 corresponding to simulated exposure level 3 Figure 3 (marked as ), the weight of the simulated exposure sequence 4 corresponding to the simulated exposure level 4 Figure 4 (marked as ); Then, the weight Figures 1 to 4 They are decomposed into Gaussian pyramids, and the simulated exposure sequences 1 to 4 are decomposed into Laplacian pyramids, respectively. The number of layers of the decomposed Gaussian pyramid and Laplacian pyramid is the same.

[0149] The following will First, use Gaussian filter to decompose Smoothing is performed, and then downsampling is performed (usually taking one pixel every other row and column) to obtain an image with a lower resolution. Repeat the above process to obtain a Gaussian pyramid with S layers.

[0150] Similarly, the weight corresponding to the simulated exposure sequence 2 under the simulated exposure level 2 can be set as Figure 2 (marked as ) is decomposed into a Gaussian pyramid with S layers The weight corresponding to the simulated exposure sequence 3 under the simulated exposure level 3 Figure 3 (marked as ) is decomposed into a Gaussian pyramid with S layers The weight corresponding to the simulated exposure sequence 4 at the simulated exposure level 4 Figure 4 (marked as ) is decomposed into a Gaussian pyramid with S layers

[0151] The following is an example of decomposing the simulated exposure sequence 1 under the first simulated exposure level (simulated exposure level 1) into a Laplacian pyramid. Let the rth layer of the Gaussian pyramid of the simulated exposure sequence 1 be G r , the rth layer of the Laplace pyramid is L r , r takes the value of 1, 2, 3, ..., S, then L r =G r -U(G r+1 ), where U(G r+1 ) indicates the r+1 Upsampling (usually by interpolation methods such as bilinear interpolation) and Gaussian filtering are performed to make it similar in size to G r In this way, starting from the highest resolution L1 layer, each layer of the Laplacian pyramid is calculated layer by layer, thereby obtaining the simulated exposure sequence 1 (marked as I expk_1 (x,y)) corresponds to the Laplace pyramid L S [I expk_1 (x,y)].

[0152] Similarly, referring to the above method, the simulated exposure sequence 2 (marked as 1) under the second simulated exposure level (simulated exposure level 2) can be expk_2 (x,y)) is decomposed into a Laplacian pyramid L S [I expk_2 (x, y)]; simulated exposure sequence 3 (labeled as I) under the third simulated exposure level (simulated exposure level 3) expk_3 (x,y)) is decomposed into a Laplacian pyramid L S [I expk_3 (x, y)]; simulated exposure sequence 4 (labeled as I) under the fourth simulated exposure level (simulated exposure level 4) expk_4 (x,y)) is decomposed into a Laplacian pyramid L S [I expk_4 (x,y)].

[0153] Next, the simulated exposure sequences 1 to 4 are weighted fused according to the above formula (12) to obtain a fused enhanced image; then, the fused enhanced image is upsampled according to the above formula (13) to obtain a final enhanced image, which is output to the image display device of the electronic rearview mirror system.

[0154] In other embodiments, based on the scene type, a gamma correction value of each sub-image block is determined, and gamma correction processing is performed on the corresponding sub-image block based on the gamma correction value to obtain a simulated exposure sequence, including:

[0155] If the scene type is the second night vision scene, then, for each sub-image block, based on the structural feature information corresponding to the sub-image block, determining a second average brightness response factor and a third average brightness response factor corresponding to the sub-image block; wherein the sum of the second average brightness response factor and the third average brightness response factor is 1;

[0156] determining an image enhancement coefficient corresponding to the sub-image block according to the second average brightness response factor and the third average brightness response factor;

[0157] determining a second gamma correction value for the sub-image block based on a block pixel intensity value and an image enhancement coefficient of the sub-image block;

[0158] Gamma correction processing is performed on the corresponding sub-image block based on the second gamma correction value to obtain a simulated exposure sequence.

[0159] To avoid artificial selection of simulated multi-exposure images, the brightness response curve G(x) in the dark night vision scene is used here to calculate the second average brightness response factor of each sub-image block. The mathematical expression of the second average brightness response factor is shown in the following formula (14):

[0160]

[0161] In formula (14), I represents the second average brightness response factor of the mth sub-image block under the kth simulated exposure level, k represents the kth simulated exposure level, and in the night vision scene with light, the value of k is generally 1 to 2; subk_m_i represents the brightness intensity value of the i-th pixel in the m-th sub-image block, and n represents the total number of pixels in the m-th sub-image block.

[0162] The third average brightness response factor can be calculated according to the following formula (15):

[0163]

[0164] In formula (15), represents the third average brightness response factor of the mth sub-image block under the kth simulated exposure level.

[0165] The image enhancement coefficient corresponding to each sub-image block can be calculated according to the following formula (16).

[0166]

[0167] In formula (16), represents the image enhancement coefficient of the mth sub-image block under the kth simulated exposure level; MAX() represents taking the larger of the two.

[0168] The second weighted probability density function value of each sub-image block can be calculated according to the following formula (17).

[0169]

[0170] In formula (17), represents the second weighted probability density function, pdf() represents the pixel probability density function, represents the pixel intensity value of the mth sub-image block at the kth simulated exposure level, represents the second weighted probability density function value of the mth sub-image block at the kth simulated exposure level; represents the pixel probability density function value of the mth sub-image block under the kth simulated exposure level; represents the minimum pixel probability density function value of the mth sub-image block under the kth simulated exposure level; represents the maximum pixel probability density function value of the mth sub-image block under the kth simulated exposure level; represents the image enhancement coefficient of the mth sub-image block at the kth simulated exposure level.

[0171] The second gamma correction value of each sub-image block is calculated according to the following formula (18).

[0172]

[0173] In formula (18), represents the second gamma correction value of the m-th sub-image block at the k-th simulated exposure level; represents the pixel intensity value of the mth sub-image block at the kth simulated exposure level; represents the second weighted probability density function value of the mth sub-image block at the kth simulated exposure level.

[0174] When the image enhancement coefficient of the mth sub-image block at the kth simulated exposure level When the value of the second weighted probability density function of the mth sub-image block at the kth simulated exposure level is greater than The larger the value is, the lower the second gamma correction value of the mth sub-image block at the kth simulated exposure level is. The smaller the value, the more obvious the enhancement effect will be, which can significantly improve the image brightness and contrast, effectively suppress over-enhancement, and appropriately increase color saturation.

[0175] Local adaptive gamma correction processing can be performed on each sub-image block according to the following formula (19) to obtain an enhanced image block corresponding to each sub-image block.

[0176]

[0177] In formula (19), represents the enhanced image block corresponding to the m-th sub-image block; represents the intensity value of each pixel of the mth sub-image block at the kth simulated exposure level; represents the maximum pixel intensity value of the mth sub-image block under the kth simulated exposure level; represents the second gamma correction value of the mth sub-image block at the kth simulated exposure level.

[0178] It is understandable that the processing process of weighted fusion of the simulated exposure sequences corresponding to each simulated exposure level to obtain the final enhanced image can be referred to in the above-mentioned embodiment, and the processing process of weighted fusion of the simulated exposure sequences corresponding to each simulated exposure level to obtain the final enhanced image will not be repeated here.

[0179] Through the above implementation, the dark areas of the original image collected in the night vision scene with light can be generally enhanced (the enhancement is more drastic), and the bright areas can be brightness-compensated enhanced (the enhancement is more moderate) to avoid excessive enhancement, thereby improving the brightness and contrast of the original image as a whole and improving the color saturation of the image.

[0180] Figure 4 This is a flowchart of an image enhancement processing method provided in an application example of this application.

[0181] See also Figure 4 First, an original image of the vehicle in the current driving scene is collected by an image acquisition device of the electronic rearview mirror system and transmitted to an image processing device of the electronic rearview mirror system; the image processing device performs relative total variation processing on the original image to extract the image main structure of the original image; the image processing device performs clustering and segmentation on the original image based on the image main structure to obtain m sub-image blocks; the image processing device selects a first image enhancement processing strategy or a second image enhancement processing strategy according to the scene type corresponding to the current driving scene to perform local adaptive gamma correction processing on the m sub-image blocks to obtain enhanced image sequences under various simulated exposure levels (including enhanced image blocks corresponding to various sub-image blocks); the image processing device performs guided filtering denoising and sharpening processing on the enhanced image sequence, and outputs a simulated exposure sequence corresponding to the enhanced image sequence under various simulated exposure levels; weighted fusion is performed on the enhanced image sequence under various simulated exposure levels to obtain a final enhanced image, and outputs it to an image display device of the electronic rearview mirror system.

[0182] The image enhancement processing method provided in the embodiment of the present application not only significantly improves the brightness and contrast of the image, but also well preserves the detail information in the image, and the enhancement effect is satisfactory; at the same time, the method also takes into account computational efficiency and performance, ensures real-time image processing capabilities during vehicle driving, enhances the robustness of the algorithm, and can maintain a stable image enhancement effect in different scenarios, significantly improving the usability of the electronic rearview mirror system at night or in low light conditions, thereby providing the driver with clearer and more reliable visual information.

[0183] All of the above optional technical solutions can be combined in any way to form optional embodiments of the present application, and will not be described in detail here.

[0184] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0185] Figure 5 This is a schematic diagram of the structure of an image enhancement processing device provided in an embodiment of the present application. Figure 5 As shown, the image enhancement processing device 500 includes:

[0186] An acquisition module 501 is configured to acquire an original image of the vehicle in a current driving scene and divide the original image into at least two sub-image blocks;

[0187] A first determining module 502 is configured to determine a scene type corresponding to the current driving scene;

[0188] A second determining module 503 is configured to determine at least one simulated exposure level based on structural feature information of each sub-image block;

[0189] a third determining module 504 configured to determine, for each simulated exposure level, a gamma correction value corresponding to each sub-image block based on the scene type, and perform gamma correction processing on the corresponding sub-image block based on the gamma correction value to obtain a simulated exposure sequence, wherein different gamma correction values correspond to different scene types;

[0190] The fusion module 505 is configured to perform weighted fusion on the simulated exposure sequences corresponding to the various simulated exposure levels to obtain a final enhanced image.

[0191] In some embodiments, the third determining module 504 may include:

[0192] a first determining unit configured to determine, for each sub-image block, a first average brightness response factor corresponding to the sub-image block based on structural feature information corresponding to the sub-image block if the scene type is a first night vision scene;

[0193] a second determining unit configured to determine, for each sub-image block, a first gamma correction value corresponding to the sub-image block based on a first average brightness response factor corresponding to the sub-image block;

[0194] The first correction unit is configured to perform gamma correction processing on the corresponding sub-image block based on the first gamma correction value to obtain a simulated exposure sequence.

[0195] In some embodiments, the second determining unit includes:

[0196] A first determining component is configured to determine an image enhancement weight value corresponding to the sub-image block based on the first average brightness response factor;

[0197] The second determining component is configured to determine a first gamma correction value corresponding to the sub-image block based on the block pixel intensity value and the image enhancement weight value of the sub-image block.

[0198] In some embodiments, the second determining component includes:

[0199] a first determining device configured to determine a first weighted probability density function value corresponding to the sub-image block based on a block pixel intensity value, a maximum block pixel intensity value, a minimum block pixel intensity value, and an image enhancement weight value of the sub-image block;

[0200] a second determining device configured to determine a weighted cumulative distribution function value corresponding to the sub-image block based on the first weighted probability density function value;

[0201] The third determining device is configured to determine a first gamma correction value corresponding to the sub-image block based on the weighted cumulative distribution function value.

[0202] In some embodiments, the fusion module 505 includes:

[0203] a weight determination unit configured to determine, for each simulated exposure level, a weight map of a simulated exposure sequence corresponding to the simulated exposure level;

[0204] a fusion unit configured to perform weighted fusion on each simulated exposure sequence based on a weight map corresponding to each simulated exposure sequence to obtain a fused enhanced image;

[0205] The upsampling unit is configured to upsample the fused enhanced image to obtain a final enhanced image.

[0206] In some embodiments, the above-mentioned fusion unit may be specifically configured as follows:

[0207] Decomposing the weight map corresponding to each simulated exposure sequence into a Gaussian pyramid, and decomposing each simulated exposure sequence into a Laplacian pyramid, wherein the Gaussian pyramid and the Laplacian pyramid have the same number of layers;

[0208] The Gaussian pyramids corresponding to the simulated exposure sequences are used to perform weighted fusion on the Laplacian pyramids corresponding to the simulated exposure sequences to obtain a fused enhanced image.

[0209] In some other embodiments, the third determining module 504 may further include:

[0210] a third determining unit configured to, if the scene type is the second night vision scene, determine, for each sub-image block, based on structural feature information corresponding to the sub-image block, a second average brightness response factor and a third average brightness response factor corresponding to the sub-image block; wherein the sum of the second average brightness response factor and the third average brightness response factor is 1;

[0211] a fourth determining unit configured to determine an image enhancement coefficient corresponding to the sub-image block according to the second average brightness response factor and the third average brightness response factor;

[0212] a fifth determining unit configured to determine a second gamma correction value of the sub-image block based on the block pixel intensity value and the image enhancement coefficient of the sub-image block;

[0213] The second correction unit is configured to perform gamma correction processing on the corresponding sub-image block based on the second gamma correction value to obtain a simulated exposure sequence.

[0214] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0215] Figure 6 This is a schematic diagram of the structure of an electronic rearview mirror system provided by an embodiment of the present application. Figure 6 The electronic rearview mirror system 600 of the embodiment of the present application includes:

[0216] An image acquisition device 601, an image processing device 602 and an image display device 603 connected to the image acquisition device 601; the image processing device 602 includes: Figure 5 The image enhancement processing device 500 shown;

[0217] The image acquisition device 601 is configured to acquire the original image of the vehicle in the current driving scene and transmit it to the image processing device 602;

[0218] The image processing device 602 is configured to process the original image to obtain a final enhanced image, and output the final enhanced image to the image display device 603;

[0219] The image display device 603 is configured to display the final enhanced image.

[0220] Figure 7 Schematic diagram of an electronic device 700 provided in an embodiment of the present application. Figure 7 As shown, the electronic device 700 of this embodiment includes: a processor 701, a memory 702, and a computer program 703 stored in the memory 702 and executable by the processor 701. When the processor 701 executes the computer program 703, the steps of the above-described method embodiments are implemented. Alternatively, when the processor 701 executes the computer program 703, the functions of the modules / units in the above-described device embodiments are implemented.

[0221] The electronic device 700 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 700 may include but is not limited to a processor 701 and a memory 702. Those skilled in the art will appreciate that Figure 7 The electronic device 700 is merely an example and does not limit the electronic device 700 . The electronic device 700 may include more or fewer components than shown in the figure, or different components.

[0222] The processor 701 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0223] The memory 702 may be an internal storage unit of the electronic device 700, such as a hard disk or memory of the electronic device 700. The memory 702 may also be an external storage device of the electronic device 700, such as a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, etc. equipped on the electronic device 700. The memory 702 may also include both an internal storage unit of the electronic device 700 and an external storage device. The memory 702 is used to store computer programs and other programs and data required by the electronic device.

[0224] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0225] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0226] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. An image enhancement processing method, characterized in that: include: Obtaining an original image of the vehicle in a current driving scene, and dividing the original image into at least two sub-image blocks; Determining a scene type corresponding to the current driving scene; determining at least one simulated exposure level based on structural feature information of each of the sub-image blocks; For each of the simulated exposure levels, determining a gamma correction value corresponding to each of the sub-image blocks based on the scene type, and performing gamma correction processing on the corresponding sub-image blocks based on the gamma correction value to obtain a simulated exposure sequence, wherein different gamma correction values correspond to different scene types; The simulated exposure sequences corresponding to the simulated exposure levels are weightedly fused to obtain a final enhanced image.

2. The method according to claim 1, characterized in that Determining a gamma correction value of each of the sub-image blocks based on the scene type, and performing gamma correction processing on the corresponding sub-image blocks based on the gamma correction value to obtain a simulated exposure sequence, including: If the scene type is a first night vision scene, then determining, for each of the sub-image blocks, a first average brightness response factor corresponding to the sub-image block based on structural feature information corresponding to the sub-image block; For each of the sub-image blocks, determining a first gamma correction value corresponding to the sub-image block based on a first average brightness response factor corresponding to the sub-image block; Gamma correction processing is performed on the corresponding sub-image block based on the first gamma correction value to obtain a simulated exposure sequence.

3. The method according to claim 2, characterized in that Determining a first gamma correction value corresponding to the sub-image block based on a first average brightness response factor corresponding to the sub-image block includes: determining an image enhancement weight value corresponding to the sub-image block based on the first average brightness response factor; Based on the block pixel intensity value of the sub-image block and the image enhancement weight value, a first gamma correction value corresponding to the sub-image block is determined.

4. The method according to claim 3, characterized in that Determining a first gamma correction value corresponding to the sub-image block based on the block pixel intensity value of the sub-image block and the image enhancement weight value includes: Determining a first weighted probability density function value corresponding to the sub-image block based on a block pixel intensity value, a maximum block pixel intensity value, a minimum block pixel intensity value, and the image enhancement weight value of the sub-image block; Determining a weighted cumulative distribution function value corresponding to the sub-image block based on the first weighted probability density function value; Based on the weighted cumulative distribution function value, a first gamma correction value corresponding to the sub-image block is determined.

5. The method according to claim 2, characterized in that Performing weighted fusion on the simulated exposure sequences corresponding to the simulated exposure levels to obtain a final enhanced image, including: For each of the simulated exposure levels, determining a weight map of a simulated exposure sequence corresponding to the simulated exposure level; Based on the weight maps corresponding to the respective simulated exposure sequences, performing weighted fusion on the respective simulated exposure sequences to obtain a fused enhanced image; The fused enhanced image is up-sampled to obtain a final enhanced image.

6. The method according to claim 5, characterized in that Based on the weight maps corresponding to the respective simulated exposure sequences, weighted fusion is performed on the respective simulated exposure sequences to obtain a fused enhanced image, including: Decomposing the weight map corresponding to each of the simulated exposure sequences into a Gaussian pyramid, and decomposing each of the simulated exposure sequences into a Laplacian pyramid, wherein the Gaussian pyramid and the Laplacian pyramid have the same number of layers; The Gaussian pyramids corresponding to the respective simulated exposure sequences are used to perform weighted fusion on the Laplacian pyramids corresponding to the respective simulated exposure sequences to obtain a fused enhanced image.

7. The method according to claim 1, characterized in that Determining a gamma correction value of each of the sub-image blocks based on the scene type, and performing gamma correction processing on the corresponding sub-image blocks based on the gamma correction value to obtain a simulated exposure sequence, including: If the scene type is the second night vision scene, then, for each of the sub-image blocks, determining, based on the structural feature information corresponding to the sub-image block, a second average brightness response factor and a third average brightness response factor corresponding to the sub-image block; wherein the sum of the second average brightness response factor and the third average brightness response factor is 1; determining an image enhancement coefficient corresponding to the sub-image block according to the second average brightness response factor and the third average brightness response factor; determining a second gamma correction value for the sub-image block based on a block pixel intensity value of the sub-image block and the image enhancement coefficient; Gamma correction processing is performed on the corresponding sub-image block based on the second gamma correction value to obtain a simulated exposure sequence.

8. An image enhancement processing device, characterized in that: include: an acquisition module configured to acquire an original image of the vehicle in a current driving scene and segment the original image into at least two sub-image blocks; A first determining module is configured to determine a scene type corresponding to the current driving scene; a second determining module configured to determine at least one simulated exposure level based on structural feature information of each of the sub-image blocks; a third determining module configured to determine, for each of the simulated exposure levels, a gamma correction value corresponding to each of the sub-image blocks based on the scene type, and perform gamma correction processing on the corresponding sub-image blocks based on the gamma correction value to obtain a simulated exposure sequence, wherein different gamma correction values correspond to different scene types; The fusion module is configured to perform weighted fusion on the simulated exposure sequences corresponding to the simulated exposure levels to obtain a final enhanced image.

9. An electronic rearview mirror system, characterized in that: include: An image acquisition device, and an image processing device and an image display device connected to the image acquisition device; the image processing device includes the image enhancement processing device according to claim 8; The image acquisition device is configured to acquire an original image of the vehicle in the current driving scene and transmit the original image to the image processing device; The image processing device is configured to process the original image to obtain a final enhanced image, and output the final enhanced image to the image display device; The image display device is configured to display the final enhanced image.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images