Image quality optimization system, method and apparatus

Abstract

The application discloses an image quality optimization system, method and device, the system comprises an image sensor, an image signal processor, a neural network processor, a controller and a display processing unit; the image sensor is used for transmitting a first original image to the image signal processor, and transmitting a second original image to the neural network processor; the image signal processor is used for determining the exposure of the first original image; the neural network processor is used for determining the first fog concentration according to the second original image; the controller is used for determining the target defogging intensity according to the exposure and the first fog concentration; the image signal processor is further used for performing defogging and noise reduction processing on the first original image according to the target defogging intensity, obtaining a target image, and transmitting the target image to the display processing unit, so that the display processing unit performs display enhancement processing on the target image. The application can optimize system resource consumption, guarantee real-time output of vehicle-mounted images, and realize accurate defogging.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image quality optimization system, method and apparatus. Background Technology

[0002] During vehicle operation, fog concentration varies and ambient light and color temperature differ significantly, which can easily cause image blurring and reduced visibility. It is necessary to adapt to different fog conditions to ensure image quality. At the same time, the on-board vision chip has strict limitations on data bandwidth and processing latency, and a balance must be struck between imaging effect and bandwidth and latency performance.

[0003] However, traditional defogging solutions cannot adaptively adjust the defogging intensity, requiring the defogging function to be forcibly activated. In fog-free or lightly foggy scenes, this leads to an overall darkening of the image and loss of image details, affecting normal imaging results. Alternatively, deep learning defogging solutions using neural network processors can adapt to different fog concentrations, but this approach requires multiple memory read / write operations and independent inference processing, significantly consuming system bandwidth and increasing data processing latency. This makes it difficult to meet the low-bandwidth, low-latency, and low-cost requirements of automotive electronic rearview mirror systems. Summary of the Invention

[0004] This application provides an image quality optimization system, method, and apparatus to optimize system resource consumption, ensure real-time output of vehicle images, and achieve accurate dehazing.

[0005] In a first aspect, embodiments of this application provide an image quality optimization system, including an image sensor, an image signal processor, a neural network processor, a controller, and a display processing unit; The image sensor is used to transmit a first raw image acquired to the image signal processor via a first communication method, and to perform downsampling processing on the first raw image to obtain a second raw image, and to transmit the second raw image to the neural network processor via a second communication method, wherein the real-time performance of the first communication method is higher than that of the second communication method; The image signal processor is used to determine the exposure of the first original image and transmit the exposure to the controller through the first communication method; The neural network processor is configured to determine a first fog concentration based on the second original image, and transmit the first fog concentration to the controller via the first communication method; The controller is configured to determine the target defogging intensity based on the exposure amount and the first fog concentration, and transmit the target defogging intensity to the image signal processor via the second communication method; The image signal processor is further configured to perform dehazing and noise reduction processing on the first original image according to the target dehazing intensity to obtain a target image, and transmit the target image to the display processing unit through the first communication method so that the display processing unit can perform display enhancement processing on the target image.

[0006] Specifically, when the controller determines the target defogging intensity based on the exposure amount and the first fog concentration, it is used for: Obtain the vehicle's speed; The first correction factor is determined based on the driving speed; The first fog concentration is adjusted according to the first correction coefficient to obtain the second fog concentration; The target defogging intensity is determined based on the exposure amount and the second fog concentration.

[0007] Specifically, when the controller determines the target defogging intensity based on the exposure amount and the second fog concentration, it is used for: Determine the exposure node range in which the exposure amount is located; Determine the fog node range where the second fog concentration is located; Based on the exposure node range and the fog node range, multiple reference defogging intensities are obtained; Determine the first relative position of the exposure amount within the exposure node interval; Determine the second relative position of the first fog concentration within the fog node interval; The target defogging intensity is obtained by fusing the plurality of reference defogging intensities based on the first relative position and the second relative position.

[0008] Specifically, when the controller determines the target defogging intensity based on the exposure amount and the second fog concentration, it is used for: The second correction factor is determined based on the exposure amount; The second fog concentration is adjusted according to the second correction coefficient to obtain the third fog concentration; The target defogging intensity is obtained by performing time-domain filtering on the third fog concentration.

[0009] Wherein, after receiving the target dehazing intensity, the image signal processor is used to perform dehazing and noise reduction processing on the first original image according to the target dehazing intensity to obtain the target image, specifically for: Establish a coordinate mapping relationship between the target dehazing intensity and the first original image; Based on the coordinate mapping relationship, the dehazing intensity of the adjacent regions of each pixel in the first original image is determined; The dehazing intensity of each pixel is determined based on the dehazing intensity of the adjacent regions; The first original image is dehazed based on the dehazing intensity of each pixel to obtain a reference image; The noise reduction intensity of each pixel is determined based on the dehazing intensity of each pixel; The reference image is denoised according to the denoising intensity to obtain the target image.

[0010] Specifically, when the image signal processor determines the denoising intensity of each pixel based on the dehazing intensity of each pixel, it is used to: The first noise amplification factor is determined based on the dehazing intensity of each pixel; Determine the first noise parameter of the first original image; Determine the second noise parameter of the reference image; The second noise amplification factor is determined based on the first noise parameter and the second noise parameter; The reference noise reduction intensity is determined based on the first noise amplification factor and the second noise amplification factor; Determine the edge intensity of each pixel in the reference image; The reference denoising intensity is adjusted based on the edge intensity to obtain the denoising intensity of each pixel.

[0011] Specifically, when the image signal processor performs dehazing processing on the first original image based on the dehazing intensity of each pixel to obtain a reference image, it is used for: Determine the fog category of the first original image; Determine the fog removal strategy based on the fog type; Based on the fog removal strategy and the defogging intensity of each pixel, the first original image is defogging to obtain the reference image.

[0012] Specifically, when the neural network processor determines the first fog concentration based on the second original image, it is used for: Extract the fog distribution features from the second original image; Scene recognition is performed on the second original image to obtain the recognition result; Based on the fog distribution characteristics and the recognition results, the second original image is segmented to obtain multiple image blocks; Fog concentration is detected for the multiple image blocks to obtain the first fog concentration for each image block.

[0013] Secondly, embodiments of this application provide an image quality optimization method applied to a controller of an image quality optimization system. The image quality optimization system further includes an image sensor, an image signal processor, a neural network processor, and a display processing unit, comprising: The exposure of a first raw image is received from the image signal processor, the first raw image being acquired by the image sensor and transmitted to the image signal processor; The system receives a first fog concentration from a second raw image of the neural network processor, wherein the second raw image is a downsampled image of the first raw image transmitted to the neural network processor by the image sensor. The target defogging intensity is determined based on the exposure amount and the first fog concentration, and the target defogging intensity is transmitted to the image signal processor so that the image signal processor performs defogging and noise reduction processing on the first original image to generate the target image, and then the target image is transmitted to the display processing unit for display enhancement processing.

[0014] Thirdly, embodiments of this application provide an image quality optimization device, applied to a controller of an image quality optimization system, wherein the image quality optimization system further includes an image sensor, an image signal processor, a neural network processor, and a display processing unit, comprising: The first receiving unit is used to receive the exposure of a first original image from the image signal processor, wherein the first original image is acquired by the image sensor and transmitted to the image signal processor. The second receiving unit is used to receive the first fog concentration from the second original image of the neural network processor. The second original image is transmitted to the neural network processor after the image sensor performs downsampling processing on the first original image. The determining unit is configured to determine the target defogging intensity based on the exposure amount and the first fog concentration, and transmit the target defogging intensity to the image signal processor so that the image signal processor performs defogging and noise reduction processing on the first original image to generate the target image, and then transmits the target image to the display processing unit for display enhancement processing.

[0015] Fourthly, embodiments of this application provide an electronic device, including a memory, a processor, and executable program code stored in the memory and executable on the processor, wherein the processor executes the executable program code and performs the steps described in the first aspect.

[0016] Fifthly, embodiments of this application provide a computer-readable storage medium storing executable program code, the executable program code including execution instructions for performing the steps described in the first aspect.

[0017] Sixthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of embodiments of this application. The computer program product may be a software installation package.

[0018] As can be seen, in this embodiment, an image hierarchical processing strategy is adopted, in which the image sensor simultaneously sends the first original image and the downsampled second original image, realizing parallel differentiated processing of dual images. This improves the data transmission efficiency of the main link constructed by the image sensor, image signal processor, and display processing unit, and reduces the bandwidth consumption and processing latency of the main link. After the image signal processor completes the exposure calculation and the neural network processor completes the fog detection, both processing results are sent to the controller. The controller determines the defogging intensity adapted to the current driving scenario based on the two processing results, and then sends the defogging intensity parameters to the image signal processor. Due to the small data volume and low bandwidth consumption, even if the transmission is carried out through the second communication method, the system's requirements for transmission efficiency and real-time performance can be met. Finally, the image signal processor achieves accurate defogging and noise reduction processing for different fog conditions, effectively optimizing system resource consumption and improving operating efficiency while ensuring real-time output of vehicle images, thereby improving image clarity and driving visibility safety in foggy environments. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a system architecture diagram of a defogging solution for an in-vehicle vision chip provided in an embodiment of this application; Figure 2 This is a system architecture diagram of another vehicle vision chip defogging solution provided in the embodiments of this application; Figure 3 This is a system architecture diagram of an image quality optimization system provided in an embodiment of this application; Figure 4This is a flowchart illustrating a method for determining the denoising intensity of each pixel using an image signal processor, as provided in an embodiment of this application. Figure 5 This is a flowchart illustrating an image quality optimization method provided in an embodiment of this application; Figure 6 This is a functional unit block diagram of an image quality optimization device provided in an embodiment of this application; Figure 7 This is a functional unit block diagram of another image quality optimization device provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of an electronic device proposed in an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0022] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0023] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0024] During vehicle operation, fog concentration varies and ambient light and color temperature differ significantly, which can easily cause image blurring and reduced visibility. It is necessary to adapt to different fog conditions to ensure image quality. At the same time, the on-board vision chip has strict limitations on data bandwidth and processing latency, and a balance must be struck between imaging effect and bandwidth and latency performance.

[0025] However, traditional defogging solutions cannot adaptively adjust the defogging intensity. Please refer to [link / reference]. Figure 1 , Figure 1 This is a system architecture diagram of a defogging solution for an automotive vision chip provided in an embodiment of this application, as shown below. Figure 1 As shown, the system architecture includes an image sensor, an image signal processor, a display processing unit, and a display terminal, which are connected sequentially via direct links. The image sensor acquires images during vehicle movement and transmits the raw image data to the image signal processor. The image signal processor integrates a dehazing submodule, which performs dehazing processing on the input raw image with fixed parameters. The processed image data is then transmitted from the image signal processor to the display processing unit, which performs display adaptation processing on the image before transmitting it to the display terminal, thus completing the entire process of image acquisition, dehazing processing, and final display. However, in this traditional dehazing solution, the dehazing intensity of the dehazing submodule cannot be adaptively adjusted according to the actual scene; the dehazing function is forcibly enabled. In fog-free or lightly foggy scenes, the fixed-intensity dehazing processing leads to an overall darkening of the image brightness and loss of image details, affecting normal imaging results.

[0026] Furthermore, dehazing can also be performed using a system architecture based on deep learning-based dehazing solutions. Please refer to [link / reference]. Figure 2 , Figure 2 This is a system architecture diagram of another defogging solution for automotive vision chips provided in the embodiments of this application, such as... Figure 2 As shown, the solution includes an image sensor, an image signal processor, a neural network processor integrating a defogging submodule, a display processing unit, and a display terminal. The image sensor and image signal processor are interconnected via a direct hardware connection, indicated by solid arrows, enabling pipelined processing with low transmission latency. Data exchange between the image signal processor and the neural network processor, and between the neural network processor and the display processing unit, requires double data rate synchronous dynamic random access memory (DDR), preventing direct pipelined processing and resulting in higher processing latency, indicated by dashed arrows. The display processing unit and display terminal are also interconnected via a direct hardware connection to ensure low data transmission latency.

[0027] In this data transfer process, the image sensor transmits the raw image data to the image signal processor via a direct hardware link. The image signal processor then transmits the image data to the neural network processor via DDR. The neural network processor runs a deep learning defogging model to detect fog. After processing, the neural network processor sends the defogging data back to the display processing unit via DDR. Finally, the display processing unit performs display adaptation processing and transmits the data to the display terminal via the direct hardware link for image output. While this solution can adapt to scenarios with different fog concentrations and achieve targeted defogging, it requires multiple DDR memory read / write operations and independent inference processing, which significantly consumes system bandwidth and increases data processing latency. This makes it difficult to meet the requirements of in-vehicle electronic rearview mirror systems for low bandwidth, low latency, and low cost.

[0028] To address the aforementioned problems, this application provides an image quality optimization system, method, and apparatus. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0029] Please see Figure 3 , Figure 3 This is a system architecture diagram of an image quality optimization system provided in an embodiment of this application. For example... Figure 3 As shown, the image quality optimization system 100 includes an image sensor 101, an image signal processor 102, a neural network processor 103, a controller 104, and a display processing unit 105.

[0030] The image sensor 101, image signal processor 102, and display processing unit 105 together form the main system link, which are interconnected through a first communication method. This main link is the processing path for vehicle images, used to realize the entire data flow task of image acquisition, defogging and noise reduction optimization, and display output, supporting the display function of the vehicle electronic rearview mirror.

[0031] The first communication method, indicated by a solid arrow, is a high-speed point-to-point dedicated bus communication. The original image and quality-optimized data transmitted on the main link can be directly transmitted point-to-point without the need for relay scheduling through other functional modules or multiple data transfers relying on shared memory. This ensures the continuous and stable transmission process of the main link, avoids increased bandwidth resource consumption, effectively reduces end-to-end transmission latency, meets the real-time display requirements of vehicle electronic rearview mirrors, and avoids the bandwidth loss and latency fluctuation problems caused by traditional relay transmission.

[0032] The image sensor 101 and the neural network processor 103 are connected via a second communication method, the neural network processor 103 and the controller 104 are connected via a first communication method, the image signal processor 102 is unidirectionally connected to the controller 104 via the first communication method, and the controller 104 is unidirectionally connected to the image signal processor 102 via the second communication method.

[0033] The second communication method, indicated by the dashed arrow, is a cache-based relay transmission implemented using the system's shared storage. This method requires the data to be transmitted to be written into the shared storage area for temporary storage before the target module reads the data from the shared storage. The transmission process involves data caching and scheduling, and its real-time performance is lower than that of the first communication method.

[0034] Among them, the image sensor 101 is mainly used to collect visual information of the vehicle's external foggy scene and generate the first raw image. This image is the raw RAW data directly output by the sensor, that is, the raw photosensitive data obtained after the photosensitive element converts light into electrical signals and performs analog-to-digital conversion. It has not undergone image quality processing such as white balance, sharpening, and noise reduction, and retains the original brightness, contrast characteristics and detail information of the image under foggy conditions. The resolution is adapted to the high-definition acquisition requirements of the vehicle's electronic rearview mirror and is used for subsequent defogging and noise reduction processing.

[0035] The image sensor completes initialization configurations such as acquisition frame rate, exposure parameters, and output format after power-on. It temporarily stores the generated first raw image in the internal on-chip buffer and then transmits data through two parallel paths. One path is through the high-speed point-to-point dedicated bus corresponding to the first communication mode. After the raw RAW data is encapsulated and verified, it is directly transmitted to the image signal processor 102. The low latency characteristics of direct transmission ensure that the image data is delivered in real time.

[0036] One of the routing image sensors performs downsampling processing on the first original image, converting high-resolution data into low-resolution data. This reduces the data volume while preserving the overall distribution characteristics of foggy weather, thereby generating a second original image. This low-resolution image is temporarily stored in the system's shared storage area via a second communication method, and then read from it by the neural network processor 103. This reduces the bandwidth usage of the high-speed bus of the first communication method, while also meeting the real-time requirements of the fog detection process, providing lightweight input data for the neural network processor 103 to perform fog concentration detection.

[0037] The image signal processor 102, upon startup, enters a data receiving state, continuously receiving the first raw image transmitted by the image sensor 101. It performs preliminary verification of the received data, confirming that the data transmission is error-free and without loss, before completing the reception of the first raw image. After reception, it analyzes the photosensitivity parameters corresponding to the first raw image, extracting the exposure time, image sensor analog gain, image sensor digital gain, and image signal processor digital gain. Then, according to preset calculation logic, it multiplies the obtained exposure time, image sensor analog gain, image sensor digital gain, and image signal processor digital gain to obtain the exposure value corresponding to the first raw image.

[0038] In addition to calculating the exposure, the system can also indirectly determine the lighting conditions of the shooting environment based on the linear decreasing relationship between the exposure and the ambient illuminance of the camera. This determination result can provide real-time ambient brightness information for the automatic exposure algorithm, thereby dynamically adjusting parameters such as the exposure time of the next frame, the analog gain of the image sensor, the digital gain of the image sensor, and the digital gain of the image signal processor. This ensures that the brightness of the image output by the image sensor 101 is always kept within a suitable range, avoiding overexposure or underexposure.

[0039] Once the exposure is calculated, the exposure data is transmitted to the controller 104 via the first communication method to ensure the timeliness and continuity of data flow.

[0040] The neural network processor 103 performs fog detection-related operations according to the neural network acceleration hardware architecture. First, it reads the second raw image, downsampled by the image sensor, from the system's shared storage area via a second communication method. After acquiring the second raw image, it performs preprocessing operations on the image using an adaptation model, matching the image data to the input standard of the built-in fog detection network model. Then, it initiates neural network inference operations, extracting features from the image based on the pre-trained fog detection model to identify features such as contrast changes and abnormal brightness distribution caused by fog, thereby detecting the fog distribution in the image. After completing the overall fog detection, the neural network processor divides the image into multiple local regions according to preset region division rules, calculating the fog concentration value for each region to determine the first fog concentration in each region of the image.

[0041] After obtaining the first fog concentration in each region, the neural network processor transmits the first fog concentration data to the controller 104 through a first communication method. The transmission characteristics of the first communication method ensure the timeliness of data transmission and provide support for the controller to perform subsequent calculations in conjunction with the exposure parameters.

[0042] The controller 104 internally carries and runs a firmware program. This firmware is pre-programmed code with a fixed operating environment. The controller executes the firmware code instructions line by line to perform mathematical calculations on various image-related parameters and logical scheduling operations between modules. All calculations and scheduling behaviors follow the firmware's preset program logic. The firmware program also includes a scene adaptation application for real-time identification of the actual scene corresponding to the current vehicle image, such as highway driving, urban road, tunnel, and nighttime driving scenarios. Based on the differences in the characteristics of different scenes, the firmware adjusts the built-in calculation parameters and logical weights, converting scene features into adjustment signals that can participate in calculations and feeding them back to the firmware's calculation logic to ensure that subsequent parameter calculations are more closely aligned with the current scene requirements.

[0043] The controller receives two externally transmitted parameter data: one is the exposure parameter transmitted from the image signal processor, and the other is the first fog concentration parameter transmitted from the neural network processor. After receiving the two types of parameters, the controller first preprocesses the parameters, and then adjusts the built-in calculation logic based on the scene feedback from the scene adaptation application. It performs joint calculation on the preprocessed exposure and the first fog concentration. During the calculation, auxiliary parameters such as vehicle speed can also be combined to complete coefficient correction and numerical adjustment. Finally, the target defogging intensity adapted to the current image scene is calculated. This value can characterize the defogging processing strength of each local area after the first original image is segmented.

[0044] After the target defogging intensity is generated, the controller initiates data transmission through the second communication method. After the controller writes the data into the shared storage, the image signal processor reads the data from the shared storage. This realizes one-way data transmission from the controller to the image signal processor. Since the transmitted data volume is small and the bandwidth usage is low, even if the transmission is carried out through the second communication method, the system's requirements for transmission efficiency and real-time performance can be met, reducing the occupation of transmission resources of the first communication method.

[0045] The image signal processor 102 is also used to receive the target dehazing intensity sent by the controller 104, and perform a dehazing operation on the first original image according to the target dehazing intensity. At the same time, the dehazing operation will amplify the noise signal in the image to varying degrees. Therefore, the image signal processor will continue to perform noise reduction processing on the dehazed image. After continuous processing of dehazing and noise reduction, a target image that meets the display requirements is finally generated.

[0046] After the target image is generated, the image signal processor transmits the image data to the display processing unit 105 through the first communication method. After receiving the target image output by the image signal processor 102, the display processing unit 105 performs display enhancement processing on the target image. The processing operations include brightness adjustment, contrast optimization, color correction and sharpening adjustment. The processed image can adapt to the output characteristics of the vehicle display panel and finally complete the image output of the vehicle electronic rearview mirror.

[0047] Furthermore, the image quality optimization system 100 also includes a display terminal 106, which is interconnected with the display processing unit 105 through a first communication method, and is used to display the output image of the display processing unit 105.

[0048] As can be seen, this application adopts an image hierarchical processing strategy, in which the image sensor simultaneously sends a first original image and a downsampled second original image, realizing parallel differentiated processing of dual images. This improves the data transmission efficiency of the main link constructed by the image sensor, image signal processor, and display processing unit, and reduces the bandwidth consumption and processing latency of the main link. After the image signal processor completes the exposure calculation and the neural network processor completes the fog detection, both processing results are sent to the controller. The controller determines the defogging intensity adapted to the current driving scenario based on the two processing results, and then sends the defogging intensity parameters to the image signal processor. Due to the small data volume and low bandwidth consumption, even if the transmission is carried out through the second communication method, the system's requirements for transmission efficiency and real-time performance can be met. Finally, the image signal processor achieves precise defogging and noise reduction processing for different fog conditions, effectively optimizing system resource consumption and improving operating efficiency while ensuring real-time output of vehicle images, thereby improving image clarity and driving visibility safety in foggy environments.

[0049] In one possible embodiment, the system architecture of this application is not limited to dehazing. Other vehicle image quality optimization scenarios, such as contrast enhancement, halo suppression, and dynamic range compression, can all be implemented using the current system architecture.

[0050] In one possible embodiment, when the controller determines the target defogging intensity based on the exposure amount and the first fog concentration, it is specifically configured to: acquire the vehicle's driving speed; determine a first correction coefficient based on the driving speed; adjust the first fog concentration based on the first correction coefficient to obtain a second fog concentration; and determine the target defogging intensity based on the exposure amount and the second fog concentration.

[0051] The controller collects the vehicle's current speed parameters in real time. This speed is the instantaneous speed of the vehicle in real time. The speed data is continuously acquired at fixed intervals to ensure the real-time performance and validity of the speed parameters.

[0052] This includes calculating the speed difference between the real-time driving speed and the preset speed. The preset speed can be dynamically determined based on different external environmental conditions and driving conditions. For example, it can be configured differently based on environmental conditions such as current light intensity and weather type, or based on the driving scenario such as highways, urban roads, rural roads, and tunnels. It can also adaptively adjust the preset speed in conjunction with the vehicle's driving mode.

[0053] The controller performs time-series analysis on the driving speed acquired from multiple consecutive frames to determine the speed change trend, such as constant speed, acceleration, or deceleration. Simultaneously, it calculates the speed fluctuation amplitude per unit time to obtain the speed fluctuation intensity. Based on the speed difference, speed fluctuation intensity, speed change trend, and their corresponding preset weights, a first correction coefficient for the current vehicle speed state is determined.

[0054] Specifically, the first fog concentration output by the neural network processor is multiplied by the first correction coefficient to complete the vehicle speed adaptive correction of the fog concentration, thus obtaining the second fog concentration.

[0055] In one possible embodiment, when the controller determines the target defogging intensity based on the exposure amount and the second fog concentration, it is specifically configured to: determine the exposure node interval where the exposure amount is located; determine the fog node interval where the second fog concentration is located; obtain a plurality of reference defogging intensities based on the exposure node interval and the fog node interval; determine a first relative position of the exposure amount within the exposure node interval; determine a second relative position of the first fog concentration within the fog node interval; and fuse the plurality of reference defogging intensities based on the first relative position and the second relative position to obtain the target defogging intensity.

[0056] Specifically, the real-time exposure value is compared with the values ​​of each exposure node in turn to determine which two adjacent exposure nodes the exposure value falls between, thus obtaining the exposure node interval corresponding to the exposure value; and the second fog concentration is compared with the values ​​of each fog concentration node in turn to determine which two adjacent fog concentration nodes the second fog concentration falls between, thus obtaining the fog node interval corresponding to the second fog concentration.

[0057] The process involves retrieving a pre-stored mapping table between exposure, fog concentration, and defogging intensity. Based on the exposure node interval and the fog node interval, a rectangular region formed by the intersection of these two intervals is located in the mapping table. Multiple preset reference defogging intensities corresponding to the vertices of this rectangular region are extracted, including a first reference defogging intensity, a second reference defogging intensity, a third reference defogging intensity, and a fourth reference defogging intensity. Specifically, the first reference defogging intensity is determined based on the upper limit points of both the exposure and fog node intervals; the second reference defogging intensity is determined based on the upper limit points of both the exposure and fog node intervals; the third reference defogging intensity is determined based on the lower limit points of both the exposure and fog node intervals; and the fourth reference defogging intensity is determined based on the lower limit points of both the exposure and fog node intervals.

[0058] Specifically, based on the real-time exposure and the two endpoint values ​​of the exposure node interval, the offset of the exposure within the interval is calculated to obtain the first relative position, which includes the first difference between the real-time exposure and the upper limit value, and the second difference between the real-time exposure and the lower limit value.

[0059] Specifically, based on the values ​​of the two endpoints of the second fog concentration and the fog node interval, the offset of the second fog concentration within the interval is calculated to obtain the second relative position, which includes the third difference between the second fog concentration and the upper limit point value, and the fourth difference between the second fog concentration and the lower limit point value.

[0060] First, based on the first relative position, linear interpolation is performed on multiple sets of reference defogging intensities along the exposure dimension to obtain two intermediate interpolation results, namely a0 and a1. Then, based on the second relative position, a second linear interpolation is performed on the above intermediate interpolation results along the fog concentration dimension. Through bilinear interpolation fusion processing, the target defogging intensity adapted to the current exposure and fog concentration is finally obtained.

[0061] Specifically, the difference between the third reference defogging intensity and the first reference defogging intensity is calculated, multiplied by the first difference, and then divided by the second difference to obtain the first interpolation component. The first reference defogging intensity is then added to the first interpolation component to obtain the intermediate interpolation result a0. Similarly, the difference between the fourth reference defogging intensity and the second reference defogging intensity is calculated, multiplied by the first difference, and then divided by the second difference to obtain the second interpolation component. The second reference defogging intensity is then added to the second interpolation component to obtain the intermediate interpolation result a1. Next, the difference between the intermediate interpolation result a1 and the intermediate interpolation result a0 is calculated, multiplied by the third difference, and then divided by the fourth difference to obtain the third interpolation component. The intermediate interpolation result a0 is then added to the third interpolation component to obtain the target defogging intensity.

[0062] For example, to finely control the defogging intensity by combining fog concentration and exposure, a two-dimensional lookup table Dehaze_str

[16]

[16] can be set up to store the defogging intensity. The exposure and fog concentration are divided into 16 nodes, where the rows correspond to the exposure node Exposure

[16] and the columns correspond to the fog concentration node Haze_Str

[16] . The target defogging intensity is calculated using bilinear interpolation. First, based on the real-time exposure and fog concentration, it is determined that the exposure falls between the m-m+1 exposure nodes and the fog concentration falls between the n-n+1 fog concentration nodes. The intermediate interpolation results a0 and a1 are calculated along the exposure dimension, as follows: a0=Dehaze_str[m][n]+(Exposure-Exposure[m]) ×(Dehaze_str[m+1][n]-Dehaze_str[m][n]) / (Exposure[m+1]-Exposure[m]); a1=Dehaze_str[m][n+1]+(Exposure-Exposure[m]) ×(Dehaze_str[m+1][n+1]-Dehaze_str[m][n+1]) / (Exposure[m+1]-Exposure[m]).

[0063] Then, a second interpolation is performed along the fog concentration dimension to obtain the final defogging intensity, as shown in the following formula: Dehaze_Str=a0+(Haze_Str-Haze_Str[n])×(a1-a0) / (Haze_Str[n+1]-Haze_Str[n]).

[0064] As can be seen, in this embodiment, the fog concentration is adaptively corrected based on vehicle speed, and the defogging intensity is calculated based on both exposure and fog concentration. This allows the defogging intensity to dynamically match driving conditions, ambient brightness, and fog conditions, improving scene adaptability. Furthermore, the defogging intensity calculation is completed with only lightweight numerical calculations and small data transmissions, eliminating the need to transmit entire frames of images or occupy significant bus bandwidth. This reduces system computing power and storage overhead, avoids transmission congestion and latency, and fully guarantees real-time processing and stable output of in-vehicle images. In summary, this embodiment effectively improves image clarity and contrast in foggy conditions while optimizing system resource consumption, providing drivers with more reliable rear visibility and further enhancing driving safety.

[0065] In one possible embodiment, when the controller determines the target defogging intensity based on the exposure amount and the second fog concentration, it specifically performs the following steps: determining a second correction coefficient based on the exposure amount; adjusting the second fog concentration based on the second correction coefficient to obtain a third fog concentration; and performing time-domain filtering on the third fog concentration to obtain the target defogging intensity.

[0066] Multiple exposure thresholds are preset to divide the exposure into multiple ranges, such as severely underexposed, moderately underexposed, slightly underexposed, normal exposure, slightly overexposed, moderately overexposed, and severely overexposed ranges. Each exposure range corresponds to a correction coefficient.

[0067] Among these, low-contrast whitening in foggy weather is easily misjudged as overexposure due to exposure levels. Therefore, image contrast can be introduced as an auxiliary judgment to weight the correction coefficient and improve the robustness of fog detection.

[0068] Specifically, the contrast index of the image is calculated, such as gray-level variance, gradient mean, and brightness difference between bright and dark areas. A mapping relationship between contrast and weighting factor is established. The exposure correction coefficient determined based on the exposure amount is multiplied by the contrast weighting factor to obtain the second correction coefficient.

[0069] Specifically, the second fog concentration is corrected by a second correction factor to obtain the accurate fog concentration after eliminating exposure interference. This is achieved by multiplying the second correction factor by the second fog concentration to obtain the third fog concentration.

[0070] The fog concentration is a real-time value of inter-frame fluctuations. Directly using it for defogging may cause image flickering and intensity jumps. Therefore, temporal filtering is used to eliminate noise and abrupt changes, outputting a stable, continuous, and flicker-free target defogging intensity. Specifically, the target defogging intensity of the current frame is calculated by weighted fusion of the filter coefficient, the target defogging intensity of the previous frame, and the third fog concentration of the current frame. That is, target defogging intensity = filter coefficient × target defogging intensity of the previous frame + (1 - filter coefficient) × third fog concentration. The smoothing process can be completed frame by frame according to this iterative filtering rule, thereby effectively suppressing inter-frame fluctuations in defogging intensity and preventing image flickering.

[0071] In one possible embodiment, when the neural network processor determines the first fog concentration based on the second original image, it specifically performs the following steps: extracting fog distribution features from the second original image; performing scene recognition on the second original image to obtain a recognition result; segmenting the second original image based on the fog distribution features and the recognition result to obtain multiple image blocks; and performing fog concentration detection on the multiple image blocks to obtain the first fog concentration for each image block.

[0072] The system utilizes a pre-trained lightweight feature extraction network to extract multi-scale spatial features from the second original image, focusing on capturing visual features strongly correlated with fog. Then, through operations such as convolution and pooling, redundant information is compressed, outputting a fog distribution feature map representing fog density and coverage. This map identifies different distribution patterns, including localized thin fog, dense fog, and edge fog areas, adapting to various in-vehicle fog conditions such as rain fog, haze fog, and backlight fog. Subsequently, the model's built-in scene classification branch performs inference calculations to identify the current in-vehicle driving scene category and outputs scene recognition results, such as highways, urban roads, rural roads, tunnels (inside and outside), backlighting, low-light conditions at night, and rain-soaked areas.

[0073] In this process, adaptive region segmentation is performed on the second original image by combining the density of fog distribution features with prior information from scene recognition. For example, in areas with uniform fog distribution and open scenes, the image is segmented into larger image blocks; in areas with uneven fog distribution and complex scenes, the image is segmented into smaller image blocks; and in fog-free or clear areas, the image blocks are appropriately merged. Finally, multiple image blocks are obtained.

[0074] In this process, each segmented image block is treated as an independent processing unit and input into the fog concentration detection model. The fog features of each image block are numerically fitted through a fully connected layer or a lightweight convolutional layer, and the fog concentration value is output to obtain the first fog concentration of each image block, forming a set of regional fog concentrations corresponding to the image block grid.

[0075] As can be seen, in this embodiment, adaptive image segmentation using fog features and scene information can refine the segmentation in complex fog areas to ensure accuracy, while simplifying the segmentation in uniform fog areas or open scenes to reduce computational load, thus ensuring detection accuracy and processing efficiency. Detecting the first fog concentration at the regional level for each image block accurately reflects the uneven distribution of actual fog, while simultaneously reducing the computational power consumption and inference latency of the neural network module.

[0076] In one possible embodiment, after receiving the target dehazing intensity, the image signal processor is configured to perform dehazing and noise reduction processing on the first original image according to the target dehazing intensity to obtain the target image. Specifically, this is configured to: establish a coordinate mapping relationship between the target dehazing intensity and the first original image; determine the dehazing intensity of the adjacent regions of each pixel in the first original image according to the coordinate mapping relationship; determine the dehazing intensity of each pixel according to the dehazing intensity of the adjacent regions; perform dehazing processing on the first original image according to the dehazing intensity of each pixel to obtain a reference image; determine the noise reduction intensity of each pixel according to the dehazing intensity of each pixel; and perform noise reduction processing on the reference image according to the noise reduction intensity to obtain the target image.

[0077] The target dehazing intensity is represented by a region-level grid. Based on the grid size of the target dehazing intensity and the resolution of the first original image, a coordinate scaling transformation relationship is constructed. This maps the spatial coordinates of each pixel in the first original image to the grid coordinates corresponding to the target dehazing intensity, establishing a spatial correspondence between the two. For example, in this embodiment, the target dehazing intensity uses a 128×72 region-level grid, and the first original image is a high-resolution image (1280×720) captured by the vehicle. A mapping relationship is constructed based on the resolution ratio to determine the corresponding position of each pixel within the grid.

[0078] Specifically, based on the coordinate mapping relationship, the position of a single pixel within the dehazing intensity grid is located, and the dehazing intensity of the surrounding neighborhood at that position is extracted, completing the matching and docking of the pixel with the dehazing intensity of its neighborhood. Based on the dehazing intensities of multiple neighborhoods of that pixel, a spatial weighted fusion calculation is performed to convert the region-level dehazing intensity into a pixel-by-pixel dehazing intensity consistent with the resolution of the first original image.

[0079] The image signal processor includes a dehazing unit and a denoising unit. The pixel-by-pixel dehazing intensity is used as an adjustment parameter. The dehazing unit performs dehazing on the first original image and outputs a reference image. Dehazing amplifies image noise, so the denoising intensity needs to be matched to the degree of noise amplification. Specifically, a correlation is established between dehazing intensity and denoising intensity. An appropriate denoising intensity is assigned to each pixel based on its dehazing intensity, forming a pixel-by-pixel denoising intensity parameter. This pixel-by-pixel denoising intensity is input to the denoising unit, which performs denoising on the reference image based on this intensity, ultimately obtaining the target image.

[0080] As can be seen, in this embodiment, establishing coordinate mapping enables a precise spatial correspondence between the dehazing intensity of a region and the pixels of the original image. By weighted fusion of the intensities of adjacent regions, a pixel-by-pixel dehazing intensity is obtained. Adaptive dehazing is then performed based on this pixel-by-pixel dehazing intensity, which avoids blocky artifacts and improves the naturalness of the image. Combining the dehazing intensity with the pixel-by-pixel noise reduction intensity allows for targeted control of noise amplified during the dehazing process. This preserves image details while reducing noise, improving the clarity and stability of foggy imaging and optimizing the imaging effect of the vehicle's electronic rearview mirror.

[0081] In one possible embodiment, please refer to Figure 4 , Figure 4 This is a flowchart illustrating a method for determining the denoising intensity of each pixel using an image signal processor, as provided in an embodiment of this application. Figure 4 The steps for determining the denoising intensity of each pixel based on the dehazing intensity of each pixel are shown below: S410, determine the first noise amplification factor based on the dehazing intensity of each pixel.

[0082] In this process, a mapping relationship between dehazing intensity and noise amplification coefficient is established. Based on the dehazing intensity of each pixel, the first noise amplification coefficient corresponding to that pixel is calculated. The dehazing intensity and this coefficient are positively correlated, that is, the higher the dehazing intensity, the larger the first noise amplification coefficient.

[0083] S420, determine the first noise parameter of the first original image.

[0084] In the first original image, a neighborhood region is selected centered on each pixel, and the gray-level variance, gradient standard deviation, or mean difference between adjacent pixels in the neighborhood is calculated to obtain the first noise parameter, which is used to characterize the noise level of the original image.

[0085] Alternatively, a high-pass filter can be applied to the first original image to extract noise components and obtain a first noise parameter; or the pixel fluctuation value of a uniform dark region in the image can be calculated as the first noise parameter.

[0086] S430, determine the second noise parameter of the reference image.

[0087] The second noise parameter of the dehazed reference image is determined using the same calculation method as the first noise parameter, and is used to characterize the actual noise level of the image after dehazing.

[0088] S440, determine the second noise amplification factor based on the first noise parameter and the second noise parameter.

[0089] Specifically, the ratio of the second noise parameter to the first noise parameter is calculated, and this ratio is used as the second noise amplification factor to obtain the actual noise amplification magnitude brought about by the defogging treatment.

[0090] S450, determine the reference noise reduction intensity based on the first noise amplification factor and the second noise amplification factor.

[0091] One approach is to select the larger of the two noise amplification coefficients as the reference denoising strength; another approach is to adaptively allocate weights according to the scene conditions and fuse the two coefficients to obtain the reference denoising strength.

[0092] S460, determine the edge intensity of each pixel in the reference image.

[0093] One approach is to use an edge detection operator to calculate the gradient magnitude of a pixel to obtain its edge strength. A larger gradient magnitude indicates a more pronounced edge feature at the pixel's location. Alternatively, the sum of the absolute differences in grayscale between the current pixel and its neighboring pixels can be calculated and used as the edge strength of the current pixel. Another approach is to use the Laplacian operator to calculate the second-order derivative and use the absolute value of the derivative as the edge strength.

[0094] S470, the reference denoising intensity is adjusted according to the edge intensity to obtain the denoising intensity of each pixel.

[0095] Specifically, for pixels with high edge strength, the reference denoising intensity is reduced by a preset ratio; for pixels with low edge strength, the reference denoising intensity is increased by a preset ratio, thereby obtaining the denoising intensity of each pixel.

[0096] As can be seen, in this embodiment, the noise reduction requirements of each pixel can be accurately matched, and the noise changes during the dehazing process can be matched. While suppressing noise, the image edge details are preserved, avoiding image problems caused by unreasonable noise reduction intensity, and improving the integrity and purity of the imaging effect.

[0097] In one possible embodiment, when the image signal processor is used to perform dehazing processing on the first original image according to the dehazing intensity of each pixel to obtain a reference image, it is specifically used to: determine the fog category of the first original image; determine a fog removal strategy according to the fog category; and perform dehazing processing on the first original image according to the fog removal strategy and the dehazing intensity of each pixel to obtain the reference image.

[0098] The fog category is determined based on the distribution and numerical differences of fog concentration in different regions of the image, such as light fog, dense fog, rain fog, haze fog, localized fog, and uniform fog. Different fog categories are matched with different fog removal strategies. The fog removal strategy is based on the defogging intensity of a single pixel, and the application method of adjusting the defogging intensity is adapted to the processing needs of different fog conditions.

[0099] For example, if there is uniform fog across the entire area, the corresponding fog removal strategy can be to globally and gently adapt the defogging intensity, adjusting it slightly and proportionally according to the defogging intensity value of each pixel. This eliminates the effects of fog while avoiding color shifts and loss of details in dark areas.

[0100] In this process, after determining the removal strategy that matches the fog type, the first original image is dehazed pixel by pixel by combining the strategy requirements and the dehazing intensity of each pixel to complete the dehazing optimization reference image. This image can achieve a natural and stable dehazing effect while adapting to different fog types.

[0101] In one possible embodiment, please refer to Figure 5 , Figure 5 This is a flowchart illustrating an image quality optimization method provided in an embodiment of this application. The method is applied to the controller of an image quality optimization system, which further includes an image sensor, an image signal processor, a neural network processor, and a display processing unit, such as... Figure 5 As shown, the method includes the following steps: S510, receive the exposure of the first raw image from the image signal processor.

[0102] The first original image is acquired by the image sensor and then transmitted to the image signal processor.

[0103] The exposure value is obtained by the image signal processor from the first original image. It reflects the ambient light level at the time of image acquisition. The value of the exposure value is negatively correlated with the subsequent adjustment of the defogging intensity. The lower the illumination, the smaller the exposure value. From a debugging perspective, the defogging intensity should not be too high to avoid excessive amplification of noise during the defogging process. Conversely, the larger the exposure value, the lower the corresponding defogging intensity can be appropriately reduced to balance the defogging effect and noise control. The image signal processor then sends the exposure value to the controller through the inter-module interaction channel.

[0104] S520, receiving the first fog concentration from the second raw image of the neural network processor.

[0105] The second original image is the image sensor that downsamples the first original image and then transmits it to the neural network processor.

[0106] In order to reduce the computational load of the neural network processor, the image sensor performs downsampling processing on the first original image to reduce the resolution, thereby obtaining a second original image. The second original image is then transmitted to the neural network processor, which extracts fog distribution features from the second original image and divides the image into multiple image blocks based on the scene recognition results. Fog detection is performed on each image block to obtain the corresponding first fog concentration. Subsequently, the first fog concentration data containing fog information of each region is sent to the controller.

[0107] S530, determine the target defogging intensity based on the exposure amount and the first fog concentration.

[0108] The target dehazing intensity is transmitted to the image signal processor, so that the image signal processor performs dehazing and noise reduction processing on the first original image to generate the target image, and then the target image is transmitted to the display processing unit for display enhancement processing.

[0109] The controller normalizes the exposure and the first fog concentration using its internal firmware program, and adjusts the fog concentration value by combining it with a correction coefficient corresponding to the vehicle's driving speed to obtain the target defogging intensity adapted to different image areas. Then, the target defogging intensity is written to shared storage for relay transmission. The image signal processor reads the target defogging intensity from the shared storage and performs regional defogging operation on the first original image based on the intensity. At the same time, it performs corresponding noise reduction processing based on the noise amplification during the defogging process, thereby generating a target image with optimized image quality. The image signal processor then directly transmits the target image to the display processing unit, which performs brightness, contrast, and color enhancement processing on the target image, and finally completes the output of the in-vehicle electronic rearview mirror.

[0110] As can be seen, in this embodiment, the controller synchronously acquires image exposure and fog concentration information, adaptively calculates the target defogging intensity to match the actual scene, and with the orderly data interaction between multiple modules, the image defogging and noise reduction processing can adapt to changes in lighting and fog conditions. At the same time, based on a stable transmission mechanism, it can continuously output real-time vehicle display images, improving the overall effect of image quality optimization.

[0111] For examples consistent with the above embodiments, please refer to... Figure 6 , Figure 6 This is a functional unit block diagram of an image quality optimization device provided in an embodiment of this application, such as... Figure 6 As shown, the image quality optimization device 60 is applied to the controller of an image quality optimization system. The image quality optimization system further includes an image sensor, an image signal processor, a neural network processor, and a display processing unit. The image quality optimization device 60 includes: a first receiving unit 61, used to receive the exposure of a first original image from the image signal processor, the first original image being acquired by the image sensor and transmitted to the image signal processor; a second receiving unit 62, used to receive the first fog concentration of a second original image from the neural network processor, the second original image being downsampled by the image sensor and transmitted to the neural network processor; and a determining unit 63, used to determine the target defogging intensity based on the exposure and the first fog concentration, and transmit the target defogging intensity to the image signal processor, so that the image signal processor performs defogging and noise reduction processing on the first original image to generate a target image, and then transmits the target image to the display processing unit for display enhancement processing.

[0112] It is understood that since the method embodiments and the device embodiments are different presentations of the same technical concept, the content of the method embodiment section in this application should be adapted to the device embodiment section in a synchronous manner, and will not be repeated here.

[0113] In the case of using integrated units, please refer to Figure 7 , Figure 7 This is a functional unit block diagram of another image quality optimization device provided in the embodiments of this application, such as... Figure 7 As shown, the image quality optimization device 60 includes a processing module 602 and a communication module 601. The processing module 602 controls and manages the operation of the image quality optimization device 60, for example, executing the steps of the first receiving unit 61, the second receiving unit 62, and the determining unit 63, and / or performing other processes of the technology described herein. The communication module 601 is used for interaction between the image quality optimization device 60 and other devices. Wherein, as... Figure 7 As shown, the image quality optimization device 60 may further include a storage module 603, which is used to store the program code and data of the image quality optimization device 60.

[0114] The processing module 602 may be a processor or controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0115] The communication module 601 can be a transceiver, RF circuit, or communication interface, etc. The storage module 603 can be a memory.

[0116] All relevant content for each scenario involved in the above method embodiments can be referenced from the functional descriptions of the corresponding functional modules, and will not be repeated here. The above image quality optimization device 60 can perform the above... Figure 5 The image quality optimization method shown.

[0117] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device proposed in an embodiment of this application, as shown below. Figure 8As shown, the electronic device 800 includes a processor 810, a memory 820, a communication interface 830, and one or more programs 821. The one or more programs 821 are stored in the memory and configured to be executed by the processor. When the program is executed, it includes some or all of the steps of any image quality optimization method described in the above method embodiments. The processor, memory, and communication interface are interconnected and complete communication between them.

[0118] The memory can be volatile memory such as Dynamic Random Access Memory (DRAM) or non-volatile memory such as a hard disk drive. The memory stores a set of executable program code, and the processor calls the executable program code stored in the memory to execute some or all of the steps of the image quality optimization process performed by any image quality optimization system as described in the above-described image quality optimization method embodiments.

[0119] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any image quality optimization process performed by the image quality optimization system described in the above method embodiments, wherein the computer includes an electronic device.

[0120] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include an electronic device.

[0121] It should be noted that, for the sake of simplicity, the aforementioned methods are described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are optional, and the actions and modules involved are not necessarily essential to this application.

[0122] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0123] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0124] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0125] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0126] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0127] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0128] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The above description of the embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image quality optimization system, characterized in that, It includes an image sensor, an image signal processor, a neural network processor, a controller, and a display processing unit; The image sensor is used to transmit a first raw image acquired to the image signal processor via a first communication method, and to perform downsampling processing on the first raw image to obtain a second raw image, and to transmit the second raw image to the neural network processor via a second communication method, wherein the real-time performance of the first communication method is higher than that of the second communication method; The image signal processor is used to determine the exposure of the first original image and transmit the exposure to the controller through the first communication method; The neural network processor is configured to determine a first fog concentration based on the second original image, and transmit the first fog concentration to the controller via the first communication method; The controller is configured to determine the target defogging intensity based on the exposure amount and the first fog concentration, and transmit the target defogging intensity to the image signal processor via the second communication method; The image signal processor is further configured to perform dehazing and noise reduction processing on the first original image according to the target dehazing intensity to obtain a target image, and transmit the target image to the display processing unit through the first communication method so that the display processing unit can perform display enhancement processing on the target image.

2. The system according to claim 1, characterized in that, When the controller is used to determine the target defogging intensity based on the exposure amount and the first fog concentration, it is specifically used for: Obtain the vehicle's speed; The first correction factor is determined based on the driving speed; The first fog concentration is adjusted according to the first correction coefficient to obtain the second fog concentration; The target defogging intensity is determined based on the exposure amount and the second fog concentration.

3. The system according to claim 2, characterized in that, When the controller determines the target defogging intensity based on the exposure amount and the second fog concentration, it is specifically used for: Determine the exposure node range in which the exposure amount is located; Determine the fog node range where the second fog concentration is located; Based on the exposure node range and the fog node range, multiple reference defogging intensities are obtained; Determine the first relative position of the exposure amount within the exposure node interval; Determine the second relative position of the first fog concentration within the fog node interval; The target defogging intensity is obtained by fusing the plurality of reference defogging intensities based on the first relative position and the second relative position.

4. The system according to claim 2, characterized in that, When the controller determines the target defogging intensity based on the exposure amount and the second fog concentration, it is specifically used for: The second correction factor is determined based on the exposure amount; The second fog concentration is adjusted according to the second correction coefficient to obtain the third fog concentration; The target defogging intensity is obtained by performing time-domain filtering on the third fog concentration.

5. The system according to claim 1, characterized in that, After receiving the target dehazing intensity, the image signal processor is used to perform dehazing and noise reduction processing on the first original image according to the target dehazing intensity to obtain the target image. Specifically, it is used to: Establish a coordinate mapping relationship between the target dehazing intensity and the first original image; Based on the coordinate mapping relationship, the dehazing intensity of the adjacent regions of each pixel in the first original image is determined; The dehazing intensity of each pixel is determined based on the dehazing intensity of the adjacent regions; The first original image is dehazed based on the dehazing intensity of each pixel to obtain a reference image; The noise reduction intensity of each pixel is determined based on the dehazing intensity of each pixel; The reference image is denoised according to the denoising intensity to obtain the target image.

6. The system according to claim 5, characterized in that, When the image signal processor determines the denoising intensity of each pixel based on the dehazing intensity of each pixel, it is specifically used for: The first noise amplification factor is determined based on the dehazing intensity of each pixel; Determine the first noise parameter of the first original image; Determine the second noise parameter of the reference image; The second noise amplification factor is determined based on the first noise parameter and the second noise parameter; The reference noise reduction intensity is determined based on the first noise amplification factor and the second noise amplification factor; Determine the edge intensity of each pixel in the reference image; The reference denoising intensity is adjusted based on the edge intensity to obtain the denoising intensity of each pixel.

7. The system according to claim 5, characterized in that, The image signal processor is used to perform dehazing processing on the first original image according to the dehazing intensity of each pixel to obtain a reference image, specifically for: Determine the fog category of the first original image; Determine the fog removal strategy based on the fog type; Based on the fog removal strategy and the defogging intensity of each pixel, the first original image is defogging to obtain the reference image.

8. The system according to claim 1, characterized in that, When the neural network processor determines the first fog concentration based on the second original image, it is specifically used for: Extract the fog distribution features from the second original image; Scene recognition is performed on the second original image to obtain the recognition result; Based on the fog distribution characteristics and the recognition results, the second original image is segmented to obtain multiple image blocks; Fog concentration is detected for the multiple image blocks to obtain the first fog concentration for each image block.

9. An image quality optimization method, characterized in that, A controller for an image quality optimization system, the image quality optimization system further comprising an image sensor, an image signal processor, a neural network processor, and a display processing unit, including: The exposure of a first raw image is received from the image signal processor, the first raw image being acquired by the image sensor and transmitted to the image signal processor; The system receives a first fog concentration from a second raw image of the neural network processor, wherein the second raw image is a downsampled image of the first raw image transmitted to the neural network processor by the image sensor. The target defogging intensity is determined based on the exposure amount and the first fog concentration, and the target defogging intensity is transmitted to the image signal processor so that the image signal processor performs defogging and noise reduction processing on the first original image to generate the target image, and then the target image is transmitted to the display processing unit for display enhancement processing.

10. An image quality optimization device, characterized in that, A controller for an image quality optimization system, the image quality optimization system further comprising an image sensor, an image signal processor, a neural network processor, and a display processing unit, including: The first receiving unit is used to receive the exposure of a first original image from the image signal processor, wherein the first original image is acquired by the image sensor and transmitted to the image signal processor. The second receiving unit is used to receive the first fog concentration from the second original image of the neural network processor. The second original image is transmitted to the neural network processor after the image sensor performs downsampling processing on the first original image. The determining unit is configured to determine the target defogging intensity based on the exposure amount and the first fog concentration, and transmit the target defogging intensity to the image signal processor so that the image signal processor performs defogging and noise reduction processing on the first original image to generate the target image, and then transmits the target image to the display processing unit for display enhancement processing.

Images