Image processing method and system, vehicle and computer readable storage medium

Abstract

The invention discloses an image processing method and system, a vehicle and a computer readable storage medium. The method comprises the steps of firstly determining a dark channel value of each pixel point in an input first image, then obtaining a dark primary color value of each pixel point according to the dark channel value of each pixel point, and finally performing defogging processing on the first image based on the dark primary color values to obtain a second image. Therefore, the dark channel value of each pixel point of the first image is directly determined, and the dark primary color value is determined according to the dark channel value, so that the problem that the dark primary color value is wrongly calculated in a scene depth mutation region can be avoided, operations such as transmittance image refinement can be avoided, and the halo effect of the defogged image is avoided.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an image processing method, system, vehicle, and computer-readable storage medium. Background Art

[0002] With the rapid development of intelligent vehicles, more and more vehicles are being equipped with onboard cameras. These cameras are used to display and record information about the vehicle's surroundings and identify road and traffic signs for driving assistance. However, inclement weather (such as fog, haze, and rain) can severely degrade image quality, with useful details buried in the haze, hindering the extraction of critical information. In severe cases, this can lead to the failure of intelligent driving functions, vehicle accidents, and safety hazards. Defogging technology removes the interference of fog from images through specific means, thereby obtaining higher-quality images and capturing more effective image information. However, limitations of dark channel prior algorithms in related defogging techniques, inaccurate transmittance and atmospheric light estimation, and improper image edge processing can easily lead to a halo effect in defogged images. Summary of the Invention

[0003] Embodiments of the present application provide an image processing method, system, vehicle, and computer-readable storage medium, which reduce the halo effect of image defogging processing to at least partially solve the above-mentioned technical problems.

[0004] In order to achieve the above-mentioned object, according to a first aspect of the present application, there is provided an image processing method, comprising:

[0005] For an input first image, determining a dark channel value of each pixel of the first image;

[0006] Obtaining a dark primary color value of each pixel point according to a dark channel value of each pixel point in the first image;

[0007] Dehazing is performed on the first image based on the dark primary color value to obtain a second image.

[0008] According to a second aspect of the present application, there is provided an image processing system, comprising:

[0009] a memory configured to store instructions; and

[0010] The processor is configured to call the instructions from the memory and implement the above-mentioned image processing method when executing the instructions.

[0011] According to a third aspect of the present application, a vehicle is provided, comprising an on-board image acquisition device and the above-mentioned image processing system, wherein the image processing system is configured to perform image processing on a first image acquired by the on-board image acquisition device to obtain a second image.

[0012] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which instructions are stored. When the instructions are executed by a processor, the processor is configured to perform the above-mentioned image processing method.

[0013] The present application first determines the dark channel value of each pixel in the input first image, then obtains the dark primary color value based on the dark channel value of each pixel, and finally dehazes the first image based on the dark primary color value to obtain the second image. In this way, by directly determining the dark channel value of each pixel in the first image and using it to determine the dark primary color value, the present application can avoid dark primary color value calculation errors in areas with sudden changes in scene depth and avoid operations such as transmittance map refinement, thereby avoiding the halo effect in the dehazed image.

[0014] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0016] In order to more completely understand the present application and its beneficial effects, the following description will be given in conjunction with the accompanying drawings, wherein the same drawing numbers represent the same parts in the following description.

[0017] Figure 1 A schematic diagram of an application environment of an image processing method provided in an embodiment of the present application;

[0018] Figure 2 is a flowchart of an image processing method provided in an embodiment of the present application;

[0019] Figure 3 is a flowchart of an image processing method provided in a specific embodiment of the present application;

[0020] Figure 4 A schematic diagram of the structure of an image processing system provided in an embodiment of the present application;

[0021] Figure 5This is a schematic structural diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0023] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly specifying the number of the technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more of the described features. In the description of this application, "plurality" means two or more, unless otherwise specifically qualified. In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is provided to enable anyone skilled in the art to implement and use the present application. In the following description, details are listed for illustrative purposes. It should be understood that one of ordinary skill in the art will recognize that the present application can be implemented without these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0024] The image processing method in the embodiments of the present application can be a method for processing video captured by an on-board camera. On-board video refers to video information captured by a camera system installed on a vehicle, used to record and monitor the vehicle's surroundings and the conditions inside the vehicle. In inclement weather, defogging technology is often used to remove contaminants from the video, thereby improving image quality.

[0025] Related technologies typically use a dark-primary prior dehazing algorithm based on dark-primary theory and an atmospheric scattering model to perform dehazing. For example, a foggy image is divided into N*N blocks to obtain local and global dark-primary maps. Assuming the atmospheric light composition is known, the dark-primary map can be used to roughly estimate the transmittance map. This transmittance map is then refined using software cutout methods. Next, the atmospheric light composition is estimated using a dark-primary prior. Finally, the foggy image is dehazed using a physical model of the atmospheric scattering fog map and fixed parameters.

[0026] However, refining the transmittance map through software clipping and solving a large-scale sparse linear system of equations incurs high time and space complexity, resulting in the algorithm's primary application to single-image dehazing research and unsuitable for real-time video processing. Furthermore, the dark primary theory does not apply to the sky region. The calculated transmittance of the sky region is often lower than the actual value, which can easily lead to anomalies in the dehazed sky region, such as color distortion and artifacts. Furthermore, the dehazed image is prone to appearing dark and oversaturated.

[0027] Based on this, the embodiment of the present application provides a method for determining the dark primary color value of an image based on the dark channel value of the pixel points of the image, based on the dark channel value of each pixel point. There is no need to obtain the dark primary color value in blocks or regions, which can avoid the problem of dark primary color value calculation errors in areas where the scene depth suddenly changes, and can also reduce operations such as transmittance map refinement, thereby avoiding the aperture (halo) effect in the dehazed image.

[0028] Furthermore, the present embodiment further filters the dark channel values, reducing local errors in the image's dark primary colors and minimizing detail and contrast loss in high-frequency areas after defogging, without increasing the amount of computation. Furthermore, the present embodiment can estimate the distribution of fog in the scene based on the grayscale histogram distribution information of the scene image, maximizing defogging and effectively minimizing anomalies in the sky area.

[0029] The image processing method in the embodiment of the present application can be run on a local terminal device or a server. When the data processing method is run on a server, the method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device.

[0030] In order to better understand the image processing method, system, vehicle and computer-readable storage medium provided in the embodiments of the present application, the application environment applicable to the embodiments of the present application is described below.

[0031] See also Figure 1 , Figure 1 The following is a schematic diagram showing an application environment of the image processing method provided by an embodiment of the present application. Figure 1 Taking the server 110 shown in the figure as an example, the server 110 can be connected to the terminal device 120 via a network. The network is used to provide a medium for a communication link between the server 110 and the terminal device 120. The network can include various connection types, such as wired communication links, wireless communication links, etc., which are not limited in the embodiments of the present application. Optionally, in other embodiments, the image processing method can also be run on other types of electronic devices such as smartphones and laptops.

[0032] It should be understood that Figure 1 The server 110, network and terminal device 120 are merely illustrative. Any number of servers, networks and terminal devices can be configured according to implementation needs. For example, the server 110 can be a physical server or a server cluster composed of multiple servers, and the terminal device 120 can be a mobile phone, tablet, desktop computer, laptop computer and the like. As an example, the server 110 can be a cloud server that communicates remotely with the vehicle and is used to provide data services to the vehicle. It is understandable that the embodiments of the present application can also allow multiple terminal devices 120 to access the server 110 at the same time.

[0033] The following is a detailed description of each step in conjunction with the accompanying drawings. In this embodiment, a server is used as an example. It should be noted that the order in which the following embodiments are described does not limit the preferred order of the embodiments. Although the flowcharts illustrate a logical order, in some cases, the steps shown or described may be performed in a different order than that shown in the accompanying drawings.

[0034] Figure 2 FIG. 1 is a flow chart of an image processing method provided in an embodiment of the present application. Figure 2 As shown, the image processing method includes steps 201-203, which are described in detail below.

[0035] Step 201: For an input first image, determine a dark channel value of each pixel of the first image.

[0036] In an embodiment of the present application, the first image refers to an original foggy image with fog captured by an on-board image acquisition device on a vehicle, such as an on-board camera. After acquiring the first image, the on-board image acquisition device first determines the dark channel value of each pixel point of the first image. Among them, the dark channel value refers to the minimum grayscale value of the pixel point in the three channels of red channel (R), green channel (G) and blue channel (B) in the foggy image, which can reflect the dark channel characteristics of the pixel point under foggy conditions. In a local area of ​​a fog-free image, the pixel value of at least one color channel is very low, so the dark channel value can be used to defog and enhance the contrast of the image.

[0037] Step 202: Obtain a dark primary color value of each pixel in the first image according to the dark channel value of each pixel.

[0038] Among them, in a possible implementation, in step 202, the dark channel values ​​corresponding to each pixel in the first image can be filtered to obtain the dark primary color value of each pixel, and then the dark primary color value of the entire first image can be obtained, and the final dark primary color map of the first image can be obtained for subsequent defogging. It should be understood that in this embodiment, the dark channel value of each pixel can be directly used as the dark primary color value of each pixel, and the method of filtering each dark channel value can also be understood as performing high-frequency area filtering on the dark primary color map of the first image obtained after the dark channel value is used as the dark primary color value. It can be achieved by introducing filtering to reduce local errors in the high-frequency area of ​​the dark primary color map, so as to reduce the detail loss and contrast loss after defogging in the high-frequency area, and will not increase too much computational complexity.

[0039] Similarly, in other embodiments, filtering may not be performed and the dark channel value of each pixel may be directly used as its corresponding dark primary color value, which may be determined according to the actual application scenario. In this embodiment, the dark channel value of each pixel needs to be filtered to obtain the dark primary color value for exemplary explanation.

[0040] In this embodiment, the method of filtering the dark channel values ​​corresponding to each pixel point in the first image is not limited. For example, it can be implemented by median filtering, mean filtering, extreme value filtering, etc., but is not limited thereto.

[0041] As an example, considering that the dark channel value prior is not applicable for large white objects such as the sky, and to reduce local errors in the dark color map and minimize detail and contrast loss after dehazing high-frequency areas, it is necessary to filter the dark channel values ​​of multiple pixels to obtain the dark color values ​​of the first image. Dark color values ​​refer to the phenomenon that the values ​​of certain channels in the first image are particularly low under specific conditions. They are used to describe the color characteristics of certain areas, thereby identifying specific areas.

[0042] In the related art, the dark primary color value is usually obtained by fixing or moving blocks, which can easily lead to errors in the calculation of the dark primary color value in the area where the scene depth suddenly changes, such as the situation where the side with a low dark primary color value invades the other side of the edge, thereby causing a corresponding offset at the edge of the transmittance graph and a halo effect. Therefore, in one implementation of the embodiment of the present application, multiple dark channel values ​​can be filtered by median filtering to obtain the dark primary color value of the first image. Median filtering replaces the value of a pixel with the median value in the neighborhood of a pixel, thereby effectively removing the noise in the area and maintaining the edge features of the first image. An exemplary implementation method for median filtering will be introduced below.

[0043] Step 203: Defogging the first image based on the dark primary color value to obtain a second image.

[0044] In this embodiment, the second image refers to the clear image obtained by dehazing the first image. Determining the dark primary color value using a single-pixel median filter can reduce noise and minimize loss of detail and contrast after dehazing. This approach also mitigates the aperture effect caused by traditional block or region processing, reduces computational complexity, and is suitable for real-time video applications.

[0045] For example, suppose a car camera captures a first image under foggy conditions, which contains a white car. By determining the dark channel value of each pixel in the first image, a map reflecting the dark channel characteristics of the image can be obtained. This dark channel map is then subjected to a median filter to remove noise and optimize the distribution of dark channel values, obtaining the dark primary color value. Finally, using the dark primary color prior theory and an atmospheric scattering model, the original foggy image is dehazed based on the dark primary color value, resulting in a clear dehazed image, which makes the outline and details of the white car more clearly visible.

[0046] As can be seen from the above, the embodiment of the present application, by processing dark channel values ​​in a manner that better preserves image edge information and reduces the halo phenomenon caused by local depth of field mutations, thereby improving the efficiency of image dehazing and ensuring image quality. Furthermore, if a median filter is used, compared to complex algorithms such as guided filtering, the calculation speed is faster and the resource consumption is reduced, achieving the dual effects of dehazing efficiency and image quality improvement.

[0047] In step 201, for each pixel, a first grayscale value of the red channel, a second grayscale value of the blue channel, and a third grayscale value of the green channel are obtained. Then, the minimum grayscale value among the first grayscale value, the second grayscale value, and the third grayscale value is used as the dark channel value of the pixel.

[0048] In an embodiment of the present application, the first grayscale value refers to the grayscale value of the red channel of the first image before defogging, the second grayscale value refers to the grayscale value of the blue channel of the first image before defogging, and the third grayscale value refers to the grayscale value of the green channel of the third image before defogging. For obtaining the dark channel value of each pixel, the minimum grayscale value among the first grayscale value, the second grayscale value and the third grayscale value can be selected as the dark channel value of the pixel. Grayscale values ​​are usually represented by integers between 0 and 255, where 0 can represent black and 255 can represent white. The grayscale values ​​of the three channels are compared, and the minimum grayscale value is taken as the dark channel value of the pixel. The method is simple and efficient, and can accurately reflect the dark channel characteristics of the pixel in the foggy image, providing basic data for subsequent defogging.

[0049] In one example, the dark channel value can be calculated using the following formula:

[0050]

[0051] Wherein, (x, y) is the coordinate value of the pixel point in the first image, dc(x, y) is the dark channel value of the pixel point with coordinates (x, y), I c (x, y) is the grayscale value of the pixel with coordinates (x, y) in channel c, R is the red channel of the pixel, G is the green channel of the pixel, and B is the blue channel of the pixel.

[0052] For example, for a specific pixel P(x,y) in the first image, its red channel's first grayscale value is r(x,y) = 100, its blue channel's second grayscale value is b(x,y) = 120, and its green channel's third grayscale value is g(x,y) = 80. By comparing these three values, the minimum value, g(x,y) = 80, is taken as the pixel's dark channel value. This dark channel value represents the grayscale value of the darkest of the three channels under foggy conditions, reflecting the extent of fog impact on the pixel. Based on this, the same processing is performed on every pixel in the first image to obtain the dark channel value for each pixel in the entire first image. This set of dark channel values ​​represents the lowest grayscale value for each pixel in the first image across different color channels, providing a data foundation for subsequent dehazing. For example, in dehazing algorithms, dark channel values ​​can help determine the distribution of fog within an image.

[0053] In step 202, a plurality of dark channel values ​​may be subjected to horizontal median filtering to obtain a first dark primary color value of the first image. Then, a plurality of dark channel values ​​may be subjected to vertical median filtering to obtain a second dark primary color value of the first image. Finally, the dark primary color value of the first image may be obtained based on the first and second dark primary color values.

[0054] As another example, when median filtering is performed on the dark channel values ​​of each pixel in the first image, median filtering can be performed in both horizontal and vertical directions. Horizontal median filtering can effectively address horizontal noise and detail in the first image, while vertical median filtering can effectively address vertical noise and detail in the first image. The dark primary color value of the first image obtained by horizontal median filtering is a first dark primary color value, and the dark primary color value of the first image obtained by vertical median filtering is a second dark primary color value. The first and second dark primary color values ​​are then combined to obtain the dark primary color value of the entire first image. By performing median filtering twice in different directions, the noise in the dark primary color image of the first image can be more comprehensively reduced, reducing detail and contrast loss in the dehazed image, and improving the stability of the dehazing effect. In the implementation of this application, horizontal median filtering can be performed first, followed by vertical median filtering. Alternatively, vertical median filtering can be performed first, followed by horizontal median filtering. This is not limited here.

[0055] As an example, horizontal filtering may include the following steps. First, obtain a pre-set first filter window, where the first filter window refers to a window for median filtering in the horizontal direction. The first filter window covers an odd number of pixels in the horizontal direction. For example, it can be set to a first filter window with a row and column size of 1×5. Then, on the first image, slide the first filter window horizontally on each row of pixels in the first image to traverse each pixel of the first image. For each sliding operation, sort the first pixel in the first filter window to obtain a first sorting result. The first sorting result is the sorting result of all first pixel points in the first filter window. For example, arrange in order from large to small or from small to large. Then, according to the first sorting result, obtain the dark channel value of the first pixel in the middle position as the first dark primary color value corresponding to the currently traversed pixel.

[0056] For example, suppose the dark channel value sequence for the first horizontal pixel row in the first image is [40, 20, 10, 30, 50, 70, 60, 80, 100, 90]. Use a horizontal median filter window covering five pixels and slide the window from left to right, sorting the five values ​​within the window each time and taking the middle value. For example, the values ​​in the first window are 40, 20, 10, 30, and 50, which are sorted to 10, 20, 30, 40, and 50, with the middle value of 30 serving as the first dark primary value for the currently traversed pixel. Similarly, slide the horizontal median filter window over all pixels in the row in this manner, sorting the dark channel values ​​within the window each time and taking the middle value. This yields the first dark primary value for all pixels in the row after horizontal median filtering. Repeat this process for each pixel row in the first image, ultimately yielding the first dark primary value sequence for the entire first image. Each first dark primary color value in the first dark primary color value sequence can reflect the dark primary color of the first image after median filtering in the horizontal direction, which helps to reduce noise and outliers in the horizontal direction. It should be understood that when traversing to the last few (such as the last 4) pixels in the pixel row, the number of pixels covered by the horizontal median filter window in the pixel row will be smaller than the size of the window. In this case, an interpolation method can be used to make the number of pixels in the window traversed each time reach the size corresponding to the window. For example, when traversing to the fourth to last pixel, a pixel can be inserted on the back side to perform a window sliding operation. The inserted pixel can be the first pixel in the next pixel row or the last pixel in the previous pixel row. By analogy, when traversing the third to last, second to last, and first pixel, a similar interpolation method can also be used to achieve this, which will not be repeated here.

[0057] As another example, vertical filtering may include the following steps. First, a pre-set second filter window is obtained. The second filter window refers to a window for performing median filtering in the vertical direction. The second filter window covers an odd number of pixels in the vertical direction. For example, the first filter window can be set to a row and column size of 5×1. Then, on the first image, the second filter window is vertically slid across each column of pixels in the first image to traverse each pixel in the first image. For each sliding operation, the second pixels within the second filter window are sorted according to the dark channel values ​​corresponding to each second pixel within the second filter window to obtain a second sorting result. The second sorting result is the sorting result of all second pixels within the second filter window. For example, the order is from largest to smallest or from smallest to largest. Next, based on the second sorting result, the dark channel value of the second pixel in the middle is obtained as the second dark primary color value corresponding to the currently traversed pixel. For example, assume that the sliding window starts from the first pixel in the nth column of pixels. When the window slides to a certain position, the dark channel values ​​of the five pixels within the window are 60, 90, 70, 80, and 100, respectively. These five dark channel values ​​are sorted to 60, 70, 80, 90, and 100. The dark channel value of the middle pixel, 80, is used as the second primary dark color value for the corresponding position in the column after vertical median filtering. Similarly, by sliding the window over all pixels in the column, sorting the dark channel values ​​of the pixels within the window each time and taking the median value, we can obtain the second primary dark color value for all pixels in the column after vertical median filtering. The same operation is performed for each column of the first image, ultimately resulting in a sequence of second primary dark color values ​​for the entire first image. Each second primary dark color value in this sequence reflects the vertical primary dark color of the first image after median filtering, helping to reduce vertical noise and outliers.

[0058] Similar to the median filtering in the horizontal direction, vertical median filtering is performed on the pixel columns in the vertical direction to obtain a second dark primary color value sequence. Finally, the two sequences are fused to obtain the final dark primary color value of the first image for subsequent dehazing processing. For example, after obtaining the first dark primary color value and the second dark primary color value corresponding to each pixel of the first image, the final dark primary color value can be obtained by weighted average calculation. For example, the first dark primary color value and the second dark primary color value corresponding to each pixel point can be first obtained, and then the first dark primary color value and the second dark primary color value are weightedly fused according to the first weight and the second weight pre-set for the first dark primary color value and the second dark primary color value, to obtain the final dark primary color value of each pixel point, and then the dark primary color value of the first image is obtained. Among them, the dark primary color value of the first image includes the dark primary color value corresponding to each pixel point in the first image.

[0059] As an example, assume that the first weight of the first dark primary color value in the comprehensive calculation is 0.4, and the second weight of the second dark primary color value is 0.6. For a pixel in an image, its first dark primary color value is a and its second dark primary color value is b. Then, the dark primary color value of this pixel is c = 0.4*a + 0.6*b. By performing the corresponding calculation for all pixels in the first image, the dark primary color value of each pixel in the entire first image can be obtained.

[0060] By comprehensively considering the results of horizontal and vertical median filtering, this approach can more comprehensively reduce noise and local errors in the dark primary image, providing more accurate dark primary information for subsequent dehazing. This reduces noise in the dark primary image of the first image, thus reducing noise in the dehazed image. Furthermore, it also mitigates local errors in the dark primary image, reducing detail and contrast loss in high-frequency areas after dehazing.

[0061] For another example, in another embodiment of step 202, also using the entire image as an example, the median filtering process can also be performed on a set of dark channel values ​​to remove noise and outliers, making the image's dark primary color values ​​smoother and more accurate. As an example, assume that in a 3*3 local area, the dark channel values ​​are 80, 90, 100, 110, 120, 130, 140, 150, and 160, respectively. These values ​​are sorted as 80, 90, 100, 110, 120, 130, 140, 150, and 160, and the value in the middle is 120. Then, the dark primary color value of the center pixel of this 3*3 area after median filtering is 120. Similar operations are performed on all such local areas in the entire first image to obtain the dark primary color value of the entire first image. Compared to the image obtained by directly using the dark channel values, the image composed of these dark primary color values ​​reduces noise and local errors and is more suitable for subsequent dehazing processing.

[0062] For another example, each pixel point in the first image can be targeted in turn, and then the pixel point can be used as a reference to obtain multiple target pixel points in the set image area, and the average value of the dark channel values ​​corresponding to the multiple target pixel points can be calculated as the dark original color value of the currently traversed pixel point. The set image area can be a rectangular area with the currently traversed pixel point as the center. For example, for each pixel point, 9 target pixel points in a 3*3 area with the pixel point as the center can be used, and then the average value of the dark channel values ​​of the 9 target pixel points can be calculated as the dark original color value of the pixel point. The set image area can also be a rectangular area containing the currently traversed pixel point, such as if one of the vertices of the rectangular area is the currently traversed pixel point. This embodiment does not specifically limit this.

[0063] In step 203, the following steps 2031-2033 may be included, which are described in detail as follows.

[0064] Step 2031: Obtain a grayscale histogram of the first image. In this embodiment, the grayscale histogram can reflect the distribution of pixels of different grayscale values ​​in the first image.

[0065] As an example, by obtaining the grayscale histogram of the first image, the number of pixels at each grayscale level in the first image can be counted. Assuming the grayscale range of the first image is from 0 to 255, each pixel in the first image is traversed and the grayscale value of each pixel is recorded. For example, when a pixel with a grayscale value of 50 is traversed, the count of grayscale level 50 is incremented by 1. This operation is repeated for all pixels in the first image, resulting in an array containing 256 elements. Each element corresponds to a grayscale level, and its value represents the number of pixels at that grayscale level in the first image. This array, the grayscale histogram of the first image, can intuitively reflect the distribution of different grayscale levels in the first image, providing data support for the subsequent determination of dehazing parameters. Furthermore, by analyzing the grayscale histogram, information such as the overall brightness distribution of the image and areas of concentrated grayscale can be obtained, which can be used to subsequently determine appropriate dehazing parameters.

[0066] Step 2032: Determine defogging parameters for the first image based on the grayscale histogram. In this embodiment, the defogging parameters are parameters used for subsequent defogging processing. As an example, the defogging parameters in this embodiment of the application may include a transmittance adjustment factor, a transmittance lower limit, and a tolerance range.

[0067] As an example, after obtaining the grayscale histogram of the first image, the dehazing parameters can be determined based on the grayscale histogram. For example, the grayscale distribution range, effective grayscale number, grayscale variance and other information of the first image can be first determined according to the grayscale histogram. For example, the minimum grayscale value and the maximum grayscale value of the pixels in the first image are found as the two endpoints of the grayscale distribution range through the grayscale histogram, and then the grayscale distribution range is determined. By counting the number of grayscale levels whose number of pixels is greater than the set proportion of the total pixels (for example, 0.02%), the effective grayscale number is obtained, and then the grayscale variance is calculated based on the number of pixels and grayscale value of each grayscale level in the grayscale histogram and the average brightness of the first image, and the transmittance adjustment factor is determined based on the information such as the grayscale distribution range, the effective grayscale number and the grayscale variance.

[0068] As an example only, the grayscale variance can be calculated as follows:

[0069]

[0070] Among them, grayVar represents the grayscale variance (which can also be understood as the grayscale distribution variance of the grayscale histogram), grayAve is the average brightness of the first image, hisSum is the sum of the histogram, which identifies the total number of pixels in the first image, and hist(i) is the number of pixels at the i-th grayscale level.

[0071] Among them, the average brightness grayAve can be expressed as:

[0072]

[0073] For example, if the grayscale distribution range is large, it means that the image has large grayscale differences, and a larger transmittance adjustment factor may be required to enhance the dehazing effect. If the number of valid grayscales is large, it means that the image's grayscale distribution is relatively uniform, and the transmittance adjustment factor can be relatively small. At the same time, the image's atmospheric light value and grayscale variance are determined based on the grayscale histogram. The atmospheric light adjustment factor is then determined based on the atmospheric light value, and the grayscale variance adjustment factor is determined based on the grayscale variance. Finally, the tolerance range is determined by combining the set minimum value, set maximum value, atmospheric light adjustment factor, and grayscale variance adjustment factor. For example, the minimum value is set to 48 and the maximum value is set to 72. In this way, by comprehensively considering the atmospheric light adjustment factor and grayscale variance adjustment factor, an appropriate tolerance range is determined to ensure that the dehazing process does not over- or under-dehaze, especially to prevent anomalies in areas such as the sky.

[0074] Step 2033: Defogging the first image based on the defogging parameters to obtain a second image.

[0075] Compared to the related art, where defogging is performed with fixed values, the embodiments of the present application can flexibly determine the defogging parameters for the defogging process based on the grayscale histogram, and adaptively adjust the defogging intensity according to actual conditions, thereby improving the adaptability and stability of the defogging effect. The transmittance adjustment factor is used to adjust the defogging intensity overall. The lower limit of the transmittance is used to limit the maximum defogging intensity to prevent anomalies in the sky area. The tolerance range is used to limit the defogging intensity in the near-sky area to prevent color distortion. Then, the image is defogged based on these parameters to obtain a clear second image.

[0076] In one example, the transmittance adjustment factor can be calculated by first determining the grayscale distribution range, effective grayscale number, and grayscale variance of the first image based on the grayscale histogram, and then determining the transmittance adjustment factor based on the grayscale distribution range, effective grayscale number, and grayscale variance.

[0077] The gray-scale distribution range can represent the difference between the maximum and minimum gray-scale values of pixel points in the first image, reflecting the contrast of the image. The effective number of gray scales can represent the number of pixel points with valid gray-scale values in the first image. For example, the number of gray-scale levels with the number of pixels greater than 0.02% of the total number of pixels reflects the richness of the gray scale of the image. The gray-scale variance can represent the degree of dispersion of the gray-scale values of the image, that is, the size of the difference in the bright-dark distribution of the image gray scale, reflecting the richness of details of the image. Then, the transmittance adjustment factor is determined according to these parameters. For example, when the gray-scale distribution range is wide, the effective number of gray scales is large, and the gray-scale variance is large, it indicates that the image has rich details and high contrast, and the transmittance adjustment factor can be appropriately increased to enhance the defogging effect. On the contrary, when the gray-scale distribution range is narrow, the effective number of gray scales is small, or the gray-scale variance is small, the transmittance adjustment factor is reduced to avoid image distortion caused by excessive defogging.

[0078] Specifically, the perspective rate adjustment factor can be calculated by the following formula:

[0079]

[0080] where w is the perspective rate adjustment factor, grayRange is the gray-scale distribution range, grayValid is the effective number of gray scales, and grayVar is the gray-scale variance. w1 is a constant set for the transmittance adjustment factor, which can be determined according to the actual situation. a1, a2, and a3 are contrast constants set based on the gray-scale variance grayVar, and a1 < a2 < a3. N is a proportional constant set according to the values of a1, a2, and a3. For example, if a_1, a_2, and a_3 are 200, 800, and 3200 respectively, then N can take the value of 100, and the specific value is not limited.

[0081] The above formula comprehensively considers factors such as the gray-scale distribution range, effective number of gray scales, and gray-scale variance of the first image, uses conditional judgment and mathematical operations, and calculates the corresponding values of the transmittance adjustment factor according to the parameter combinations under different conditions. For example, when the gray-scale distribution range is less than a1 and the effective number of gray scales is less than one-third of the gray-scale distribution range, and when the gray-scale variance is less than or equal to 800, the value of the transmittance adjustment factor is w1. When the gray-scale variance is greater than a2 and less than a3, the value of the transmittance adjustment factor is w1 - (gray-scale variance - a2) / N. When the gray-scale variance is greater than or equal to a3, the transmittance adjustment factor is 0. ch

[0082] In one example, the calculation of the tolerance range can be obtained by the following steps. First, based on the grayscale histogram, the atmospheric light value and grayscale variance of the first image are determined. The atmospheric light value represents the brightness value of the brightest area in the first image, which usually corresponds to the sky or other bright objects. Then, based on the atmospheric light value, the atmospheric light adjustment factor of the first image is determined. Next, based on the grayscale variance, the grayscale variance adjustment factor of the first image is determined. Finally, based on the set minimum value, the set maximum value, the atmospheric light adjustment factor and the grayscale variance adjustment factor, the tolerance range of the first image is determined. The tolerance range is used to limit the adjustment range of the transmittance to prevent image anomalies caused by excessive adjustment during the defogging process.

[0083] In the related art, the calculation of the atmospheric light value usually takes the average of the maximum values ​​of the dark primary color images of adjacent frames, which is difficult to adapt to scene changes and easily leads to calculation errors or fails to achieve a smooth effect. Therefore, in the embodiment of the present application, the calculation of the atmospheric light value can be obtained by the following steps. First, the brightness value of each pixel in the grayscale histogram is obtained, and the pixels are sorted according to the size of the brightness value, and multiple candidate pixels are determined based on the sorting results. Then, the atmospheric light value of the first image is determined based on the brightness values ​​corresponding to the candidate pixels. For example, based on the sorting results, the pixels that rank in the top set percentage can be selected as candidate pixels, such as the pixels with the highest brightness of 0.1% (ranked in the top thousandth) as candidate pixels. For example, assuming that there are 1 million pixels in the grayscale histogram of the first image, after sorting from high to low according to the brightness value, the top 0.1%, or 1,000 pixels, are taken as candidate pixels. The average brightness value of these 1,000 pixels is calculated, assuming it is 200 (range 0-255), then the atmospheric light value A=200. This value will be used in subsequent transmittance calculations and dehazing to ensure accurate processing of the brightest areas in the image. The brightness calculation here can be the average brightness of the red, green, and blue channels to prevent flickering in the dehazed video caused by sudden changes in atmospheric light values ​​when the scene changes.

[0084] In one example, the lower limit of transmittance may be 128.

[0085] In one example, the atmospheric light adjustment factor can be calculated using the following formula:

[0086]

[0087] Among them, ratio_A is the atmospheric light adjustment factor, A is the atmospheric light value, r1, r2, r3, r4, and r5 are factor constants set for the atmospheric light adjustment factor, and A0, A1, A2, A3, and A4 are atmospheric light value comparison parameters set based on the average brightness value. For example, according to the value range of the average brightness value, the value range of A0, A1, A2, A3, and A4 can be [180, 255], and A0>A1>A2>A3>A4.

[0088] This segmented adjustment of atmospheric light values ​​can more flexibly adapt to defogging requirements under different brightness conditions and optimize transmittance calculation and defogging effects.

[0089] In an example, the grayscale variance adjustment factor can be calculated using the following formula:

[0090]

[0091] Among them, ratio_Var is the grayscale variance adjustment factor, grayVar is the grayscale variance, v1, v2, v3, v4, and v5 are factor constants set for the grayscale variance adjustment factor, and v1>v2>v3>v4>v5, g1, g2, g3, and g4 are variance comparison constants set for the grayscale variance, and g1>g2>g3>g4.

[0092] This segmented adjustment method can more accurately reflect the grayscale distribution characteristics of the first image, optimize the determination of the tolerance range and the defogging effect.

[0093] In an example, the tolerance range can be calculated using the following formula:

[0094] R=R_min(R_max-R_min)*(ratio_A / 1024)*(ratio_Var / 1024)

[0095] Among them, R is the tolerance range, R_min is the set minimum value, R_max is the set maximum value, ratio_A is the atmospheric light adjustment factor, and ratio_Var is the grayscale variance adjustment factor.

[0096] The above formula comprehensively considers the set minimum and maximum values, the atmospheric light adjustment factor, and the grayscale variance adjustment factor. By normalizing the adjustment factors to a range of 0–1 and combining them with the set minimum and maximum values, the tolerance range is calculated. This method dynamically adjusts the tolerance range, improving the flexibility and adaptability of the dehazing process and preventing image anomalies caused by excessive transmittance adjustment.

[0097] In an embodiment of the present application, defogging is performed on the first image based on the defogging parameters to obtain the second image, which may include the following steps. First, the transmittance of each pixel in the first image is determined based on the dark primary color value, the defogging parameters and the atmospheric light value of the first image. The transmittance reflects the proportion of light that is not scattered when transmitted in the atmosphere and is a key parameter in the defogging process. Then, based on the transmittance of each pixel in the first image and the atmospheric light value of the first image, the target grayscale value of each pixel in the first image after defogging is determined. The target grayscale value is the grayscale value obtained after the pixel is defogged. Finally, based on the target grayscale value of each pixel in the first image after defogging, the second image is obtained.

[0098] The transmittance can be calculated using the following formula:

[0099]

[0100] Where t(x, y) is the transmittance of the pixel (x, y), dc(x, y) is the dark channel value of the pixel (x, y), w is the perspective adjustment factor, t0 is the lower limit of the transmittance, R is the tolerance range, and A is the atmospheric light value.

[0101] This dynamic adjustment method can more accurately reflect the transmission characteristics of pixels and optimize the dehazing effect.

[0102] In the embodiment of the present application, the target grayscale value may include a fourth grayscale value of the red channel, a fifth grayscale value of the blue channel, and a sixth grayscale value of the green channel of the pixel after dehazing. The fourth grayscale value is the grayscale value of the red channel of the pixel of the second image, the fifth grayscale value is the grayscale value of the blue channel of the pixel of the second image, and the sixth grayscale value is the grayscale value of the green channel of the pixel of the second image.

[0103] The fourth grayscale value, the fifth grayscale value, and the sixth grayscale value can be calculated using the following formulas:

[0104]

[0105] Among them, r_o(x, y) is the fourth grayscale value of the pixel point (x, y), g_o(x, y) is the fifth grayscale value of the pixel point (x, y), b_o(x, y) is the sixth grayscale value of the pixel point (x, y), r_i(x, y) is the first grayscale value of the pixel point (x, y), b_i(x, y) is the second grayscale value of the pixel point (x, y), g_i(x, y) is the third grayscale value of the pixel point (x, y), and t(x, y) is the transmittance of the pixel point (x, y).

[0106] Based on the atmospheric scattering model, the grayscale values ​​of the original image, atmospheric light values, and transmittance are used to calculate the grayscale values ​​of the restored fog-free image. This effectively removes fog from the first image, improving its clarity and detail.

[0107] The second image after defogging may appear dark. Based on this, in the embodiment of the present application, histogram equalization processing can be performed on the pixels, such as peak shaving and valley filling. Specifically, the following steps 204-207 may be included.

[0108] Step 204: Obtain a brightness component histogram of the second image.

[0109] As an example, after completing the dehazing process to obtain the second image, in order to further optimize the visual effect of the image, the brightness component histogram of the second image can be obtained. Taking the Y component in the common YUV color space as an example, the Y component represents the brightness information of the image. After the second image is converted to the YUV color space, each pixel in the second image can be traversed to extract the value of its Y component. For example, for the pixel point P(i,j) located in the i-th row and j-th column in the image, its Y component value Y(i,j) is obtained. The Y component values ​​of all pixels are counted to create an array. The index of the array corresponds to the value range of the Y component (usually 0-255), and the value of the array element represents the number of pixels in the image where the Y component value appears.

[0110] Assume the image has 10,000 pixels. A statistical analysis reveals that there are 200 pixels with a Y component value of 50, 150 pixels with a Y component value of 51, and so on. This array forms the Y component histogram of the second image, also known as the luminance component histogram. This intuitively displays the distribution of different luminance values ​​within the image, providing an important basis for subsequent analysis of the image's luminance characteristics and targeted processing. By analyzing the luminance component histogram, we can understand information such as the image's overall luminance level, areas of concentrated luminance, and the degree of luminance dispersion, providing data support for operations such as adjusting the brightness and contrast of the second image.

[0111] Step 205 : Determine a first pixel and a second pixel according to the first brightness value of each pixel in the brightness component histogram.

[0112] For example, based on the acquired luminance component histogram of the second image, a first valid grayscale and a second valid grayscale of the luminance component histogram are first determined. As an example, the first valid grayscale may be a grayscale value increasing direction from 0 to 255, and the first grayscale value in the histogram exceeding 0.02% of the total number of pixels may be represented by hist_t0; the second valid grayscale is a grayscale value decreasing direction from 255 to 0, and the first grayscale value in the histogram exceeding 0.02% of the total number of pixels may represent the highest valid grayscale, for example, may be represented by hist_t1.

[0113] For example, if the total number of pixels in an image is 10,000, 0.02% is 200 pixels. Starting from grayscale level 0 of the luminance component histogram, pixel count is calculated. When the number of pixels at a grayscale level, such as grayscale level 30, exceeds 200 for the first time, 30 is the first valid grayscale, and hist_t0 = 30. Starting from grayscale level 255, pixel count is calculated in reverse order. When the number of pixels at a grayscale level, such as grayscale level 220, exceeds 200 for the first time, 220 is the second valid grayscale, and hist_t1 = 220.

[0114] Then, a grayscale threshold is determined based on the first effective grayscale and the second effective grayscale.

[0115] As an example, the grayscale threshold can be set using the following formula.

[0116]

[0117] Wherein, hist_cutThr is a set grayscale threshold, where hist_cutThr0 and hist_cutThr1 are preset parameters. As an example, hist_cutThr0 = pixNum * 0.2%, hist_cutThr1 = pixNum * 0.3%, where pixNum is the total number of pixels, where h1 and h2 are preset comparison parameters for the first valid grayscale and the second valid grayscale, respectively, and h1>h2.

[0118] As another example, the average value of the two may be taken as the grayscale threshold. For example, if the first valid grayscale is 30 and the second valid grayscale is 220, then the grayscale threshold is set to (30+220)÷2=125.

[0119] For each pixel, if the brightness value of the pixel is greater than the set grayscale threshold, the pixel is regarded as the first pixel; if the brightness value of the pixel is less than or equal to the set grayscale threshold, the pixel is regarded as the second pixel. For example, for pixel A, its brightness value is 130, which is greater than the set grayscale threshold of 125, so pixel A is classified as the first pixel; for pixel B, its brightness value is 120, which is less than the set grayscale threshold, so pixel B is classified as the second pixel. In this way, the pixels in the image are divided into two categories, where the first pixel refers to a pixel with a higher grayscale value, which may be overexposed or have abnormal brightness after dehazing, and the second pixel refers to a pixel with a lower grayscale value. This lays the foundation for subsequent differentiated processing of different pixels (such as peak clipping and valley filling operations), and helps to adjust the brightness distribution and contrast of the image.

[0120] Step 206 : performing a peak clipping operation on the first pixel points and a valley filling operation on the second pixel points, and calculating a cumulative histogram of the second image.

[0121] In this embodiment, a peak clipping operation is performed on the determined first pixel to adjust excessively high brightness values ​​in the image, making the brightness distribution more uniform. As an example, a peak clipping threshold can be set, such that the first pixel with a brightness value exceeding 200 needs to be processed. For the first pixel with a brightness value greater than 200, its brightness value is reduced to an appropriate range. For example, a pixel with a brightness value of 220 is adjusted to 200, and a pixel with a brightness value of 230 is also adjusted to 200, and so on. In this way, excessively high brightness values ​​can be suppressed, avoiding excessive concentration of pixels at certain grayscale levels.

[0122] At the same time, a valley filling operation is performed on the second pixel. This is to increase the brightness of the lower brightness areas in the image and reduce the sparse brightness areas. A target range for valley filling is determined. For example, the brightness of the second pixel with a brightness value less than 50 is appropriately increased. For a second pixel with a brightness value of 30, its brightness value can be adjusted to 40; for a second pixel with a brightness value of 40, it can be adjusted to 45, and so on. In this way, the valley filling operation on the second pixel increases the brightness of the originally darker areas, improving the overall brightness distribution of the image.

[0123] After performing the peak clipping operation on the first pixel and the valley filling operation on the second pixel, the cumulative histogram of the second image is further calculated. The cumulative histogram can reflect the cumulative number of pixels from the lowest grayscale level to each grayscale level. For example, starting from grayscale level 0, the number of pixels at grayscale level 0 is used as the value of the cumulative histogram at grayscale level 0. For grayscale level 1, the sum of the number of pixels at grayscale levels 0 and 1 is used as the value of the cumulative histogram at grayscale level 1. Similarly, for each grayscale level i, the value of the cumulative histogram at grayscale level i is equal to the sum of the number of pixels from grayscale level 0 to grayscale level i.

[0124] As an example, after performing a peak clipping operation on the first pixel point of the second image and a valley filling operation on the second pixel point, the number of adjusted grayscale values ​​i can be calculated by the following formula.

[0125]

[0126] Where hist_adj[i] is the number of grayscale values ​​i after adjustment, hist[i] is the number of grayscale values ​​i in the original grayscale histogram, and hist_cutSum is the total number of pixels cut off during the peak clipping operation.

[0127] Among them, hist_cutSum is calculated by the following formula.

[0128]

[0129] For grayscale values ​​where i < hist_t0 and are located in the low-brightness region of the grayscale histogram, no adjustment is usually required, and the grayscale value remains unchanged. However, for grayscale values ​​where i ≥ hist_t0 and are located in the high-brightness region of the grayscale histogram, adjustment is required to fill the gaps left by peak clipping. The adjusted grayscale histogram is the original value plus the adjustment amount, which is the total number of clipped pixels, hist_cutSum, evenly distributed over the grayscale range from hist_t0 to 255.

[0130] Furthermore, after the peak clipping and valley filling operations, the number of pixels at grayscale level 0 is 100, and the number of pixels at grayscale level 1 is 150. Therefore, the value of the cumulative histogram at grayscale level 0 is 100, and the value at grayscale level 1 is 100 + 150 = 250. By calculating the cumulative histogram, we can clearly understand the cumulative distribution of pixels in the image at different grayscale levels, providing accurate data support for subsequent histogram equalization operations, which helps to further optimize the brightness and contrast of the image.

[0131] As an example, the cumulative histogram can be expressed by the following formula.

[0132]

[0133] Wherein, hist_sum[i] represents the cumulative number of all pixels with grayscale value from 0 to i, and hist_adj[j] is the number of pixels with grayscale value j after adjustment.

[0134] Step 207 : performing an equalization operation on the brightness component histogram according to the cumulative histogram to obtain a second brightness value corresponding to each pixel in the second image.

[0135] In this embodiment, balancing the luminance component histogram based on the calculated cumulative histogram can make the luminance distribution of the second image more uniform, thereby enhancing the image's contrast and visual quality. The balancing operation can be implemented using a mapping function that maps each grayscale level in the original luminance component histogram to a new grayscale level, thereby making the new luminance distribution more uniform.

[0136] Assume that in the cumulative histogram, the cumulative number of pixels at gray level 0 is 100, the cumulative number of pixels at gray level 1 is 250, and the total number of pixels is 10,000. Then, the new gray level to which gray level 0 is mapped can be calculated as follows: new gray level = 255 × (100 / 10,000) ≈ 3; the new gray level to which gray level 1 is mapped = 255 × (250 / 10,000) ≈ 6.

[0137] For each pixel in the image, the corresponding new grayscale level is found based on the position of its original grayscale level in the cumulative histogram. This new grayscale level is the second brightness value of the pixel after the histogram equalization operation. For example, for a pixel with an original grayscale level of 5, by searching the cumulative histogram and calculating the corresponding new grayscale level as 8 according to the above mapping relationship, 8 is then the second brightness value of the pixel.

[0138] As an example, the histogram equalization process can also be expressed by the following calculation formula.

[0139]

[0140] y_out = hist_dst[y_in]

[0141] Among them, y_in is the input Y component, and y_out is the Y component after histogram equalization.

[0142] By performing this operation on all pixels in the second image, the image, which originally had uneven brightness distribution, obtains a corresponding second brightness value for each pixel after histogram equalization. The second image thus constructed has been significantly improved in terms of brightness and contrast. The originally darker or low-contrast areas have become clearer, the details of the image are easier to observe, and the overall visual effect is significantly improved, which is more in line with the viewing habits of the human eye and improves the quality and usability of the image.

[0143] In the embodiment of the present application, the dehazed second image may also be oversaturated. When the saturation of the second image is greater than a set saturation, it indicates that the dehazed second image may have a color oversaturation problem. The set saturation refers to a threshold value for determining whether the dehazed second image is oversaturated. Based on this, the embodiment of the present application may also include a step of adjusting the saturation of the dehazed second image, specifically including the following steps 208 and 209.

[0144] Step 208 : In response to the saturation of the second image being greater than the set saturation, obtaining the first chromaticity component and the second chromaticity component of the first image, and the third chromaticity component and the fourth chromaticity component of the second image.

[0145] As an example, a saturation threshold can be set, for example, 0.75. When the saturation of the second image is detected to be greater than the set saturation threshold, steps 208 and 209 are performed to optimize the color representation of the second image. Both the first and second images can be converted into the YUV color space to obtain the relevant chrominance components.

[0146] At this time, corresponding chrominance components can be obtained from the first image and the second image respectively. In the YUV color space, the first chrominance component and the third chrominance component are U components, and the second chrominance component and the fourth chrominance component are V components.

[0147] For the first image, traverse each pixel therein and extract the U component of each pixel as the first chrominance component and the V component as the second chrominance component. For example, for the pixel P(m,n) located at the mth row and nth column in the first image, obtain the U component value U1(m,n) and the V component value V1(m,n) from its YUV color representation. By processing all pixels, the U component set and the V component set of the first image are obtained. These two sets can fully express the chrominance information of the first image.

[0148] Similarly, for the second image, each pixel is traversed, and the U component of each pixel is obtained as the third chromaticity component and the V component is obtained as the fourth chromaticity component. Assume that the pixel Q(i,j) in the second image is located at the i-th row and j-th column, and its U component value U2(i,j) and V component value V2(i,j) are obtained. In this way, the U component set and V component set of the second image are obtained to reflect the current chromaticity of the second image.

[0149] By acquiring the chroma components of the first and second images, a data foundation is provided for subsequent color processing based on the original and current chroma information of the images to reduce the saturation of the second image. This chroma component information helps analyze changes in image color, allowing targeted adjustments to be made, resulting in more natural and realistic colors in the processed image.

[0150] Step 209: Perform color processing on the second image according to the first chromaticity component, the second chromaticity component, the third chromaticity component, and the fourth chromaticity component to obtain a target third chromaticity component and a target fourth chromaticity component of the second image to reduce the saturation of the second image.

[0151] In this embodiment, after obtaining the first chromaticity component (U1), the second chromaticity component (V1) of the first image and the third chromaticity component (U2) and the fourth chromaticity component (V2) of the second image, color processing can be performed on the second image based on these chromaticity components to reduce the saturation of the second image and make the image color more natural.

[0152] Then, the third chroma component (U2) and the fourth chroma component (V2) of each pixel of the second image are adjusted to obtain a target third chroma component and a target fourth chroma component. For example, a weighted average calculation can be performed on the first chroma component and the third chroma component to obtain the target third chroma component, and then a weighted average calculation can be performed on the second chroma component and the fourth chroma component to obtain the target fourth chroma component.

[0153] For another example, the target third chromaticity component and the target fourth chromaticity component may also be calculated using the following formulas.

[0154] u_out=(u_ori*(32-w)+u_defog*w) / 32

[0155] v_out=(v_ori*(32-w)+u_defog*w) / 32

[0156] Among them, u_ori is the first chroma component, u_defog is the third chroma component, v_ori is the second chroma component, v_defog is the fourth chroma component, w is the adjustment factor, usually ranging from 0 to 32. The larger the value of w, the higher the adjustment intensity. u_out is the target third chroma component, and v_out is the target fourth chroma component.

[0157] By weightedly adjusting the chromaticity components of all pixels in the second image, the saturation of the entire second image is reduced. This adjustment reduces the image's colors from being overly vivid and oversaturated, resulting in a more natural color representation while preserving the image's primary color characteristics and detailed information. This results in a more realistic and comfortable color palette, aligning with the human eye's visual habits, improving overall image quality and viewing experience, and making it more suitable for a variety of applications.

[0158] Furthermore, before step 201, i.e., before determining the dark channel value of each pixel of the first image, the first image may be upsampled to convert the format of the first image from the first image format to the second image format. Upsampling refers to the process of increasing the resolution of the first image from a lower resolution to a higher resolution. Through upsampling, each pixel of the first image is enlarged, thereby increasing the size of the first image so that it contains more pixels.

[0159] Correspondingly, after step 203, i.e., after dehazing the first image based on the dark primary color value to obtain the second image, the second image may be downsampled to convert the second image format from the second image format to the first image format. Downsampling refers to the process of reducing the spatial resolution of the first image from a higher resolution to a lower resolution.

[0160] As can be seen from the above, the first image format is an image with a lower resolution, and the second image format is an image with a higher resolution. This image format conversion method can ensure the compatibility and efficiency of the image processing process while meeting the real-time requirements of vehicle-mounted video processing.

[0161] Images captured by this vehicle-mounted camera are typically in the YUV420 format, which consumes less bandwidth during storage and transmission but has lower resolution. Before dehazing, the image is first upsampled and converted to the YUV444 format. This improves the spatial resolution and ensures that each pixel has complete color information. This allows for more accurate reflection of image detail and color characteristics during dark channel calculation and dehazing. After dehazing, the resulting second image is downsampled and converted back to the YUV420 format for subsequent storage, transmission, and display, reducing data volume and improving processing efficiency.

[0162] Therefore, as an example, the first image format may be in yuv420 format, and the second image format may be in yuv444 format.

[0163] Among them, the luminance component Y in the YUV420 format is stored at full resolution, while the chrominance components U and V are stored at half resolution, that is, each 2x2 pixel block shares a U and V value. This format occupies less bandwidth during storage and transmission, but will cause the loss of image details. The YUV444 format stores complete Y, U, and V components for each pixel, which can retain all the detailed information of the image, but occupies a larger storage space. In the embodiment of the present application, by converting the format of upsampling and downsampling the image, the detail retention and calculation accuracy in the dehazing process are guaranteed, and the storage and transmission efficiency requirements of the on-board video system are met.

[0164] In addition, in the embodiment of the present application, the grayscale histogram can be obtained in the YUV444 format. However, since the dark channel value and the dark primary color value involve the three color channels of the pixel point, the YUV444 format needs to be further converted to the RGB444 format. After obtaining the histogram equalization processing of the second image and the color adjustment for oversaturation, it is necessary to first convert the RGB444 to YUV444 and then adjust it in the YUV444 format.

[0165] The following is a description of a specific embodiment. Figure 3 FIG. 1 is a flow chart of an image processing method provided in a specific embodiment of the present application. The image processing method includes steps 301-314.

[0166] Step 301: Input a first image (yuv420 format).

[0167] Step 302: Upsample the first image and convert the first image into yuv444 format.

[0168] Step 303: Obtain a grayscale histogram of the first image, and obtain a y component of the grayscale histogram.

[0169] Step 304: Perform color space conversion on the first image, converting the first image into RGB444 format.

[0170] Step 305: Obtain the dark channel value of the first image.

[0171] Step 306: Perform median filtering based on the dark channel value to obtain a dark original color value.

[0172] Step 307: Obtain the atmospheric light value of the first image.

[0173] Step 308: Determine defogging parameters for the first image.

[0174] Step 309: Obtain a second image based on the defogging parameters.

[0175] Step 310: Perform color space conversion on the second image and convert the second image into yuv444 format.

[0176] Step 311: Perform histogram equalization processing on the second image.

[0177] Step 312: Perform color adjustment on the second image by using the UV component of the brightness histogram to reduce oversaturation.

[0178] Step 313: downsample the second image and convert the second image into yuv420 format.

[0179] Step 314: Output the second image (in yuv420 format).

[0180] Figure 4 FIG. 1 is a structural diagram of an image processing system provided in an embodiment of the present application. Figure 4 As shown, the image processing system 400 may include a memory 401 and a processor 402. The memory 401 is configured to store instructions. The processor 402 is configured to call instructions from the memory 401 and implement any image processing method described in any embodiment of the present application when executing the instructions.

[0181] Figure 5 FIG. 5 is a schematic diagram of the structure of a vehicle 500 provided in an embodiment of the present application. Figure 5 As shown, the vehicle may include an on-board image acquisition device 501 and the above-mentioned image processing system 400, and the image processing system 400 is used to perform image processing on the first image acquired by the on-board image acquisition device 501 to obtain a second image.

[0182] The vehicle in the embodiments of the present application may be a fuel vehicle, a plug-in hybrid vehicle, a new energy vehicle, etc., and the present application does not make any specific limitations on this.

[0183] An embodiment of the present application further provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed by a processor, the processor is configured to execute the above-mentioned image processing method.

[0184] Since the instructions stored in the image processing system, the vehicle, and the computer-readable storage medium can execute the steps in any image method provided in the embodiments of the present application, the beneficial effects that can be achieved by any image processing method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0185] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0186] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0187] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0188] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0189] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0190] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0191] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated communication signals and carrier waves.

[0192] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0193] The above are merely preferred embodiments of the present application and do not constitute any form of limitation to the present application. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. An image processing method, characterized in that: include: For an input first image, determining a dark channel value of each pixel of the first image; Obtaining a dark primary color value of each pixel point according to a dark channel value of each pixel point in the first image; Dehazing is performed on the first image based on the dark primary color value to obtain a second image.

2. The method according to claim 1, characterized in that Determining the dark channel value of each pixel of the first image includes: For each pixel, obtaining a first grayscale value of a red channel, a second grayscale value of a blue channel, and a third grayscale value of a green channel of the pixel; The minimum grayscale value among the first grayscale value, the second grayscale value, and the third grayscale value is used as the dark channel value of the pixel point.

3. The method according to claim 1, characterized in that According to the dark channel value of each pixel in the first image, Obtaining the dark primary color value of each pixel, including: The dark channel values ​​corresponding to each pixel in the first image are filtered to obtain a dark original color value of each pixel.

4. The method according to claim 3, characterized in that Filtering the dark channel values ​​corresponding to the pixels in the first image to obtain the dark original color value of each pixel includes: Perform median filtering on the dark channel values ​​corresponding to each of the pixel points to obtain a dark primary color value of the first image, where the dark primary color value of the first image includes the dark primary color value of each of the pixel points.

5. The method according to claim 4, characterized in that The performing median filtering on the dark channel values ​​corresponding to the respective pixel points to obtain the dark original color value of the first image includes: Performing horizontal median filtering on the plurality of dark channel values ​​to obtain a first dark primary color value of the first image; Performing vertical median filtering on the plurality of dark channel values ​​to obtain a second dark primary color value of the first image; A dark primary color value of the first image is obtained according to the first dark primary color value and the second dark primary color value.

6. The method according to claim 5, characterized in that Obtaining the dark primary color value of the first image according to the first dark primary color value and the second dark primary color value includes: Obtaining a first dark primary color value and a second dark primary color value corresponding to each pixel point in the first image; The first dark primary color value and the second dark primary color value are weightedly fused according to the first weight and the second weight pre-set for the first dark primary color value and the second dark primary color value respectively to obtain the dark primary color value of each pixel point, and then obtain the dark primary color value of the first image.

7. The method according to claim 5, characterized in that The performing horizontal median filtering on the plurality of dark channel values ​​to obtain a first dark primary color value of the first image includes: Obtaining a preset first filtering window, where the first filtering window covers an odd number of pixels in a horizontal direction; On the first image, horizontally sliding the first filtering window on each row of pixels of the first image to traverse each pixel point of the first image; For each sliding operation, sorting the first pixels in the first filtering window according to the dark channel value of each of the first pixels in the first filtering window to obtain a first sorting result; According to the first sorting result, the dark channel value of the first pixel point at the middle position is obtained as the first dark original color value corresponding to the currently traversed pixel point.

8. The method according to claim 5, characterized in that The performing vertical median filtering on the plurality of dark channel values ​​to obtain a second dark primary color value of the first image includes: Obtaining a preset second filtering window, where the second filtering window covers an odd number of pixels in a vertical direction; On the first image, vertically sliding the second filtering window on each column of pixels of the first image to traverse each pixel point of the first image; For each sliding operation, sorting the second pixel points in the second filtering window according to the dark channel value of each second pixel point in the second filtering window to obtain a second sorting result; According to the second sorting result, the dark channel value of the second pixel point at the middle position is obtained as the second dark original color value corresponding to the currently traversed pixel point.

9. The method according to claim 1, characterized in that The performing defogging on the first image based on the dark primary color value to obtain a second image includes: Obtaining a grayscale histogram of the first image; Determining defogging parameters of the first image according to the grayscale histogram, the defogging parameters including a transmittance adjustment factor, a transmittance lower limit value, and a tolerance range; Defogging is performed on the first image based on the defogging parameters to obtain the second image.

10. The method according to claim 9, characterized in that Determining the defogging parameters of the first image according to the grayscale histogram includes: Determining a grayscale distribution range, a valid grayscale number, and a grayscale variance of the first image according to the grayscale histogram; The transmittance adjustment factor is determined according to the grayscale distribution range, the effective grayscale number, and the grayscale variance.

11. The method according to claim 10, characterized in that Determining the grayscale distribution range, effective grayscale number, and grayscale variance of the first image according to the grayscale histogram includes: Determining the grayscale distribution range according to the maximum grayscale value and the minimum grayscale value of each pixel of the first image in the grayscale histogram; Counting the number of grayscale levels in which the number of pixels in the grayscale histogram is greater than a set ratio of total pixels to obtain the effective grayscale number; The grayscale variance is calculated based on the number of pixels and grayscale value of each grayscale level in the grayscale histogram and the average brightness of the first image.

12. The method according to claim 9, characterized in that Determining the defogging parameters of the first image according to the grayscale histogram includes: determining an atmospheric light value and a grayscale variance of the first image according to the grayscale histogram; determining an atmospheric light adjustment factor of the first image according to the atmospheric light value; determining a grayscale variance adjustment factor of the first image according to the grayscale variance; A tolerance range of the first image is determined according to the set minimum value, the set maximum value, the atmospheric light adjustment factor, and the grayscale variance adjustment factor.

13. The method according to claim 12, characterized in that Determining the atmospheric light value of the first image according to the grayscale histogram includes: Obtaining the brightness value of each pixel in the grayscale histogram, sorting the pixels according to the brightness values, and determining a plurality of candidate pixels according to the sorting results; The atmospheric light value of the first image is determined according to the brightness values ​​corresponding to the candidate pixel points.

14. The method according to claim 13, characterized in that Determine multiple candidate pixels based on the sorting results: According to the sorting result, sequentially selecting pixel points of a set proportion as the candidate pixel points; The determining the atmospheric light value of the first image according to the brightness values ​​corresponding to the candidate pixel points includes: Calculate the average brightness value of the candidate pixels, and use the average brightness value as the atmospheric light value.

15. The method according to claim 12, characterized in that The performing defogging on the first image based on the defogging parameters to obtain the second image includes: determining a transmittance of each pixel in the first image according to the dark primary color value, the defogging parameter, and the atmospheric light value of the first image; determining a target grayscale value of each pixel in the first image after defogging according to a transmittance of each pixel in the first image and an atmospheric light value of the first image; The second image is obtained based on the target grayscale value of each pixel in the first image after defogging.

16. The method according to any one of claims 1 to 15, characterized in that Also includes: Acquire a luminance component histogram of the second image; Determine, based on the first brightness value of each pixel in the brightness component histogram, a first pixel whose grayscale value is greater than a set grayscale threshold and a second pixel whose grayscale value is less than or equal to the set grayscale threshold; performing a peak clipping operation on the first pixel points, and calculating a cumulative histogram of the second image; The brightness component histogram is equalized according to the cumulative histogram to obtain a second brightness value corresponding to each pixel.

17. The method according to claim 16, characterized in that The step of determining, based on the first brightness value of each pixel in the brightness component histogram, a first pixel whose grayscale value is greater than a set grayscale threshold and a second pixel whose grayscale value is less than or equal to the set grayscale threshold, comprises: Determining a first valid grayscale and a second valid grayscale of the luminance component histogram according to the luminance component histogram; Determining a set grayscale threshold according to the first valid grayscale and the second valid grayscale; Traverse each pixel point in the second image, and if the grayscale value of the pixel point is greater than the set grayscale threshold, use the pixel point as the first pixel point; if the grayscale value of the pixel point is less than or equal to the set grayscale threshold, use the pixel point as the second pixel point.

18. The method according to any one of claims 1 to 15, characterized in that Also includes: In response to the saturation of the second image being greater than a set saturation, acquiring a first chroma component and a second chroma component of the first image, and a third chroma component and a fourth chroma component of the second image; Color processing is performed on the second image according to the first chromaticity component, the second chromaticity component, the third chromaticity component, and the fourth chromaticity component to obtain a target third chromaticity component and a target fourth chromaticity component of the second image, so as to reduce the saturation of the second image.

19. The method according to claim 18, characterized in that The performing color processing on the second image according to the first chroma component, the second chroma component, the third chroma component, and the fourth chroma component to obtain a target third chroma component and a target fourth chroma component of the second image includes: A weighted average calculation is performed on the first chroma component and the third chroma component to obtain the target third chroma component, and a weighted average calculation is performed on the second chroma component and the fourth chroma component to obtain the target fourth chroma component.

20. The method according to any one of claims 1 to 15, characterized in that Before determining the dark channel value of each pixel of the first image, the method further includes: performing upsampling processing on the first image to convert the format of the first image from a first image format to a second image format; After performing defogging processing on the first image based on the dark primary color value to obtain the second image, the method further includes: Downsampling is performed on the second image to convert the format of the second image from the second image format to the first image format.

21. The method according to claim 20, characterized in that The first image format is yuv420 format, and the second image format is yuv444 format.

22. An image processing system, characterized in that: include: a memory configured to store instructions; as well as A processor is configured to call the instructions from the memory and implement the image processing method according to any one of claims 1 to 21 when executing the instructions.

23. A vehicle, characterized in that: It comprises a vehicle-mounted image acquisition device and the image processing system according to claim 22, wherein the image processing system is used to perform image processing on a first image acquired by the vehicle-mounted image acquisition device to obtain a second image.

24. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, which, when executed by a processor, enable the processor to be configured to execute the image processing method according to any one of claims 1 to 21.

25. A computer program product, characterized in that The method comprises a computer program or an instruction, which, when executed by a processor, implements the image processing method according to any one of claims 1 to 21.

Images