Image processing method, device and storage medium

Abstract

The present application provides an image processing method and device. The method comprises: dividing a first image into at least two sub-regions and estimating the depth of field of each sub-region; adjusting the contrast of each sub-region based on the depth of field of each sub-region to obtain an adjusted second image; and displaying the adjusted second image. The method of the embodiment of the present application can improve the depth of field of the image while improving the contrast, thereby enhancing the image display effect.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device and storage medium. Background Art

[0002] With the rapid development of technology and the television industry, intelligent technology has become an inevitable trend in the current television industry. More and more users choose to use their televisions to watch online videos. Currently, there is a rich variety of video content on the Internet, but the quality is uneven. For example, the image brightness and darkness are not good, which affects the viewing experience.

[0003] Currently, contrast adjustment schemes generally enhance contrast based on the average brightness of image partitions. For example, in areas with darker average brightness, the brightness is reduced, and in areas with brighter average brightness, the brightness is increased to increase the contrast between light and dark, thereby achieving local contrast enhancement. After enhancement, distant scenes may appear clearer than nearby scenes. However, based on visual experience, distant scenes in an image are generally blurrier than nearby scenes. Therefore, when the clarity of distant scenes is higher than that of nearby scenes, the human eye will experience an inverted perception, affecting the overall display effect of the picture. Summary of the Invention

[0004] The present application provides an image processing method, device, and storage medium to improve the contrast of an image, thereby improving the display effect of the image.

[0005] In a first aspect, the present application provides an image processing method, comprising:

[0006] Dividing the first image into at least two sub-regions, and estimating the depth of field of each of the sub-regions respectively;

[0007] adjusting the contrast of each of the sub-regions according to the depth of field of each of the sub-regions to obtain an adjusted second image;

[0008] The adjusted second image is displayed.

[0009] In a second aspect, the present application provides an image processing device, comprising:

[0010] a preprocessing module, configured to divide the first image into at least two sub-regions and estimate the depth of field of each of the sub-regions;

[0011] a processing module, configured to adjust the contrast of each of the sub-regions according to the depth of field of each of the sub-regions to obtain an adjusted second image;

[0012] A display module is configured to display the adjusted second image.

[0013] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method according to any one of the first aspects is implemented.

[0014] In a fourth aspect, an embodiment of the present application provides an electronic device, including:

[0015] processor; and

[0016] a memory for storing executable instructions of the processor;

[0017] The processor is configured to perform the method according to any one of the first aspects by executing the executable instructions.

[0018] The image processing method, device, and storage medium provided in the embodiments of the present application divide a first image into at least two sub-regions, and estimate the depth of field of each of the sub-regions respectively; adjust the contrast of each of the sub-regions according to the depth of field of each of the sub-regions to obtain an adjusted second image; and display the adjusted second image. In the above scheme, when enhancing the regional contrast of the image by judging the depth of field of each region, contrast adjustment processing is performed in combination with the depth of field of each region in the image, so that the processed image can improve the overall depth of field of the image while improving the contrast, avoiding the problems of inverted depth of field and reduced depth of field of the image due to contrast enhancement, maintaining the original depth of the picture, and achieving a better display effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0020] Figure 1 This is a flowchart of an embodiment of the image processing method provided by the present application;

[0021] Figure 2 This is a schematic diagram showing the principles of an embodiment of an image processing method provided by this application;

[0022] Figure 3 is a structural diagram of an embodiment of an image processing device provided by the present application;

[0023] Figure 4 It is a structural diagram of an electronic device embodiment provided by this application.

[0024] The above drawings illustrate specific embodiments of the present disclosure, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present disclosure in any way, but rather to illustrate the concepts of the present disclosure to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0025] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0026] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0027] First, let’s introduce the nouns involved in this application:

[0028] Depth of field (DOF) refers to the range of distances in front of and behind the subject that allows for a sharp image to be obtained at the leading edge of a camera lens or other imager. After focusing is achieved, the range of distances in front of and behind the focal point that allows for a sharp image is called the depth of field.

[0029] The method provided in the embodiment of the present application is applied in an image processing scenario, for example, a display device adjusts the contrast of an image before displaying the image to improve the image display effect.

[0030] The method provided in this application can be implemented by a display device, such as a processor, executing corresponding software code, or by the display device executing the corresponding software code while interacting with a server, for example, where the server controls the display device to implement the image processing method. The display device and the server can be connected via a network.

[0031] Among them, display devices include, for example, televisions, personal computers, tablet computers and other terminal devices.

[0032] According to the depth of field of the image, for distant scenes, when the contrast is the same, the distant scenes will appear blurrier than the nearby scenes. In related technologies, the contrast is enhanced according to the average brightness of the partitions in the image. For example, the brightness of the area with darker average brightness is increased, and the brightness of the area with brighter average brightness is reduced to achieve contrast enhancement. After the enhancement, the distant scene will appear clearer than the nearby scene, which affects the overall structure of the picture and reduces the overall depth of field of the picture.

[0033] The method of the embodiment of the present application, by judging the depth of field of each area, avoids the problems of image depth inversion and depth reduction caused by inconsistent contrast enhancement of different depth of field areas when enhancing the regional contrast of the image, and maintains the original depth of the picture while improving the image contrast.

[0034] The following specific embodiments are used to describe the technical solution of the present application in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0035] Figure 1 This is a flow chart of an embodiment of the image processing method provided by this application. Figure 1 、 Figure 2 As shown, the method provided in this embodiment includes:

[0036] Step 101: Divide a first image into at least two sub-regions, and estimate the depth of field of each sub-region respectively;

[0037] Specifically, a first image to be processed is obtained, and the first image is divided into regions to obtain at least two sub-regions, and the depth of field of each sub-region is estimated respectively. When enhancing the contrast of the image, the depth of field of each sub-region is considered and contrast processing is performed respectively.

[0038] In one embodiment, the depth of field estimation of a sub-region can be implemented using a machine learning model, such as a machine learning model trained using a depth of field estimation algorithm. The input of the machine learning model may include image information of the sub-region, and the output includes the depth of field of the sub-region. For example, the machine learning model estimates the depth of field by determining the clarity of different locations in the image, the size of objects, and the object's vanishing point.

[0039] Before training the machine learning model, a large amount of training data can be obtained to train the established machine learning model through the training data. The training data may include, for example: multiple images, the depth of field of each image, the depth of field of each sub-area in each image, etc.

[0040] Step 102: Adjust the contrast of each sub-region according to the depth of field of each sub-region to obtain an adjusted second image;

[0041] Step 103: Display the adjusted second image.

[0042] Specifically, when enhancing the contrast of an image, the contrast of each sub-region is adjusted in combination with the depth of field of each sub-region to obtain a second image after adjustment. For example, the area with a small depth of field is the foreground, and the area with a large depth of field is the background. According to the differences between the foreground and the background, the contrast is enhanced separately, and the adjustment degree of each sub-region is determined according to the size of the depth of field, ensuring that while enhancing the image contrast, the sense of depth of the image is improved.

[0043] In one embodiment, as Figure 2 shown, before displaying the image, the following processing can be performed:

[0044] Compare the depth of field of the second image and the first image;

[0045] If the depth of field of the second image is greater than that of the first image, display the second image;

[0046] Correspondingly, if the depth of field of the second image is less than or equal to that of the first image, the second image is used as the first image, and the steps of the foregoing method are re-executed.

[0047] Among them, after the contrast enhancement processing, the depth of field of the new second image can be estimated, for example, by using a pre-trained depth of field estimation model for estimation. Before processing the first image or before performing the depth of field comparison, the depth of field of the first image can also be estimated by using a pre-trained depth of field estimation model.

[0048] If the depth of field of the second image is greater than that of the first image, that is, the depth of field of the image after contrast adjustment is greater than that of the original image, it indicates that the processing effect is good, and while improving the contrast, the sense of depth of the image is improved.

[0049] If the depth of field of the second image is less than or equal to that of the first image, it indicates that the processing is not ideal, and the processing is re-performed, that is, the second image is used as the first image, and the steps of the foregoing method are re-executed.

[0050] For example, the depth of field of the second image after adjustment is L1, and the depth of field of the original first image is L2. If L1 > L2, it indicates that the processing effect is good, and while improving the contrast, the sense of depth of the image is improved.

[0051] If L1 < L2, it indicates that the depth of field of the processed image has decreased, and the solution of this method is re-executed, that is, the contrast is re-adjusted until L1 > L2.

[0052] The method of this embodiment divides a first image into at least two sub-regions and estimates the depth of field of each sub-region respectively; adjusts the contrast of each sub-region based on the depth of field of each sub-region to obtain an adjusted second image; and displays the adjusted second image. In the above scheme, when enhancing the regional contrast of the image by judging the depth of field of each region, contrast adjustment processing is performed in combination with the depth of field of each region in the image. This allows the processed image to have an improved overall depth of field while improving the contrast, avoiding the problems of inverted depth of field or reduced depth of field caused by contrast enhancement, maintaining the original depth of the picture, and achieving a better display effect.

[0053] Based on the above embodiment, optionally, step 102 can be specifically implemented in the following manner:

[0054] Determining a contrast adjustment coefficient corresponding to each sub-region according to the depth of field of each sub-region;

[0055] The contrast of each sub-region is adjusted according to the contrast adjustment coefficient corresponding to each sub-region to obtain a second image.

[0056] Specifically, after estimating the depth of field of each sub-region, the contrast adjustment coefficient corresponding to each sub-region is determined according to the depth of field of each sub-region. For example, the smaller the depth of field of the sub-region, the larger the contrast adjustment coefficient corresponding to the sub-region, and the larger the depth of field of the sub-region, the smaller the contrast adjustment coefficient corresponding to the sub-region.

[0057] In one embodiment, the depth of field of a sub-region is inversely proportional to the contrast adjustment coefficient corresponding to the sub-region.

[0058] In practical applications, for example, a corresponding relationship between the depth of field and the contrast adjustment coefficient can be established in advance. When processing an image, the contrast adjustment coefficient corresponding to the depth of field can be directly found through the corresponding relationship.

[0059] Then, the contrast of each sub-region is adjusted according to the determined contrast adjustment coefficient to obtain a second image.

[0060] The contrast of each sub-region is adjusted according to the determined contrast adjustment coefficient. The brightness of each sub-region can be adjusted to achieve contrast adjustment. The following method can be used:

[0061] The adjusted brightness of the sub-region is obtained according to the determined contrast adjustment coefficient and the minimum brightness of the sub-region.

[0062] For example, you can use the following method:

[0063] Record the initial minimum brightness Y of each sub-area min , maximum brightness Ymax , and the average brightness Y of the adjacent area neighbor1 、Y neighbor2 ;

[0064] For the key area, set the contrast adjustment coefficient k1>1, and the brightness output of the area after adjustment is as follows:

[0065] Y out =k1*(Y in -Y min )+Y min

[0066] For non-critical areas, set the contrast adjustment coefficient k2 < 1. The brightness output of the area after adjustment is as follows:

[0067] Y out =k2*(Y in -Y min )+Y min

[0068] Among them, Y in Indicates the input brightness, that is, the original brightness, Y out Indicates the output brightness, that is, the adjusted brightness.

[0069] Among them, the key areas can be areas where users pay close attention, such as face areas, subtitle areas, clothing areas, etc.

[0070] Furthermore, if the maximum brightness Y of each sub-area after adjustment is max If it is greater than the theoretical maximum brightness value of the corresponding scene (which can be calculated by the model), the output brightness is smoothly transitioned based on the theoretical maximum brightness value and 90% of the maximum brightness of the sub-area, and the output brightness corresponding to the input brightness of the sub-area is the maximum brightness Y max The output brightness of more than 90% of each pixel is smoothed and controlled between 90% of the maximum brightness and the theoretical maximum brightness value.

[0071] Furthermore, the brightness relationship between adjacent sub-regions can be compared. outneighbor1 、Y outneighbor2 The size between them needs to maintain a relative relationship consistent with the input brightness. If the brightness relationship changes, a smooth transition is performed based on the original brightness.

[0072] Furthermore, “determining the contrast adjustment coefficient corresponding to each sub-region according to the depth of field of each sub-region” can be specifically implemented as follows:

[0073] Arrange the depths of field of each of the sub-areas in ascending order to obtain the depths of field of each of the sub-areas in ascending order;

[0074] The contrast adjustment coefficients corresponding to the depths of field of the sub-regions arranged in ascending order are determined, and the contrast adjustment coefficients corresponding to the depths of field of the sub-regions are arranged in descending order.

[0075] Specifically, after obtaining the depth of field of each sub-area, the depth of field can be sorted from small to large. The smaller the depth of field, the near view, and the larger the depth of field, the far view. According to the difference between the near and far view, the contrast adjustment coefficient is determined respectively. The smaller the depth of field, the larger the contrast adjustment coefficient, and the larger the depth of field, the smaller the contrast adjustment coefficient. The sequence of depths of field arranged in ascending order, and the sequence formed by the corresponding contrast adjustment coefficients is a sequence arranged in descending order. Then, according to the contrast adjustment coefficient, processing is performed, for example, using a contrast enhancement algorithm, to enhance the contrast of the image.

[0076] For example, assume that the image is divided into 10 sub-areas a1-a10, the depth of field corresponding to each sub-area is L1-L10, the depth of field in ascending order is L2, L3, L1, L5, L7, L6, L4, L8, L9, L10, and the contrast adjustment coefficients in descending order are w2, w3, w1, w5, w7, w5, w6, w4, w8, w9, w10, where w1-w10 are the contrast adjustment coefficients corresponding to sub-areas a1-a10 respectively.

[0077] In the above specific implementation, when the contrast of an image is enhanced based on a contrast enhancement algorithm, corresponding processing can be performed in combination with the depth of field of the image, especially the depth of field of each sub-area, so that the processed image can improve the overall depth of field of the image while improving the contrast, avoiding the problems of inverted depth of field and reduced depth of field caused by contrast enhancement, maintaining the original depth of the picture, and achieving a better display effect.

[0078] Figure 3 This is a structural diagram of an embodiment of the image processing device provided by this application, such as Figure 3 As shown, the image processing device of this embodiment includes:

[0079] A pre-processing module 301 is configured to divide the first image into at least two sub-regions and estimate the depth of field of each sub-region respectively;

[0080] A processing module 302 is configured to adjust the contrast of each sub-region according to the depth of field of each sub-region to obtain an adjusted second image;

[0081] The display module 303 is configured to display the adjusted second image.

[0082] In a possible implementation, the processing module 302 is configured to:

[0083] determining a contrast adjustment coefficient corresponding to each of the sub-regions according to the depth of field of each of the sub-regions;

[0084] The contrast of each of the sub-regions is adjusted according to the contrast adjustment coefficient corresponding to each of the sub-regions to obtain the second image.

[0085] In a possible implementation, the depth of field of the sub-region is inversely proportional to the contrast adjustment coefficient corresponding to the sub-region.

[0086] In a possible implementation, the processing module 302 is configured to:

[0087] Arrange the depths of field of each of the sub-areas in ascending order to obtain the depths of field of each of the sub-areas in ascending order;

[0088] The contrast adjustment coefficients corresponding to the depths of field of the sub-regions arranged in ascending order are determined, and the contrast adjustment coefficients corresponding to the depths of field of the sub-regions are arranged in descending order.

[0089] In a possible implementation, the display module 303 is configured to:

[0090] If the depth of field of the second image is greater than the depth of field of the first image, displaying the second image;

[0091] Accordingly, the processing module 302 is configured to:

[0092] If the depth of field of the second image is less than or equal to the depth of field of the first image, the second image is used as the first image, and the step of dividing the first image into at least two sub-regions and estimating the depth of field of each sub-region is re-executed.

[0093] In a possible implementation, the processing module 302 is configured to:

[0094] The depth of field of each sub-region is estimated respectively using a pre-trained depth of field estimation model.

[0095] In a possible implementation, the processing module 302 is configured to:

[0096] estimating the depth of field of the second image using a pre-trained depth of field estimation model;

[0097] The depth of field of the first image is estimated using the depth of field estimation model.

[0098] The device of this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0099] In the device of this embodiment, the preprocessing module divides the first image into at least two sub-regions and estimates the depth of field of each of the sub-regions respectively; the processing module adjusts the contrast of each of the sub-regions according to the depth of field of each of the sub-regions to obtain an adjusted second image; and the display module displays the adjusted second image. In the above scheme, when the regional contrast of the image is enhanced by judging the depth of field of each region, the contrast adjustment processing is performed in combination with the depth of field of each region in the image, so that the processed image can improve the overall depth of field of the image while improving the contrast, avoiding the problems of inverted depth of field and reduced depth of field of the image due to contrast enhancement, maintaining the original depth of the picture, and achieving a better display effect.

[0100] Figure 4 This is a structural diagram of an embodiment of a display device provided in this application, such as Figure 4 As shown, the display device includes:

[0101] A processor 401, a display 403, and a memory 402 for storing executable instructions of the processor 401. The display 403 is used to display images.

[0102] The above components may communicate via one or more buses.

[0103] The processor 401 is configured to execute the following instructions:

[0104] Dividing the first image into at least two sub-regions, and estimating the depth of field of each of the sub-regions respectively;

[0105] adjusting the contrast of each of the sub-regions according to the depth of field of each of the sub-regions to obtain an adjusted second image;

[0106] The adjusted second image is displayed.

[0107] In a possible implementation, the processor 401 is configured to:

[0108] determining a contrast adjustment coefficient corresponding to each of the sub-regions according to the depth of field of each of the sub-regions;

[0109] The contrast of each of the sub-regions is adjusted according to the contrast adjustment coefficient corresponding to each of the sub-regions to obtain the second image.

[0110] In a possible implementation, the depth of field of the sub-region is inversely proportional to the contrast adjustment coefficient corresponding to the sub-region.

[0111] In a possible implementation, the processor 401 is configured to:

[0112] Arrange the depths of field of each of the sub-areas in ascending order to obtain the depths of field of each of the sub-areas in ascending order;

[0113] The contrast adjustment coefficients corresponding to the depths of field of the sub-regions arranged in ascending order are determined, and the contrast adjustment coefficients corresponding to the depths of field of the sub-regions are arranged in descending order.

[0114] In a possible implementation, the display 403 is configured to display the second image if the depth of field of the second image is greater than the depth of field of the first image;

[0115] In a possible implementation, the processor 401 is configured to:

[0116] If the depth of field of the second image is less than or equal to the depth of field of the first image, the second image is used as the first image, and the step of dividing the first image into at least two sub-regions and estimating the depth of field of each sub-region is re-executed.

[0117] In a possible implementation, the processor 401 is configured to:

[0118] The depth of field of each sub-region is estimated respectively using a pre-trained depth of field estimation model.

[0119] In a possible implementation, the processor 401 is configured to:

[0120] estimating the depth of field of the second image using a pre-trained depth of field estimation model;

[0121] The depth of field of the first image is estimated using the depth of field estimation model.

[0122] The device of this embodiment is used to execute the corresponding method in the aforementioned method embodiment. The specific implementation process can be found in the aforementioned method embodiment and will not be repeated here.

[0123] In one embodiment, an electronic device is further provided, including:

[0124] A processor, and a memory for storing executable instructions for the processor.

[0125] The above components may communicate via one or more buses.

[0126] Among them, the processor is configured to execute the corresponding method in the aforementioned method embodiment by executing the executable instructions. The specific implementation process can be found in the aforementioned method embodiment and will not be repeated here.

[0127] A computer-readable storage medium is also provided in an embodiment of the present application, on which a computer program is stored. When the computer program is executed by a processor, the corresponding method in the aforementioned method embodiment is implemented. The specific implementation process can be found in the aforementioned method embodiment. The implementation principle and technical effects are similar and will not be repeated here.

[0128] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0129] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: include: Dividing the first image into at least two sub-regions, and estimating the depth of field of each of the sub-regions respectively; adjusting the contrast of each of the sub-regions according to the depth of field of each of the sub-regions to obtain an adjusted second image; displaying the adjusted second image; The displaying of the adjusted second image includes: if the depth of field of the second image is greater than the depth of field of the first image, displaying the second image; Correspondingly, the method also includes: if the depth of field of the second image is less than or equal to the depth of field of the first image, then taking the second image as the first image, re-executing the step of dividing the first image into at least two sub-areas, and estimating the depth of field of each of the sub-areas respectively.

2. The method according to claim 1, characterized in that The step of adjusting the contrast of each of the sub-regions according to the depth of field of each of the sub-regions to obtain an adjusted second image includes: determining a contrast adjustment coefficient corresponding to each of the sub-regions according to the depth of field of each of the sub-regions; The contrast of each of the sub-regions is adjusted according to the contrast adjustment coefficient corresponding to each of the sub-regions to obtain the second image.

3. The method according to claim 2, characterized in that The depth of field of the sub-region is inversely proportional to the contrast adjustment coefficient corresponding to the sub-region.

4. The method according to claim 2 or 3, characterized in that Determining the contrast adjustment coefficient corresponding to each of the sub-areas according to the depth of field of each of the sub-areas includes: Arrange the depths of field of each of the sub-areas in ascending order to obtain the depths of field of each of the sub-areas in ascending order; The contrast adjustment coefficients corresponding to the depths of field of the sub-regions arranged in ascending order are determined, and the contrast adjustment coefficients corresponding to the depths of field of the sub-regions are arranged in descending order.

5. The method according to any one of claims 1 to 3, characterized in that The estimating the depth of field of each of the sub-areas separately includes: The depth of field of each sub-region is estimated respectively using a pre-trained depth of field estimation model.

6. The method according to any one of claims 1 to 3, characterized in that Before displaying the second image, the method further includes: estimating the depth of field of the second image using a pre-trained depth of field estimation model; The depth of field of the first image is estimated using the depth of field estimation model.

7. An image processing device, characterized in that: a preprocessing module, configured to divide the first image into at least two sub-regions and estimate the depth of field of each of the sub-regions; a processing module, configured to adjust the contrast of each of the sub-regions according to the depth of field of each of the sub-regions to obtain an adjusted second image; A display module, configured to display the adjusted second image; The display module is specifically configured to: display the second image if the depth of field of the second image is greater than the depth of field of the first image; Correspondingly, the processing module is specifically used to: if the depth of field of the second image is less than or equal to the depth of field of the first image, then use the second image as the first image, re-execute the step of dividing the first image into at least two sub-areas, and estimate the depth of field of each sub-area respectively.

8. A display device, characterized in that: include: Processor, display; as well as a memory for storing executable instructions of the processor; The display is used to display images; The processor is configured to perform the method according to any one of claims 1 to 6 by executing the executable instructions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images