Image processing apparatus, scene detection method, storage medium, and apparatus

Abstract

The invention discloses an image processing device, a scene detection method, a storage medium and equipment, and belongs to the technical field of image processing. The image processing device comprises a scene detection module; the scene detection module is configured to obtain pixel brightness information of an image, wherein the pixel brightness information comprises a brightness value of each pixel point on the image; acquiring a first scene detection result and environment brightness information reflecting an acquisition environment of the image based on the pixel brightness information, wherein the first scene detection result indicates that an acquisition scene of the image is at least one of a night scene or a backlight scene; the first scene detection result is calibrated according to a preset corresponding relation and the environment brightness information, a second scene detection result is obtained, and the preset corresponding relation is the corresponding relation between the environment brightness information and the confidence coefficient. The image processing apparatus improves scene detection capability and robustness.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing device, a scene detection method, a storage medium and a device. Background Art

[0002] In the field of image processing technology, for images captured in night scenes or backlit scenes, the whole or part of the image is often too bright or too dark, resulting in unclear image details. Therefore, the image can be detected as a night scene or backlit scene, and the image can be processed accordingly according to the detection result, so that a clear image can be captured in a night scene or backlit environment. Summary of the invention

[0003] The present application provides an image processing device, a scene detection method, a storage medium and a device for detecting whether the image acquisition scene is a night scene or a backlit scene. The technical solution is as follows:

[0004] In a first aspect, an image processing device is provided, the image processing device comprising: a scene detection module; the scene detection module is configured to:

[0005] Acquire pixel brightness information of an image, wherein the pixel brightness information includes a brightness value of each pixel point on the image;

[0006] Acquire a first scene detection result and ambient brightness information reflecting an environment in which the image was captured based on the pixel brightness information, wherein the first scene detection result indicates that the image was captured as at least one of a night scene or a backlit scene;

[0007] The first scene detection result is calibrated according to a preset corresponding relationship and the ambient brightness information to obtain a second scene detection result, wherein the preset corresponding relationship includes a corresponding relationship between the ambient brightness information and the confidence level.

[0008] In a possible implementation, the scene detection module is configured as follows:

[0009] In a case where the pixel brightness information satisfies a backlight condition, determining that the first scene detection result includes the backlight scene, wherein the backlight condition is determined based on pixel brightness distribution characteristics of a plurality of images collected under the backlight scene;

[0010] In a case where the pixel brightness information satisfies a night scene condition, it is determined that the first scene detection result includes the night scene, and the night scene condition is determined based on pixel brightness distribution characteristics of multiple images collected under the night scene scene.

[0011] In a possible implementation, the ambient brightness information includes at least one of an ambient brightness value, a dynamic range of the ambient brightness, or a bright-dark dividing value of the ambient brightness, the ambient brightness value indicates the brightness value of the image acquisition environment, the dynamic range of the ambient brightness indicates the brightness variation range of the image acquisition environment, and the bright-dark dividing value of the ambient brightness indicates the ratio of bright and dark areas of the image acquisition environment.

[0012] In a possible implementation, the ambient brightness information includes the ambient brightness value, and the scene detection module is configured to:

[0013] Acquire average brightness information of the image based on the brightness values ​​of the respective pixels;

[0014] The ambient brightness value is acquired according to the average brightness information, the exposure time and the aperture size of the image acquisition device.

[0015] In a possible implementation, the ambient brightness information includes a dynamic range of the ambient brightness, and the scene detection module is configured to:

[0016] Based on the brightness values ​​of the pixels, a first average brightness within a bright area and a second average brightness within a dark area are obtained, and a dynamic range of the ambient brightness is obtained according to the first average brightness and the second average brightness;

[0017] Among them, the bright area range refers to the pixel range on the image whose brightness value is greater than the first preset value, and the dark area range refers to the pixel range on the image whose brightness value is less than the second preset value. The bright area range or dark area range in the night scene is different from the bright area range or dark area range in the backlight scene.

[0018] In a possible implementation, the ambient brightness information includes a bright-dark boundary value of the ambient brightness, and the scene detection module is configured as follows:

[0019] Sorting the pixels according to brightness value;

[0020] In a case where the first scene detection result includes the night scene, using the brightness value of the pixel point arranged at the first preset position as the bright-dark boundary value of the ambient brightness in the night scene;

[0021] In a case where the first scene detection result includes the backlit scene, the brightness value of the pixel point arranged at the second preset position is used as the bright-dark boundary value of the ambient brightness in the backlit scene.

[0022] In a possible implementation, the preset correspondence relationship includes a first correspondence relationship in the night scene and a second correspondence relationship in the backlight scene, and the scene detection module is configured as follows:

[0023] When the first scene detection result includes the night scene, acquiring a first confidence level of the night scene based on the ambient brightness information and the first corresponding relationship;

[0024] When the first scene detection result includes the backlit scene, acquiring a second confidence level of the backlit scene based on the ambient brightness information and the second corresponding relationship;

[0025] The second scene detection result is determined by comparing the first confidence level with the second confidence level.

[0026] In a second aspect, a scene detection method is provided, the method comprising:

[0027] Acquire pixel brightness information of an image, wherein the pixel brightness information includes a brightness value of each pixel point on the image;

[0028] Acquire a first scene detection result and ambient brightness information reflecting an environment in which the image was captured based on the pixel brightness information, wherein the first scene detection result indicates that the image was captured as at least one of the night scene or the backlit scene;

[0029] The first scene detection result is calibrated according to a preset corresponding relationship and the ambient brightness information to obtain a second scene detection result, wherein the preset corresponding relationship includes a corresponding relationship between the ambient brightness information and the confidence level.

[0030] In a possible implementation, the acquiring a first scene detection result based on the pixel brightness information includes:

[0031] In a case where the pixel brightness information satisfies a backlight condition, determining that the first scene detection result includes the backlight scene, wherein the backlight condition is determined based on pixel brightness distribution characteristics of a plurality of images collected under the backlight scene;

[0032] In a case where the pixel brightness information satisfies a night scene condition, it is determined that the first scene detection result includes the night scene, and the night scene condition is determined based on pixel brightness distribution characteristics of multiple images collected under the night scene scene.

[0033] In a possible implementation, the ambient brightness information includes at least one of an ambient brightness value, a dynamic range of the ambient brightness, or a bright-dark dividing value of the ambient brightness, the ambient brightness value indicates the brightness value of the image acquisition environment, the dynamic range of the ambient brightness indicates the brightness variation range of the image acquisition environment, and the bright-dark dividing value of the ambient brightness indicates the ratio of bright and dark areas of the image acquisition environment.

[0034] In a possible implementation, the ambient brightness information includes the ambient brightness value, and obtaining the ambient brightness value based on the pixel brightness information includes:

[0035] Acquire average brightness information of the image based on the brightness values ​​of the respective pixels;

[0036] The ambient brightness value is acquired according to the average brightness information, the exposure time and the aperture size of the image acquisition device.

[0037] In a possible implementation, the ambient brightness information includes a dynamic range of the ambient brightness, and acquiring the dynamic range of the ambient brightness based on the pixel brightness information includes:

[0038] Based on the brightness values ​​of the pixels, a first average brightness within a bright area and a second average brightness within a dark area are obtained, and a dynamic range of the ambient brightness is obtained according to the first average brightness and the second average brightness;

[0039] Among them, the bright area range refers to the pixel range on the image whose brightness value is greater than the first preset value, and the dark area range refers to the pixel range on the image whose brightness value is less than the second preset value. The bright area range or dark area range in the night scene is different from the bright area range or dark area range in the backlight scene.

[0040] In a possible implementation, the ambient brightness information includes a bright-dark boundary value of the ambient brightness, and acquiring the bright-dark boundary value of the ambient brightness based on the pixel brightness information includes:

[0041] Sorting the pixels according to brightness value;

[0042] In a case where the first scene detection result includes the night scene, using the brightness value of the pixel point arranged at the first preset position as the bright-dark boundary value of the ambient brightness in the night scene;

[0043] In a case where the first scene detection result includes the backlit scene, the brightness value of the pixel point arranged at the second preset position is used as the bright-dark boundary value of the ambient brightness in the backlit scene.

[0044] In a possible implementation, the preset correspondence relationship includes a first correspondence relationship in the night scene and a second correspondence relationship in the backlight scene; and calibrating the first scene detection result according to the preset correspondence relationship and the ambient brightness information includes:

[0045] When the first scene detection result includes the night scene, acquiring a first confidence level of the night scene based on the ambient brightness information and the first corresponding relationship;

[0046] When the first scene detection result includes the backlit scene, acquiring a second confidence level of the backlit scene based on the ambient brightness information and the second corresponding relationship;

[0047] The second scene detection result is determined by comparing the first confidence level with the second confidence level.

[0048] According to a third aspect, a computer-readable storage medium is provided, wherein at least one program code is stored in the computer-readable storage medium, and the at least one program code is loaded and executed by a processor so that a computer implements any scene detection method described in the second aspect.

[0049] According to a fourth aspect, an electronic device is provided, comprising the image processing device according to any one of the first aspects.

[0050] In summary, the technical solution provided by this application can at least bring the following beneficial effects:

[0051] For the rough first scene detection result obtained through the preliminary judgment of pixel brightness information, deeper ambient brightness information is further constructed based on the pixel brightness information, so as to calibrate the first scene detection result according to the ambient brightness information and the preset correspondence, and obtain the second scene detection result. The second scene detection result can accurately indicate whether it is a night scene or a backlit scene, thereby solving the problem that it is difficult to distinguish between night scenes and backlit scenes only through pixel brightness information, so that the same scene detection module can detect whether the current image is a backlit scene or a night scene, thereby improving the scene detection capability and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 is a schematic diagram of an image processing device provided in an embodiment of the present application;

[0054] Figure 2 is a schematic diagram of interpolating a RAW image into an RGB image provided by an embodiment of the present application;

[0055] Figure 3 is a schematic diagram of a pixel structure of a RAW image provided in an embodiment of the present application;

[0056] Figure 4 This is a schematic diagram of dividing a weight area provided in an embodiment of the present application;

[0057] Figure 5 is a flow chart of a scene detection method provided by an embodiment of the present application;

[0058] Figure 6 It is a schematic diagram of a scene detection process provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0060] At present, many electronic devices with shooting functions can adjust shooting parameters during the shooting process to make the captured image better. When shooting in a night scene or backlit environment, the whole or part of the captured image is often too bright or too dark, resulting in unclear image details. In daily shooting, night scenes or backlit environments are common scenes in real life. Therefore, during the shooting process, night scenes or backlit environments can be detected, and the captured images can be processed accordingly according to the detection results, so that clear images can be captured in night scenes or backlit environments.

[0061] In the related art, the detection method for night scene or backlight includes: analyzing the brightness grayscale histogram of the image to be processed, and determining whether the image to be processed is taken in a night scene or backlight environment according to the brightness information of each pixel. However, since the night scene or backlight environment has similar brightness distribution characteristics under certain conditions, it is impossible to accurately distinguish whether it is a night scene environment or a backlight environment based only on the brightness information of the pixel.

[0062] The present application embodiment provides an image processing device, such as Figure 1As shown, the image processing device 00 includes a scene detection module 12. The scene detection module 12 is configured to: obtain pixel brightness information of the image, the pixel brightness information includes the brightness value of each pixel on the image; obtain a first scene detection result and ambient brightness information reflecting the image acquisition environment based on the pixel brightness information, the first scene detection result indicates that the image acquisition scene is at least one of a night scene or a backlit scene; calibrate the first scene detection result according to a preset corresponding relationship and the ambient brightness information to obtain a second scene detection result, the preset corresponding relationship includes a corresponding relationship between the ambient brightness information and the confidence level.

[0063] The preset correspondence may include a first correspondence in a night scene and a second correspondence in a backlit scene, wherein the first correspondence indicates the correspondence between the ambient brightness information and the confidence level in a night scene, and the second correspondence indicates the correspondence between the ambient brightness information and the confidence level in a backlit scene. Figure 1 As shown, the image processing device 00 further includes a storage module 11, and the storage module 11 is configured to store a preset corresponding relationship.

[0064] In an embodiment of the present application, the image may be acquired by an acquisition device, for example, by a camera. The embodiment of the present application does not limit the image format, and optionally, the image may be a RAW image, an RGB image, or a YUV image. Among them, the RAW image is the raw data of the light source signal captured by the image sensor and converted into a digital signal, without any processing, compression, or noise reduction, retaining all the details of the image. The RGB image represents the color by combining different brightness values ​​of the three primary colors of red (red, R), green (green, G), and blue (blue, B). The YUV image separates the brightness information (Y) from the color information (U and V), where Y represents brightness (Luminance) and U and V represent chrominance (Chrominance).

[0065] Usually, the image data of RAW images adopts Bayer arrangement, which includes but is not limited to the arrangement order of RGGB (red, green, green, blue), GRBG (green, red, blue, green), GBRG (green, blue, red, green), or BGGR (blue, green, green, red). That is, in a basic unit, the green photosensitive element occupies two positions, and the red and blue each occupy one position. The Bayer arrangement emphasizes the sensitivity of the human eye to the green band, so the number of green photosensitive elements in the Bayer arrangement is relatively large.

[0066] In the embodiment of the present application, the image to be subjected to scene detection is referred to as the image to be detected. In the case where the image to be detected is a RAW image, since the RAW image has a mosaic feeling of obvious red, green, and blue intervals, each pixel point only includes a part of the spectrum. Therefore, the embodiment of the present application converts the RAW image into an RGB image, and then obtains the pixel brightness information through the RGB image. Among them, for the scene where the camera captures the image to be detected, since the image captured by the camera is usually a RAW image, the image to be detected is usually a RAW image.

[0067] Optionally, the method of converting the RAW image into the RGB image is not limited in the present embodiment, for example, the method of filling the missing two color channel components by interpolation. Figure 2 The RAW image shown is a schematic diagram of the interpolation of the RGB image. Figure 2 The RAW image shown in (a) is split into Figure 2 The image data of the R channel, G channel and B channel shown in (b); each pixel corresponds to only one color channel component, and the two missing color channel components of each pixel are filled by interpolation to obtain Figure 2 The complete image data of the R channel, G channel and B channel shown in (c); finally, the complete Figure 2 The RGB image shown in (d) is used to obtain the R, G, and B values ​​of each pixel.

[0068] Take the RAW image with RGGB arrangement as an example, combined with Figure 3 The following example illustrates how to interpolate to fill in the missing two color channel components. Figure 3 For the RAW data shown in (a), since the pixel at the center position needs to restore the components of the two color channels R and B, the component of the R color channel at the center position is obtained by averaging the upper and lower R components at the center position, and the component of the B color channel at the center position is obtained by averaging the left and right B components at the center position.

[0069] for Figure 3 For the RAW data shown in (b), since the pixel point at the center position needs to restore the components of the two color channels G and R, the components of the G color channel at the center position are obtained by averaging the four G components at the top, bottom, left, and right of the center position, and the components of the R color channel at the center position are obtained by averaging the four R components diagonally at the center position.

[0070] For RGB images, the brightness value of each pixel can be calculated based on the pixel values ​​of the RGB channels according to the weights. For example, the formula for calculating the brightness Y based on the RGB values ​​of the pixel is shown in the following formula (1), where the R weight value is 0.299, the G weight value is 0.587, and the B weight value is 0.114. Formula (1) is determined based on the human eye's sensitivity to different colors. The human eye is most sensitive to green, followed by red, and least sensitive to blue. Therefore, when calculating brightness, green has the largest weight, followed by red, and blue has the smallest weight.

[0071] Y=((R*0.299)+(G*0.587)+(B*0.114))=

[0072] ((R*299)+(G*587)+(B*114)) / 1000 Formula (1)

[0073] Thus, the brightness value of each pixel in the image to be detected can be calculated through the above process, that is, the pixel brightness information can be obtained. After the pixel brightness information is obtained, it can be preliminarily determined whether the image to be detected is a night scene or a backlit scene according to the pixel brightness information. The result of the preliminary judgment is the first scene detection result.

[0074] In a possible implementation, the scene detection module 12 is configured to: obtain the first scene detection result based on the pixel brightness information in a manner including: determining that the first scene detection result includes a backlight scene when the pixel brightness information satisfies a backlight condition; and determining that the first scene detection result includes a night scene when the pixel brightness information satisfies a night scene condition. The backlight condition is determined based on the pixel brightness distribution characteristics of multiple images collected under the backlight scene; and the night scene condition is determined based on the pixel brightness distribution characteristics of multiple images collected under the night scene.

[0075] When capturing images in a backlit scene, the light comes from behind the subject, causing the metering system to mistakenly believe that the environment is too dark, thereby increasing the exposure, making the background too bright and the subject too dark. In this case, the pixel brightness distribution characteristics in the backlit scene include: pixels are mainly concentrated at extremely bright and extremely dark brightness levels, while there are fewer pixels at intermediate brightness levels.

[0076] Optionally, the backlight condition includes: the maximum brightness is greater than the first threshold, the minimum brightness is less than the second threshold, the number of pixels in the first dark area is greater than the third threshold, and the number of pixels in the second dark area is greater than the fourth threshold. Wherein, the maximum brightness is the maximum value among the brightness values of each pixel point, the minimum brightness is the minimum value among the brightness values of each pixel point, the number of pixels in the first dark area is the number of pixel points with a brightness value less than the first brightness threshold among each pixel point, and the number of pixels in the second dark area is the number of pixel points with a brightness value less than the second brightness threshold among each pixel point. The magnitudes of the first threshold, the second threshold, the third threshold, the fourth threshold, the first brightness threshold, and the second brightness threshold are not limited in the embodiments of the present application and can be flexibly adjusted according to experience or the accuracy of the detection results.

[0077] Exemplarily, when max_Y > th1, min_Y < th2, and at the same time cd_hist

[30] > th3, cd_hist

[40] > th4, it is preliminarily determined that it may be a backlight scene, and at this time the first scene detection result includes a backlight scene. Wherein, max_Y represents the maximum brightness, th1 represents the first threshold, for example, th1 = 250; n_Y represents the minimum brightness, th2 represents the second threshold, for example, th2 = 20; th3 represents the third threshold, for example, th3 = 0.3 * P, where P represents the number of pixel points in the image (i.e., the total number of pixel points included in the image); th4 represents the fourth threshold, for example, th4 = 0.4 * P.

[0078] In the embodiments of the present application, cd_hist represents the brightness cumulative histogram, and cd_hist is generated based on pixel brightness information. The cumulative brightness distribution histogram shows the number and distribution of pixel points with different brightness levels in the image in the form of a digital chart. Wherein, the horizontal axis of the cumulative brightness distribution histogram represents the brightness level. For example, from 0 (the darkest tone) on the left to 255 (the brightest tone) on the right, the brightness level is divided into 256 levels, and one brightness level corresponds to one brightness value. Thus, the horizontal axis of the cumulative brightness distribution histogram includes 256 equal-width intervals (bins), the length of each interval is 1, and the value of bins is consistent with the brightness level. The vertical axis of the cumulative brightness distribution histogram represents the cumulative number of pixels at each brightness level. For example, the cumulative number of pixels at brightness level x is the sum of the number of pixel points included from brightness level 1 to brightness level x, and x is a positive integer greater than 1.

[0079] For the above-mentioned brightness cumulative histogram cd_hist, cd_hist

[30] represents the vertical axis value at bins = 30 in the brightness cumulative histogram, that is, it represents the cumulative number of pixels corresponding to brightness level 30. At this time, the first brightness threshold is 30; cd_ist

[40] represents the vertical axis value at bins = 40 in the brightness cumulative histogram, that is, it represents the cumulative number of pixels corresponding to brightness level 40. At this time, the second brightness threshold is 30.

[0080] During the process of collecting images in a night scene, since the night lights are usually very bright, especially the neon lights, street lights or car lights in the city, etc. Therefore, when the sensors of image collection devices such as cameras and cameras face night lights, since the metering system determines the exposure parameters according to the average brightness of the entire picture, if the lights account for a large proportion in the picture, it may cause insufficient exposure of other parts outside the lights, while the light parts are overexposed. In this case, the pixel brightness distribution characteristics in the night scene include: pixels are mainly concentrated in the darker brightness levels, while there are fewer pixels in the middle brightness levels.

[0081] Optionally, the night scene conditions include: the minimum brightness is less than the fifth threshold, the number of pixels in the third dark area is greater than the sixth threshold, and the number of pixels in the fourth dark area is greater than the seventh threshold. Among them, the number of pixels in the third dark area is the number of pixel points with brightness values less than the third brightness threshold among each pixel point, and the number of pixels in the fourth dark area is the number of pixel points with brightness values less than the fourth brightness threshold among each pixel point. The magnitudes of the fifth threshold, the sixth threshold, the seventh threshold, the third brightness threshold, and the fourth brightness threshold are not limited in the embodiments of the present application and can be flexibly adjusted according to experience or the accuracy of the detection results.

[0082] Exemplarily, when min_Y < th5, and at the same time cd_hist

[20] > th6, cd_hist

[30] > th7, it is preliminarily determined that it may be a night scene, and at this time the first scene detection result includes the night scene. Among them, th5 represents the first threshold, for example, th5 = 100, th6 represents the sixth threshold, for example, th6 = 0.3*P, th7 represents the seventh threshold, for example, th7 = 0.6*P. cd_hist

[20] represents the vertical axis value at bins = 20 in the brightness cumulative histogram, that is, it represents the cumulative pixel number corresponding to the brightness level 20, and at this time the third brightness threshold is 20. The meanings of other parameters are as described above and will not be elaborated here.

[0083] In this embodiment, if the pixel brightness information satisfies both the backlight condition and the night scene condition at the same time, the first scene result includes the backlight scene and the night scene; if the pixel brightness information neither satisfies the backlight condition nor satisfies the night scene condition, the first scene result is empty, that is, it includes neither the backlight scene nor the night scene. Therefore, there are the following 4 possible situations for the first scene result: only including the backlight scene, only including the night scene, including both the backlight scene and the night scene, and including neither the backlight scene nor the night scene.

[0084] When the first scene detection result is empty, there is no need to perform secondary scene detection according to the first corresponding relationship or the second corresponding relationship, and the acquisition scene of the image to be detected is directly determined to be a general scene, which is neither a night scene nor a backlit scene. Below, based on the other three cases of the first scene detection result, the implementation methods of obtaining the ambient brightness information of the image are described by examples.

[0085] Optionally, the ambient brightness information includes at least one of an ambient brightness value, a dynamic range of ambient brightness, or a light-dark boundary value of ambient brightness. The ambient brightness value indicates the brightness value of the image acquisition environment; the dynamic range of ambient brightness indicates the brightness change range of the image acquisition environment, which is used to feedback the brightness change of the image acquisition environment; reflects the brightness-dark difference in the acquisition environment; the light-dark boundary value of ambient brightness indicates the ratio of light and dark areas in the image acquisition environment, which refers to the ratio of bright areas to dark areas, and is used to feedback whether there are more light areas or dark areas in the image acquisition environment.

[0086] In case 1, the first scene detection result only includes night scene.

[0087] In this case, if the ambient brightness information includes an ambient brightness value, the ambient brightness value is used to feedback the brightness of the image acquisition environment. The scene detection module 12 is configured to: obtain the average brightness information of the image based on the brightness value of each pixel; and obtain the ambient brightness value based on the average brightness information, the exposure time of the image acquisition device, and the aperture size.

[0088] Optionally, the method for obtaining the average brightness information is not limited in the embodiment of the present application, and the average brightness value of all pixels of the image can be directly used as the average brightness information; or, a weight value of each pixel can be set, and the weighted average brightness value of all pixels of the image can be used as the average brightness information. The weight of the pixel can be obtained by dividing the area.

[0089] For example, the image is divided into multiple regions, each of which can be flexibly assigned a different weight value; then, according to the brightness value of each pixel and the weight value of the region where each pixel is located, the average brightness information of the image is obtained by weighted average calculation. The embodiment of the present application does not limit the region division method of the first image, and it can be customized according to shooting habits. For example, the first image can be divided into Figure 4 In the weight areas shown, the weight value of the most central area 1 is the largest, which is 1; the weight value of the sub-central area 2 is 0.8; and the weight value of the most peripheral area 3 is the smallest, which is 0.6.

[0090] In the application embodiment, after determining the average brightness information, the ambient brightness value can be calculated by the following formula (2). Wherein, BV represents the ambient brightness value, curry represents the average brightness information; K is an arbitrary constant, generally determined based on the exposure time, for example, K is 12.5; N is experimental data, which will be different for different camera modules, for example, N is 0.3. The working principle of the camera module is to focus the external light on the sensor through the lens, and the sensor converts the light signal into an electrical signal. After being processed by the image processing chip, it is output in a standard image format. The aperture size affects the amount of light entering the camera module, depth of field, image noise, focus speed and shutter speed.

[0091]

[0092] If the ambient brightness information includes the dynamic range of the ambient brightness. The scene detection module 12 is configured to: obtain a first average brightness within a bright area and a second average brightness within a dark area based on the brightness values ​​of each pixel point, and obtain the dynamic range of the ambient brightness according to the first average brightness and the second average brightness. Among them, the bright area range refers to a pixel range on an image whose brightness value is greater than a first preset value, and the dark area range refers to a pixel range on the image whose brightness value is less than a second preset value. In an embodiment of the present application, the bright area range or dark area range in a night scene is different from the bright area range or dark area range in a backlight scene, and is determined according to different brightness characteristics in night scene and backlight scene to improve the accuracy of obtaining the dynamic range of ambient brightness in different scenes.

[0093] For the first scene detection result including a night scene, the scene detection module 12 is configured to: obtain the first average brightness of the first bright area pixel and the second average brightness of the first dark area pixel based on the brightness value of each pixel point, and obtain the dynamic range of the ambient brightness under the night scene according to the first average brightness and the second average brightness. Among them, the brightness value of the first bright area pixel is greater than the brightness value of the first percentage of the pixel points in each pixel point, that is, the first preset value is the brightness value of the first percentage of the pixel points in each pixel point; the brightness value of the first dark area pixel is less than the brightness value of the second percentage of the pixel points in each pixel point, that is, the second preset value is the brightness value of the second percentage of the pixel points in each pixel point. Optionally, the values ​​of the first percentage and the second percentage are not limited in the embodiment of the present application, and can be flexibly adjusted according to experience or the accuracy of the detection result.

[0094] Exemplarily, given the percentage of bright pixels (bright_percent=5%) and the percentage of dark pixels (dark_percent=30%), the first percentage is calculated to be 95% and the second percentage is 30%. Taking the brightness values ​​of each pixel of the image arranged in order from small to large as an example, the first bright area pixel includes the pixel whose brightness value is after 95%*P, and the first dark area pixel includes the pixel whose brightness value is before 30%*P. The average brightness value of each pixel in the first bright area pixel is taken as the first average brightness, and the average brightness value of each pixel in the first dark area pixel is taken as the second average brightness.

[0095] After the first average brightness and the second average brightness are calculated, the dynamic range of the ambient brightness in the night scene can be calculated by the following formula (3): wherein NS_EVD represents the dynamic range of the ambient brightness in the night scene, NS_brightY represents the first average brightness, and NS_darkY represents the second average brightness.

[0096] NS_EVD=NS_brightY-NS_darkY Formula (3)

[0097] For the first scene detection result including a night scene, if the ambient brightness information includes a bright-dark boundary value of the ambient brightness, the scene detection module 12 is configured to: sort each pixel point according to the brightness value; and use the brightness value of the pixel point arranged at the first preset position as the bright-dark boundary value of the ambient brightness in the night scene.

[0098] Among them, each pixel point can be arranged in ascending order or in descending order according to the brightness value. The implementation of this application is illustrated by taking ascending order as an example. Optionally, the serial number of the first preset position is N, and the ratio of N to the number of pixels P in the image is a first value. The first value can be flexibly adjusted based on experience or the accuracy of the detection result. For example, the first value is 60%. Exemplarily, the brightness level corresponding to 60% of the vertical axis in the above-mentioned cumulative brightness histogram cd_hist can be used as the bright and dark dividing value of the ambient brightness in the night scene. That is, cd_hist[NS_CDF] = 60%, and NS_CDF represents the bright and dark dividing value of the ambient brightness in the night scene.

[0099] Case 2: The first scene detection result only includes the backlit scene.

[0100] In the second case, if the ambient brightness information includes an ambient brightness value, the method for obtaining the ambient brightness value is the same as the method for obtaining the ambient brightness value in the first case, and will not be described in detail here.

[0101] For the first scene detection result including a backlit scene, if the ambient brightness information includes a dynamic range of the ambient brightness, the scene detection module 12 is configured to: obtain a third average brightness of the second bright area pixel and a fourth average brightness of the third dark area pixel based on the brightness value of each pixel point, and obtain the dynamic range of the ambient brightness in the backlit scene according to the third average brightness and the fourth average brightness.

[0102] Among them, the brightness value of the second bright area pixel is greater than the brightness value of the third percentage of the pixels among the pixels, that is, the first preset value is the brightness value of the third percentage of the pixels among the pixels; the brightness value of the second dark area pixel is less than the brightness value of the fourth percentage of the pixels among the pixels, that is, the second preset value is the brightness value of the fourth percentage of the pixels among the pixels. Optionally, the values ​​of the third percentage and the fourth percentage are not limited in the embodiment of the present application, and can be flexibly adjusted according to experience or the accuracy of the detection results.

[0103] Exemplarily, given the percentage of bright pixels (bright_percent=12%) and the percentage of dark pixels (dark_percent=12%), the third percentage is calculated to be 88% and the fourth percentage is 12%. Taking the brightness values ​​of each pixel of the image arranged in order from small to large as an example, the third bright area pixel includes the pixel whose brightness value is after 88%*P, and the fourth dark area pixel includes the pixel whose brightness value is before 12%*P. The average brightness of each pixel in the second bright area pixel is taken as the third average brightness, and the average brightness of each pixel in the second dark area pixel is taken as the fourth average brightness.

[0104] After the third average brightness and the fourth average brightness are calculated, the dynamic range of the ambient brightness in the backlight scene can be calculated by the following formula (4): wherein BLS_EVD represents the dynamic range of the ambient brightness in the backlight scene, BLS_brightY represents the third average brightness, and BLS_darkY represents the fourth average brightness.

[0105] BLS_EVD=BLS_brightY-BLS_darkY formula (4)

[0106] For the first scene detection result including a backlit scene, if the ambient brightness information includes a bright-dark boundary value of the ambient brightness, the scene detection module 12 is configured to: sort each pixel point according to the brightness value; and use the brightness value of the pixel point at the second preset position as the bright-dark boundary value of the ambient brightness in the backlit scene.

[0107] Among them, each pixel point can be arranged in ascending order or in descending order according to the brightness value. The embodiment of the present application takes ascending order as an example for illustration. Optionally, the serial number of the second preset position is M, and the ratio of M to the number of pixels P in the image is a second value. The second value can be flexibly adjusted based on experience or the accuracy of the detection result. For example, the second value is 30%. Exemplarily, the brightness level corresponding to 30% of the vertical axis in the above-mentioned cumulative brightness histogram cd_hist can be used as the bright and dark dividing value of the ambient brightness in the backlight scene. That is, cd_hist[BLS_CDF] = 30%, and BLS_CDF represents the bright and dark dividing value of the ambient brightness in the backlight scene.

[0108] Case three: the first scene detection result includes a backlit scene and a night scene.

[0109] In the third case, if the ambient brightness information includes an ambient brightness value, the method for obtaining the ambient brightness value is the same as that in the first case, and will not be described in detail here.

[0110] If the ambient brightness information includes the dynamic range of the ambient brightness, the dynamic range of the ambient brightness in the night scene can be obtained by the method in case 1, and the dynamic range of the ambient brightness in the backlight scene can be obtained by the method in case 2. If the ambient brightness information includes the bright and dark boundary value of the ambient brightness, the bright and dark boundary value of the ambient brightness in the night scene can be obtained by the method in case 1, and the bright and dark boundary value of the ambient brightness in the backlight scene can be obtained by the method in case 2.

[0111] According to the above three situations, for night scenes and backlit scenes, different methods are used to calculate the dynamic range of ambient brightness and the bright-dark boundary value, which makes the method more robust and widely applicable.

[0112] After obtaining the ambient brightness information of the image, the first scene detection result can be calibrated according to the ambient brightness information of the image and at least one of the first corresponding relationship or the second corresponding relationship stored in the storage module 11, that is, a secondary judgment is made on whether it is a night scene or a backlight scene to obtain the final second scene detection result.

[0113] In a possible implementation, the scene detection module 12 is configured as follows: when the first scene detection result includes a night scene, obtaining a first confidence level of the night scene based on the ambient brightness information and a first correspondence relationship; when the first scene detection result does not include a night scene, determining that the first confidence level of the night scene is 0; when the first scene detection result includes a backlight scene, obtaining a second confidence level of the backlight scene based on the ambient brightness information and a second correspondence relationship; when the first scene detection result does not include a backlight scene, determining that the second confidence level of the backlight scene is 0; and then determining the second scene detection result by comparing the first confidence level with the second confidence level.

[0114] In the embodiment of the present application, the first corresponding relationship and the second corresponding relationship can be obtained by experimental statistics. For example, multiple night scene images collected in a night scene and multiple backlight images collected in a backlight scene are collected; for each of the multiple night scene images and the multiple backlight images, the pixel brightness information of each image is obtained, and the first scene detection result and the ambient brightness information of each image are obtained based on the pixel brightness information; according to the ambient brightness information of each image and the actual collection scene of each image, the first corresponding relationship and the second corresponding relationship are obtained by statistics.

[0115] Taking the example that the ambient brightness information includes the ambient brightness value, the dynamic range of the ambient brightness, and the bright-dark boundary value of the ambient brightness, the first correspondence includes the sub-correspondence between the ambient brightness value and the confidence, the sub-correspondence between the dynamic range of the ambient brightness in the night scene and the confidence, and the sub-correspondence between the bright-dark boundary value of the ambient brightness in the night scene and the confidence. Similarly, the second correspondence includes the sub-correspondence between the ambient brightness value and the confidence, the sub-correspondence between the dynamic range of the ambient brightness in the backlight scene and the confidence, and the sub-correspondence between the bright-dark boundary value of the ambient brightness in the backlight scene and the confidence.

[0116] Thus, the first confidence level includes a sub-confidence level corresponding to the ambient brightness value, a sub-confidence level corresponding to the dynamic range of ambient brightness in the night scene, and a sub-confidence level corresponding to the bright-dark boundary value of ambient brightness in the night scene. The second confidence level includes a sub-confidence level corresponding to the ambient brightness value, a sub-confidence level corresponding to the dynamic range of ambient brightness in the backlit scene, and a sub-confidence level corresponding to the bright-dark boundary value of ambient brightness in the backlit scene.

[0117] Optionally, the first correspondence may be a night scene confidence probability curve. For example, the night scene confidence probability curve may include a sub-correspondence between NS_BV Prob and BV as shown in the following formula (5), a sub-correspondence between NS_EVD Prob and BV as shown in formula (6), and a sub-correspondence between NS_CDF Prob and BV as shown in formula (7). Among them, NS_BV Prob represents the sub-confidence corresponding to the ambient brightness value BV in the night scene, NS_EVD Prob represents the sub-confidence corresponding to the dynamic range EVD of the ambient brightness in the night scene, and NS_CDF Prob represents the sub-confidence corresponding to the bright and dark boundary value of the ambient brightness in the night scene.

[0118]

[0119] Optionally, the second correspondence may be a backlight confidence probability curve. The backlight confidence probability curve may include a sub-correspondence between BLS_BV Prob and BV as shown in the following formula (8), a sub-correspondence between BLS_EVD Prob and BV as shown in formula (9), and a sub-correspondence between BLS_CDF Prob and BV as shown in formula (10). Among them, BLS_BV Prob represents the sub-confidence corresponding to the ambient brightness value BV in the backlight scene, BLS_EVD Prob represents the sub-confidence corresponding to the dynamic range EVD of the ambient brightness in the backlight scene, and BLS_CDF Prob represents the sub-confidence corresponding to the bright and dark boundary value of the ambient brightness in the backlight scene.

[0120]

[0121]

[0122] Thus, the BV, EVD and CDF obtained in the night scene are substituted into formula (5)-formula (7), and the first confidence can be calculated, that is, the first confidence is the maximum value or average value among NS_BV Prob (BV), NS_EVD Prob (EVD) and NS_CDF Prob (CDF). The BV, EVD and CDF obtained in the backlight scene are substituted into formula (8)-formula (10), and the second confidence can be calculated, that is, the second confidence is the maximum value or average value among BLS_BV Prob (BV), BLS_EVD Prob (EVD) and BLS_CDFProb (CDF).

[0123] According to the first confidence level and the second confidence level obtained above, the second scene detection result is determined by comparing the two confidence levels, and the confidence level ranges from 0 to 1. Optionally, when the first confidence level is greater than the second confidence level, the second scene detection result is a night scene; when the first confidence level is less than the second confidence level, the second scene detection result is a backlit scene.

[0124] In the embodiment of the present application, the second scene detection result may further include a corresponding confidence level, and the confidence level indicates the accuracy of the second scene detection result. That is, when the second scene detection result is a night scene, the second scene detection result also includes the first confidence level, and when the second scene detection result is a backlit scene, the second scene detection result also includes the second confidence level.

[0125] After obtaining the second scene detection result, it can be determined whether the image acquisition scene is a night scene or a backlit scene, and then the image is subjected to scene compensation based on the night scene or the backlit scene, for example, the image brightness is adjusted accordingly based on the scene to improve the clarity of the image. Optionally, according to the confidence level in the second scene detection result, the image can be subjected to scene compensation to different degrees, for example, the higher the confidence level, the greater the degree of scene compensation, and the lower the confidence level, the smaller the degree of scene compensation. The embodiments of the present application do not limit the scene compensation method.

[0126] To summarize, the image processing device provided in the embodiment of the present application, for a rough first scene detection result obtained by preliminary judgment through pixel brightness information, further constructs deeper ambient brightness information based on the pixel brightness information, so as to calibrate the first scene detection result according to the ambient brightness information and a preset correspondence, and obtain a second scene detection result. The second scene detection result can accurately indicate whether it is a night scene or a backlit scene, thereby solving the problem that it is difficult to distinguish between night scene scenes and backlit scenes only through pixel brightness information, so that the same scene detection module can detect whether the current image is a backlit scene or a night scene, thereby improving the scene detection capability and robustness.

[0127] The present application also provides a scene detection method, see Figure 5 , the scene detection method includes but is not limited to the following steps 501-503.

[0128] Step 501: Obtain pixel brightness information of an image, where the pixel brightness information includes the brightness value of each pixel on the image.

[0129] Step 502, based on the pixel brightness information, a first scene detection result and ambient brightness information reflecting the image acquisition environment are obtained, wherein the first scene detection result indicates that the image acquisition scene is at least one of a night scene and a backlit scene.

[0130] In a possible implementation, obtaining a first scene detection result based on pixel brightness information includes: when the pixel brightness information satisfies a backlight condition, determining that the first scene detection result includes a backlight scene, the backlight condition being determined based on pixel brightness distribution characteristics of multiple images collected under the backlight scene; when the pixel brightness information satisfies a night scene condition, determining that the first scene detection result includes a night scene, the night scene condition being determined based on pixel brightness distribution characteristics of multiple images collected under the night scene scene.

[0131] In a possible implementation, the backlight condition includes: the maximum brightness is greater than a first threshold, the minimum brightness is less than a second threshold, the number of pixels in the first dark area is greater than a third threshold, and the number of pixels in the second dark area is greater than a fourth threshold; the night scene condition includes: the minimum brightness is less than a fifth threshold, the number of pixels in the third dark area is greater than a sixth threshold, and the number of pixels in the fourth dark area is greater than a seventh threshold.

[0132] Among them, the maximum brightness value is the maximum value of the brightness values ​​of each pixel point, the minimum brightness value is the minimum value of the brightness values ​​of each pixel point, the number of pixels in the first dark area is the number of pixels in each pixel point whose brightness value is less than the first brightness threshold, the number of pixels in the second dark area is the number of pixels in each pixel point whose brightness value is less than the second brightness threshold, the number of pixels in the third dark area is the number of pixels in each pixel point whose brightness value is less than the third brightness threshold, and the number of pixels in the fourth dark area is the number of pixels in each pixel point whose brightness value is less than the fourth brightness threshold.

[0133] In a possible implementation, the ambient brightness information includes at least one of an ambient brightness value, a dynamic range of the ambient brightness, or a bright-dark dividing value of the ambient brightness, the ambient brightness value indicates the brightness value of the image acquisition environment, the dynamic range of the ambient brightness indicates the brightness variation range of the image acquisition environment, and the bright-dark dividing value of the ambient brightness indicates the ratio of bright and dark areas of the image acquisition environment.

[0134] When the ambient brightness information includes an ambient brightness value, obtaining the ambient brightness value based on the pixel brightness information includes: obtaining average brightness information of the image based on the brightness value of each pixel; and obtaining the ambient brightness value according to the average brightness information, exposure time of the image acquisition device, and aperture size.

[0135] In the case where the ambient brightness information includes the dynamic range of the ambient brightness, the dynamic range of the ambient brightness is obtained based on the pixel brightness information, including: obtaining a first average brightness in a bright area and a second average brightness in a dark area based on the brightness values ​​of each pixel, and obtaining the dynamic range of the ambient brightness according to the first average brightness and the second average brightness; wherein the bright area range refers to a pixel range on an image whose brightness value is greater than a first preset value, and the dark area range refers to a pixel range on an image whose brightness value is less than a second preset value, and the bright area range or dark area range in a night scene is different from the bright area range or dark area range in a backlight scene.

[0136] Exemplarily, when the first scene detection result includes a night scene, a first average brightness of the first bright area pixels and a second average brightness of the first dark area pixels are obtained based on the brightness values ​​of each pixel point, and the dynamic range of the ambient brightness in the night scene is obtained according to the first average brightness and the second average brightness; when the first scene detection result includes a backlight scene, a third average brightness of the second bright area pixels and a fourth average brightness of the third dark area pixels are obtained based on the brightness values ​​of each pixel point, and the dynamic range of the ambient brightness in the backlight scene is obtained according to the third average brightness and the fourth average brightness.

[0137] Among them, the brightness value of the first bright area pixel is greater than the brightness value of the first percentage of pixel points among all pixel points, the brightness value of the first dark area pixel is less than the brightness value of the second percentage of pixel points among all pixel points, the brightness value of the second bright area pixel is greater than the brightness value of the third percentage of pixel points among all pixel points, and the brightness value of the second dark area pixel is less than the brightness value of the fourth percentage of pixel points among all pixel points.

[0138] In a case where the ambient brightness information includes a bright-dark boundary value of the ambient brightness, the bright-dark boundary value of the ambient brightness is obtained based on the pixel brightness information, including: sorting each pixel point according to the brightness value; in a case where the first scene detection result includes a night scene, using the brightness value of the pixel point arranged at a first preset position as the bright-dark boundary value of the ambient brightness in the night scene; in a case where the first scene detection result includes a backlight scene, using the brightness value of the pixel point arranged at a second preset position as the bright-dark boundary value of the ambient brightness in the backlight scene.

[0139] Exemplarily, each pixel is sorted according to the brightness value; when the first scene detection result includes a night scene, the brightness value of the pixel ranked at the Nth position is used as the bright and dark boundary value of the ambient brightness in the night scene, and the ratio of N to the number of pixels in the image is a first value; when the first scene detection result includes a backlight scene, the brightness value of the pixel ranked at the Mth position is used as the bright and dark boundary value of the ambient brightness in the backlight scene, and the ratio of M to the number of pixels in the image is a second value.

[0140] Step 503, calibrating the first scene detection result according to a preset correspondence relationship and the ambient brightness information to obtain a second scene detection result, wherein the preset correspondence relationship includes a correspondence relationship between the ambient brightness information and the confidence level.

[0141] In an embodiment of the present application, the preset correspondence relationship includes a first correspondence relationship in a night scene and a second correspondence relationship in a backlight scene. The first correspondence relationship indicates the correspondence between the ambient brightness information and the confidence in the night scene, and the second correspondence relationship indicates the correspondence between the ambient brightness information and the confidence in the backlight scene.

[0142] In a possible implementation, the first scene detection result is calibrated according to a preset correspondence and ambient brightness information, including: when the first scene detection result includes a night scene, obtaining a first confidence level of the night scene based on the ambient brightness information and the first correspondence; when the first scene detection result includes a backlight scene, obtaining a second confidence level of the backlight scene based on the ambient brightness information and the second correspondence; and determining the second scene detection result by comparing the first confidence level with the second confidence level.

[0143] Taking the ambient brightness information including the ambient brightness value, the dynamic range of the ambient brightness and the bright / dark boundary value of the ambient brightness as an example, the first correspondence includes the sub-correspondence between the ambient brightness value and the confidence, the sub-correspondence between the dynamic range of the ambient brightness in the night scene and the confidence, and the sub-correspondence between the bright / dark boundary value of the ambient brightness in the night scene and the confidence. Then the first confidence includes the sub-confidence corresponding to the ambient brightness value, the sub-confidence corresponding to the dynamic range of the ambient brightness in the night scene and the sub-confidence corresponding to the bright / dark boundary value of the ambient brightness in the night scene.

[0144] The second correspondence includes a sub-correspondence between the ambient brightness value and the confidence, a sub-correspondence between the dynamic range of the ambient brightness in the backlight scene and the confidence, and a sub-correspondence between the bright-dark boundary value of the ambient brightness in the backlight scene and the confidence. Then the second confidence includes a sub-confidence corresponding to the ambient brightness value, a sub-confidence corresponding to the dynamic range of the ambient brightness in the backlight scene, and a sub-confidence corresponding to the bright-dark boundary value of the ambient brightness in the backlight scene.

[0145] Optionally, the method can be applied to the above-mentioned image processing device. Figure 5 For other implementations and beneficial effects of the method shown, please refer to the relevant content of the above-mentioned image processing device, which will not be repeated here.

[0146] Below, taking the scene of taking photos by using the scene detection method provided in the embodiment of the present application as an example, Figure 6 The scene detection process is illustrated with an example.

[0147] Step 1: Capture an original image through a camera and calculate brightness statistics based on the original image.

[0148] The original image refers to an image that has not been scene compensated, corresponding to the image acquired in the above step 501. The brightness statistical information includes the above pixel brightness information and the environment brightness information.

[0149] Step 2: Make a preliminary judgment of the scene based on the brightness statistical information.

[0150] Among them, the result of the preliminary judgment is the above-mentioned first scene detection result.

[0151] Step 3: Calculate the night scene confidence based on the preliminary judgment result and the night scene confidence probability curve to achieve a secondary judgment of the night scene.

[0152] Among them, the night scene confidence corresponds to the above-mentioned first confidence.

[0153] Step 4: Calculate the backlight confidence based on the result of the preliminary judgment and the backlight confidence probability curve to achieve a secondary judgment of the backlight scene.

[0154] The backlight confidence level corresponds to the second confidence level mentioned above.

[0155] Step 5, compare the night scene confidence and the backlight confidence representatives to determine whether the current scene is a general scene, a backlight scene or a backlight scene.

[0156] In summary, the scene detection method provided in the embodiment of the present application, for a rough first scene detection result obtained by preliminary judgment through pixel brightness information, further constructs deeper ambient brightness information based on the pixel brightness information, so as to calibrate the first scene detection result according to the ambient brightness information and a preset correspondence, and obtain a second scene detection result. The second scene detection result can accurately indicate whether it is a night scene or a backlit scene, thereby solving the problem that it is difficult to distinguish between night scene scenes and backlit scenes only through pixel brightness information, so that the same scene detection module can detect whether the current image is a backlit scene or a night scene, thereby improving the scene detection capability and robustness.

[0157] The embodiment of the present application also provides a computer-readable storage medium in which at least one program code is stored, and the at least one program code is loaded and executed by a processor to enable a computer to implement Figure 5 The scene detection method shown.

[0158] The present application also provides an electronic device, the electronic device comprising: Figure 1 The image processing device shown.

[0159] It should be understood that the terms used in the embodiments of the present disclosure are only used to explain the embodiments of the present disclosure, and are not intended to limit the present disclosure. Unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be the common meanings understood by people with ordinary skills in the field to which the present disclosure belongs.

[0160] For example, the words "first", "second" or "third" and similar words used in the patent application specification and claims of this disclosure do not indicate any order, quantity or importance, but are merely used to distinguish different components.

[0161] Likewise, the words “a” or “an” and the like do not denote a limitation of quantity, but rather denote the presence of at least one.

[0162] Words such as “include” or “comprising” and the like mean that the elements or objects preceding “include” or “comprising” include the elements or objects listed after “include” or “comprising” and their equivalents, and do not exclude other elements or objects.

[0163] "Up", "down", "left" or "right" etc. are only used to indicate relative position relationship. When the absolute position of the described object changes, the relative position relationship may also change accordingly. "Connected" or "coupled" refers to electrical connection.

[0164] "And / or" indicates that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship.

[0165] Technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working processes of the gate drive circuit, shift register unit, each circuit and each sub-circuit described above can refer to the corresponding processes in the method embodiment and will not be repeated here.

[0166] The above description is only an optional embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An image processing device, characterized in that: The image processing device comprises: a scene detection module; the scene detection module is configured to: Acquire pixel brightness information of an image, wherein the pixel brightness information includes a brightness value of each pixel point on the image; Acquire a first scene detection result and ambient brightness information reflecting an environment in which the image was captured based on the pixel brightness information, wherein the first scene detection result indicates that the image was captured as at least one of a night scene or a backlit scene; The first scene detection result is calibrated according to a preset corresponding relationship and the ambient brightness information to obtain a second scene detection result, wherein the preset corresponding relationship includes a corresponding relationship between the ambient brightness information and the confidence level.

2. The image processing device according to claim 1, characterized in that The scene detection module is configured as follows: In a case where the pixel brightness information satisfies a backlight condition, determining that the first scene detection result includes the backlight scene, wherein the backlight condition is determined based on pixel brightness distribution characteristics of a plurality of images collected under the backlight scene; In a case where the pixel brightness information satisfies a night scene condition, it is determined that the first scene detection result includes the night scene, and the night scene condition is determined based on pixel brightness distribution characteristics of multiple images collected under the night scene scene.

3. The image processing device according to claim 1, characterized in that The ambient brightness information includes at least one of an ambient brightness value, a dynamic range of the ambient brightness, or a bright-dark dividing value of the ambient brightness. The ambient brightness value indicates the brightness value of the image acquisition environment. The dynamic range of the ambient brightness indicates the brightness variation range of the image acquisition environment. The bright-dark dividing value of the ambient brightness indicates the ratio of bright and dark areas in the image acquisition environment.

4. The image processing device according to claim 3, characterized in that: The ambient brightness information includes the ambient brightness value, and the scene detection module is configured to: Acquire average brightness information of the image based on the brightness values ​​of the respective pixels; The ambient brightness value is acquired according to the average brightness information, the exposure time and the aperture size of the image acquisition device.

5. The image processing device according to claim 3, characterized in that: The ambient brightness information includes a dynamic range of the ambient brightness, and the scene detection module is configured as follows: Based on the brightness values ​​of the pixels, a first average brightness within a bright area and a second average brightness within a dark area are obtained, and a dynamic range of the ambient brightness is obtained according to the first average brightness and the second average brightness; Among them, the bright area range refers to the pixel range on the image whose brightness value is greater than the first preset value, and the dark area range refers to the pixel range on the image whose brightness value is less than the second preset value. The bright area range or dark area range in the night scene is different from the bright area range or dark area range in the backlight scene.

6. The image processing device according to claim 3, characterized in that: The ambient brightness information includes a bright-dark boundary value of the ambient brightness, and the scene detection module is configured as follows: Sorting the pixels according to brightness value; In a case where the first scene detection result includes the night scene, using the brightness value of the pixel point arranged at the first preset position as the bright-dark boundary value of the ambient brightness in the night scene; In a case where the first scene detection result includes the backlit scene, the brightness value of the pixel point arranged at the second preset position is used as the bright-dark boundary value of the ambient brightness in the backlit scene.

7. The image processing device according to any one of claims 1 to 6, characterized in that: The preset corresponding relationship includes a first corresponding relationship in the night scene and a second corresponding relationship in the backlight scene, and the scene detection module is configured as follows: When the first scene detection result includes the night scene, acquiring a first confidence level of the night scene based on the ambient brightness information and the first corresponding relationship; When the first scene detection result includes the backlit scene, acquiring a second confidence level of the backlit scene based on the ambient brightness information and the second corresponding relationship; The second scene detection result is determined by comparing the first confidence level with the second confidence level.

8. A scene detection method, characterized in that: The method comprises: Acquire pixel brightness information of an image, wherein the pixel brightness information includes a brightness value of each pixel point on the image; Acquire a first scene detection result and ambient brightness information reflecting an environment in which the image was captured based on the pixel brightness information, wherein the first scene detection result indicates that the image was captured as at least one of a night scene or a backlit scene; The first scene detection result is calibrated according to a preset corresponding relationship and the ambient brightness information to obtain a second scene detection result, wherein the preset corresponding relationship includes a corresponding relationship between the ambient brightness information and the confidence level.

9. The method according to claim 8, characterized in that The acquiring a first scene detection result based on the pixel brightness information includes: In a case where the pixel brightness information satisfies a backlight condition, determining that the first scene detection result includes the backlight scene, wherein the backlight condition is determined based on pixel brightness distribution characteristics of a plurality of images collected under the backlight scene; In a case where the pixel brightness information satisfies a night scene condition, it is determined that the first scene detection result includes the night scene, and the night scene condition is determined based on pixel brightness distribution characteristics of multiple images collected under the night scene scene.

10. The method according to claim 8, characterized in that The ambient brightness information includes at least one of an ambient brightness value, a dynamic range of the ambient brightness, or a bright-dark dividing value of the ambient brightness. The ambient brightness value indicates the brightness value of the image acquisition environment. The dynamic range of the ambient brightness indicates the brightness variation range of the image acquisition environment. The bright-dark dividing value of the ambient brightness indicates the ratio of bright and dark areas in the image acquisition environment.

11. The method according to claim 10, characterized in that The ambient brightness information includes the ambient brightness value, and obtaining the ambient brightness value based on the pixel brightness information includes: Acquire average brightness information of the image based on the brightness values ​​of the respective pixels; The ambient brightness value is acquired according to the average brightness information, the exposure time and the aperture size of the image acquisition device.

12. The method according to claim 10, characterized in that The ambient brightness information includes a dynamic range of the ambient brightness, and obtaining the dynamic range of the ambient brightness based on the pixel brightness information includes: Based on the brightness values ​​of the pixels, a first average brightness within a bright area and a second average brightness within a dark area are obtained, and a dynamic range of the ambient brightness is obtained according to the first average brightness and the second average brightness; Among them, the bright area range refers to the pixel range on the image whose brightness value is greater than the first preset value, and the dark area range refers to the pixel range on the image whose brightness value is less than the second preset value. The bright area range or dark area range in the night scene is different from the bright area range or dark area range in the backlight scene.

13. The method according to claim 10, characterized in that The ambient brightness information includes a bright-dark boundary value of the ambient brightness, and obtaining the bright-dark boundary value of the ambient brightness based on the pixel brightness information includes: Sorting the pixels according to brightness value; In a case where the first scene detection result includes the night scene, using the brightness value of the pixel point arranged at the first preset position as the bright-dark boundary value of the ambient brightness in the night scene; In a case where the first scene detection result includes the backlit scene, the brightness value of the pixel point arranged at the second preset position is used as the bright-dark boundary value of the ambient brightness in the backlit scene.

14. The method according to any one of claims 8 to 13, characterized in that: The preset corresponding relationship includes a first corresponding relationship in the night scene and a second corresponding relationship in the backlight scene; The calibrating the first scene detection result according to the preset corresponding relationship and the ambient brightness information includes: When the first scene detection result includes the night scene, acquiring a first confidence level of the night scene based on the ambient brightness information and the first corresponding relationship; When the first scene detection result includes the backlit scene, acquiring a second confidence level of the backlit scene based on the ambient brightness information and the second corresponding relationship; The second scene detection result is determined by comparing the first confidence level with the second confidence level.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one program code, and the at least one program code is loaded and executed by the processor to enable the computer to implement the scene detection method according to any one of claims 8 to 14.

16. An electronic device, characterized in that: The electronic device comprises the image processing device as described in any one of claims 1-7.

Images