Image capturing method, electronic device and computer readable storage medium

Abstract

The application relates to the image shooting technical field and provides an image shooting method, an electronic device and a computer readable storage medium, which comprise the following steps: after an electronic device displays a first interface comprising a shooting shutter, the electronic device obtains a first image in an HDR mode in response to a click operation of a user on the shooting shutter; wherein the first image is an image obtained after color correction of an image collected by a camera and comprises first pixels with color overflow. Then, the electronic device performs color correction on the first pixels of the first image based on color overflow values of the first pixels to obtain a second image comprising second pixels. The color overflow value is a total pixel value exceeding a pixel value range, and the pixel value range corresponds to the color depth of the HDR mode. According to the method, the electronic device performs secondary color correction on the pixels with color overflow in the HDR mode shooting scene, so that the color overflow problem of high saturation objects in the imaging process can be avoided, and the content display is not unclear.

Description

Technical Field

[0001] This application relates to the field of image capture technology, and more particularly to image capture methods, electronic devices, and computer-readable storage media. Background Technology

[0002] With the development of image capture technology and the increasing demands of users for shooting functions and image effects, the shooting modes offered by cameras in electronic devices are becoming more and more diverse. Examples include portrait mode, panorama mode, and High Dynamic Range Imaging (HDR or HDRI) mode. Currently, in HDR mode, the dynamic range of the image is typically compressed to adapt to the display capabilities of the monitor.

[0003] However, high-saturation images are prone to severe color overflow due to compression, making it difficult to distinguish the image content corresponding to the high-saturation image, resulting in loss of detail and blurriness. Summary of the Invention

[0004] This application provides an image capturing method, an electronic device, and a computer-readable storage medium to solve the problem that high-saturation imaging objects in HDR mode suffer from color overflow due to compression, resulting in indistinguishable content and loss of detail.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, an image capture method is provided, which is applied to an electronic device, the electronic device including a camera, and the electronic device supporting high dynamic range (HDR) imaging mode, the method comprising:

[0007] The electronic device displays a first interface, which includes a preview image captured by the electronic device and a shutter button. In response to a user's click on the shutter button, a first image is obtained in HDR mode. This first image is a color-corrected version of the image captured by the camera, and includes a first pixel exhibiting color overflow. Then, the electronic device generates a second image based on the first image, including a second pixel. This second pixel corresponds to the first pixel in the first image and is obtained by color correction of the first pixel based on its color overflow value. The color overflow value is the total pixel value exceeding the pixel's value range, which corresponds to the color depth of the HDR mode.

[0008] In HDR images, if a highly saturated object exhibits color overflow, its corresponding pixels will also show color overflow. Therefore, this method involves the electronic device performing secondary color correction on pixels exhibiting color overflow during the color correction stage in HDR shooting scenarios. This prevents color overflow in pixels corresponding to highly saturated objects, ensuring that highly saturated objects do not show color overflow after imaging, and that the corresponding image content is displayed clearly with distinguishable details.

[0009] In one possible implementation of the first aspect, since color correction is currently typically based on a color correction matrix, the first image is an image captured by a camera that has undergone color correction using a first color correction matrix (a standard color correction matrix). Therefore, obtaining the second image based on the first image may include: calculating the color overflow value of the first pixel; generating a second color correction matrix corresponding to the first pixel based on the color overflow value of the first pixel; and re-color correcting the corresponding first pixel in the first image using each of the second color correction matrices to obtain the second image including the second pixel.

[0010] Therefore, a second color correction matrix is ​​generated for each first pixel based on its color overflow value to achieve secondary color correction of the first pixel. This is equivalent to performing secondary color correction based on the required color correction strength of the first pixel, thereby improving the color correction effect of the first pixel and ensuring that color overflow is avoided.

[0011] In another possible implementation of the first aspect, considering the actual business requirements for color correction intensity, generating a second color correction matrix corresponding to the first pixel based on the color overflow value of the first pixel may include: determining the ratio of the color overflow value to the maximum pixel value in the pixel value range; determining the first color adjustment weight based on the product of the ratio and the intensity control threshold; and using the product of the first color adjustment weight and the standard identity matrix as the second color correction matrix.

[0012] Therefore, it can be seen that after the electronic device determines the required adjustment intensity for the first pixel through the ratio, it further modifies it with the set intensity control threshold. rio Multiply by 3. Due to the intensity regulation threshold rio 3 is a preset value based on experience, used to adjust the intensity of color correction. Therefore, it can ensure that the obtained second CCM is a CCM that simultaneously meets the adjustment intensity required by the first pixel and the adjustment intensity required by actual business needs.

[0013] In another possible implementation of the first aspect, since the typical value range of the weight is [0,1], determining the first color adjustment weight of the first pixel based on the product of the ratio and the intensity control threshold can include: if the product of the ratio and the intensity control threshold is less than or equal to 0, the first color adjustment weight is equal to 0; if the product of the ratio and the intensity control threshold is greater than or equal to 1, the first color adjustment weight is equal to 1; if the product of the ratio and the intensity control threshold is greater than 0 and less than 1, the first color adjustment weight is equal to the product of the ratio and the intensity control threshold.

[0014] In another possible implementation of the first aspect, in order to accurately adjust the color of the first pixel and avoid color overflow caused by color truncation of the first pixel, resulting in unclear and indistinguishable image content, the electronic device needs to accurately determine the color overflow value of each first pixel. Furthermore, since a pixel typically corresponds to three color components (R, G, and B), the electronic device can first calculate the absolute difference between the pixel value of each of the three components and the pixel's value range. Then, the electronic device sums these three absolute differences to obtain the color overflow value of the first pixel.

[0015] Based on this, calculating the color overflow value of the first pixel may include: calculating the absolute difference between the three component pixel values ​​of the first pixel and the pixel value range respectively; wherein, the three component pixel values ​​are the pixel values ​​corresponding to the three RGB color components; and summing the absolute differences corresponding to the three component pixel values ​​to obtain the color overflow value of the first pixel.

[0016] In another possible implementation of the first aspect, since higher pixel values ​​corresponding to the R (red) and B (blue) color components of a pixel indicate a higher degree of color vividness, and more vividly colored objects (objects with higher saturation) are more likely to exhibit color overflow, the electronic device can further combine the RGB color ratio of the first pixel to generate a second CCM, thereby improving the correction effect for color overflow areas.

[0017] Therefore, the image capture method may further include: calculating the RGB color ratio of a first pixel, and determining a second color adjustment weight corresponding to the first pixel based on the RGB color ratio; wherein, the RGB color ratio is the ratio between the pixel values ​​corresponding to the three RGB color components. Furthermore, using the product of the first color adjustment weight and the standard identity matrix as the second color correction matrix may include: using the product of the second color adjustment weight and the first color adjustment weight as a new first color adjustment weight; and using the sum of the first product and the second product as the second color correction matrix; wherein, the first product is the product of the new first color adjustment weight and the standard identity matrix; and the second product is the product of the first color correction matrix and a first difference, where the first difference is the difference between the value 1 and the second color adjustment weight.

[0018] In another possible implementation of the first aspect, since the main considerations are the influence of the R and B color components on color vibrancy, and since the relationship between the remaining two values ​​can be derived by selecting a baseline value and calculating its ratio to the other two values, the RGB color ratios can mainly include the RG ratio and the BG ratio; the RG ratio is the ratio of the pixel values ​​corresponding to the R and G color components, and the BG ratio is the ratio of the pixel values ​​corresponding to the B and G color components.

[0019] Furthermore, determining the second color adjustment weight corresponding to the first pixel based on the RGB color ratio can include: determining the RG adjustment weight based on the RG ratio corresponding to the first pixel, and determining the BG adjustment weight based on the BG ratio corresponding to the first pixel; and using the product of the RG adjustment weight and the BG adjustment weight as the second color adjustment weight corresponding to the first pixel.

[0020] In another possible implementation of the first aspect, the RG adjustment weight is determined based on the RG ratio corresponding to the first pixel, including: when the RG ratio is greater than or equal to 10 or less than or equal to 0.1, the RG adjustment weight is equal to 1; when the RG ratio is greater than or equal to 1 and less than 10, the RG adjustment weight is equal to the ratio of the second difference to the value 9; the second difference is the difference between the RG ratio and the value 1; when the RG ratio is less than or equal to 1 and greater than 0.1, the RG adjustment weight is equal to the product of the value 10 / 9 and the third difference; the third difference is the difference between the value 1 and the RG ratio.

[0021] In another possible implementation of the first aspect, the BG adjustment weight is determined based on the BG ratio corresponding to the first pixel, including: when the BG ratio is greater than or equal to 10 or less than or equal to 0.1, the BG adjustment weight is equal to 1; when the BG ratio is greater than or equal to 1 and less than 10, the BG adjustment weight is equal to the ratio of the fourth difference to the value 9; the fourth difference is the difference between the BG ratio and the value 1; when the BG ratio is less than or equal to 1 and greater than 0.1, the BG adjustment weight is equal to the product of the value 10 / 9 and the fifth difference; the fifth difference is the difference between the value 1 and the BG ratio.

[0022] In another possible implementation of the first aspect, since color overflow of a pixel means that the pixel value exceeds the range of pixel values, and image pixels usually include three color components: R, G, and B, and each color component corresponds to a component pixel value, the electronic device can determine whether a pixel is the first pixel with color overflow by judging whether the three component pixel values ​​corresponding to each pixel in the first image exceed the range of pixel values.

[0023] Based on this, detecting the first pixel in the first image that has color overflow may include: comparing the component pixel value of each pixel in the first image with the pixel value range; wherein, the component pixel value includes the pixel value corresponding to the three color components of RGB; and taking the pixel in the first image whose component pixel value is not within the pixel value range as the first pixel with color overflow.

[0024] In another possible implementation of the first aspect, the second pixel can be calculated using the following formula:

[0025] R out = a 11 × R in + a 12 × G in + a 13 × B in

[0026] G out = a 21 × R in + a 22 × G in + a 23 × B in

[0027] B out = a 31 × R in + a 32 × G in + a 33 × B in

[0028] in, R in , G in , B in It is the RGB value of the first pixel; R out , G out , B out It is the RGB value of the second pixel; a 11 , a 12 , a 13 , a 21 , a 22 , a 23 , a 31 , a 32 , a 32 It is an element in the second color correction matrix.

[0029] In another possible implementation of the first aspect, there may be discrete noise pixels in the image. In order to avoid the influence of discrete noise pixels, the image capturing method may further include: discarding the first pixel when it is determined to be a discrete noise pixel based on the neighboring pixels of the first pixel.

[0030] In another possible implementation of the first aspect, determining that the first pixel is a discrete noise pixel based on its neighboring pixels may include: converting the first image into a binary image, wherein the pixel value of the first pixel in the binary image is 1; dividing the first pixel and its neighboring pixels into pixel blocks according to a preset block size in the binary image; if there are n pixels with a pixel value of 1 in the pixel block, the first pixel is not a discrete noise pixel; if there are no n pixels with a pixel value of 1 in the pixel block, the first pixel is a discrete noise pixel.

[0031] Secondly, this application provides an image capturing method, which is applied to an electronic device, the electronic device including a camera, and the electronic device supporting high dynamic range (HDR) imaging mode, the method comprising:

[0032] The electronic device receives a user's activation of the camera application; in response to the activation, the electronic device obtains a first image in HDR mode; wherein the first image is a color-corrected image of an image captured by the camera, and the first image includes a first pixel with color overflow. Then, the electronic device obtains a second image based on the first image, including a second pixel. The second pixel corresponds to the first pixel in the first image and is obtained by color correction of the first pixel based on its color overflow value. The color overflow value is the total pixel value exceeding the pixel's value range, and the pixel value range corresponds to the color depth of the HDR mode.

[0033] It should be noted that the image capturing method provided in the second aspect differs from that provided in the first aspect in the scenario in which the electronic device is triggered to capture the image. The second aspect addresses the scenario of opening the camera to capture a preview image, while the first aspect addresses the scenario of clicking the shutter button to actually capture the image.

[0034] However, it is understandable that regardless of the scenario in which the image capture method is triggered, its implementation principle and the effect it can achieve are the same. Therefore, the beneficial effects of the second aspect and any of its possible implementations can be compared with the beneficial effects of the first aspect and any of its possible implementations, and will not be elaborated here.

[0035] In another possible implementation of the second aspect, the first image is an image captured by a camera that has undergone color correction using a first color correction matrix (a standard color correction matrix). Therefore, obtaining the second image based on the first image can include: calculating the color overflow value of the first pixel and calculating the RGB color ratio of the first pixel; wherein the RGB color ratio is the ratio between the pixel values ​​corresponding to the three color components (RGB); generating a second color correction matrix corresponding to the first pixel based on the color overflow value and the RGB color ratio; and re-color correcting the corresponding first pixel in the first image using each of the second color correction matrices to obtain the second image including the second pixel.

[0036] In another possible implementation of the second aspect, calculating the color overflow value of the first pixel may include: calculating the absolute difference between the three component pixel values ​​of the first pixel and the pixel value range respectively; wherein the three component pixel values ​​are the pixel values ​​corresponding to the three RGB color components; and summing the absolute differences corresponding to the three component pixel values ​​to obtain the color overflow value of the first pixel.

[0037] In another possible implementation of the second aspect, generating a second color correction matrix corresponding to the first pixel based on the color overflow value and RGB color ratio of the first pixel may include: calculating the ratio of the color overflow value corresponding to the first pixel to the maximum pixel value in the pixel value range; determining the first color adjustment weight of the first pixel based on the product of the ratio and the intensity control threshold; calculating the second color adjustment weight of the first pixel based on the RGB color ratio of the first pixel; using the product of the second color adjustment weight and the first color adjustment weight as the new first color adjustment weight; and using the sum of the first product and the second product as the second color correction matrix; wherein the first product is the product of the new first color adjustment weight and the standard identity matrix; and the second product is the product of the first color correction matrix and the first difference, where the first difference is the difference between the value 1 and the second color adjustment weight.

[0038] In another possible implementation of the second aspect, determining the first color adjustment weight of the first pixel based on the product of the ratio and the intensity control threshold may include: if the product of the ratio and the intensity control threshold is less than or equal to 0, the first color adjustment weight is equal to 0; if the product of the ratio and the intensity control threshold is greater than or equal to 1, the first color adjustment weight is equal to 1; if the product of the ratio and the intensity control threshold is greater than 0 and less than 1, the first color adjustment weight is equal to the product of the ratio and the intensity control threshold.

[0039] In another possible implementation of the second aspect, the RGB color ratio includes the RG ratio and the BG ratio; the RG ratio is the ratio of the pixel values ​​corresponding to the R color component and the G color component, and the BG ratio is the ratio of the pixel values ​​corresponding to the B color component and the G color component; determining the second color adjustment weight corresponding to the first pixel based on the RGB color ratio may include: determining the RG adjustment weight based on the RG ratio corresponding to the first pixel, and determining the BG adjustment weight based on the BG ratio corresponding to the first pixel; and using the product of the RG adjustment weight and the BG adjustment weight as the second color adjustment weight corresponding to the first pixel.

[0040] In another possible implementation of the second aspect, the RG adjustment weight is determined based on the RG ratio corresponding to the first pixel, including: when the RG ratio is greater than or equal to 10 or less than or equal to 0.1, the RG adjustment weight is equal to 1; when the RG ratio is greater than or equal to 1 and less than 10, the RG adjustment weight is equal to the ratio of the second difference to the value 9; the second difference is the difference between the RG ratio and the value 1; when the RG ratio is less than or equal to 1 and greater than 0.1, the RG adjustment weight is equal to the product of the value 10 / 9 and the third difference; the third difference is the difference between the value 1 and the RG ratio.

[0041] In another possible implementation of the second aspect, the BG adjustment weight is determined based on the BG ratio corresponding to the first pixel, including: when the BG ratio is greater than or equal to 10 or less than or equal to 0.1, the BG adjustment weight is equal to 1; when the BG ratio is greater than or equal to 1 and less than 10, the BG adjustment weight is equal to the ratio of the fourth difference to the value 9; the fourth difference is the difference between the BG ratio and the value 1; when the BG ratio is less than or equal to 1 and greater than 0.1, the BG adjustment weight is equal to the product of the value 10 / 9 and the fifth difference; the fifth difference is the difference between the value 1 and the BG ratio.

[0042] In another possible implementation of the second aspect, the image capturing method further includes detecting a first pixel in the first image that has color overflow, including: comparing the component pixel value of each pixel in the first image with the pixel value range respectively; wherein, the component pixel value includes the pixel value corresponding to the three color components of RGB; and taking the pixel in the first image that has a component pixel value that is not within the pixel value range as the first pixel with color overflow.

[0043] In another possible implementation of the second aspect, the second pixel can be calculated using the following formula:

[0044] R out = a 11 × R in + a 12 × G in + a 13 × B in

[0045] G out = a 21 × R in + a 22 × G in +a 23 × B in

[0046] B out = a 31 × R in + a 32 × G in + a 33 × B in

[0047] in, R in , G in , B in It is the RGB value of the first pixel; R out , G out , B out It is the RGB value of the second pixel; a 11 , a 12 , a 13 , a 21 , a 22 , a 23 , a 31 , a 32 , a 32 It is an element in the second color correction matrix.

[0048] In another possible implementation of the second aspect, the image capturing method may further include: discarding the first pixel when it is determined from the neighboring pixels of the first pixel to be a discrete noise pixel.

[0049] In another possible implementation of the second aspect, determining that the first pixel is a discrete noise pixel based on the neighboring pixels of the first pixel includes: converting the first image into a binary image, wherein the pixel value of the first pixel in the binary image is 1; dividing the first pixel and its neighboring pixels into pixel blocks according to a preset block size in the binary image; if there are n pixels with a pixel value of 1 in the pixel block, the first pixel is not a discrete noise pixel; if there are no n pixels with a pixel value of 1 in the pixel block, the first pixel is a discrete noise pixel.

[0050] Thirdly, this application provides an electronic device, comprising: one or more processors and a memory, the memory being coupled to the processor; the memory storing one or more computer program codes, the computer program codes including computer instructions; when the processor executes the computer instructions, the electronic device performs the following steps:

[0051] Display a first interface, which includes a preview image captured by the electronic device and a shutter button; in response to the user's click operation on the shutter button, obtain a first image in HDR mode; or, receive the user's operation to open the camera application; in response to the opening operation, the electronic device obtains the first image in HDR mode; wherein, the first image is an image captured by the camera after color correction, and the first image includes a first pixel with color overflow.

[0052] A second image is obtained based on the first image, including the second pixel; wherein the second pixel corresponds to the first pixel in the first image, and is obtained by color correction of the first pixel based on the color overflow value of the first pixel. The color overflow value is the total pixel value exceeding the pixel value range, and the pixel value range corresponds to the color depth of the HDR mode.

[0053] In one possible implementation of the third aspect, when the aforementioned computer instructions are executed by the processor, the electronic device further performs the following steps: calculating the color overflow value of the first pixel; generating a second color correction matrix corresponding to the first pixel based on the color overflow value of the first pixel; and re-color correcting the corresponding first pixel in the first image using each of the second color correction matrices to obtain the second image including the second pixel.

[0054] In one possible implementation of the third aspect, when the aforementioned computer instructions are executed by the processor, the electronic device further performs the following steps: determining the ratio of the color overflow value to the maximum pixel value in the pixel value range; determining a first color adjustment weight based on the product of the ratio and the intensity control threshold; and using the product of the first color adjustment weight and the standard identity matrix as a second color correction matrix.

[0055] In one possible implementation of the third aspect, when the aforementioned computer instructions are executed by the processor, the electronic device further performs the following steps: if the product of the ratio and the intensity control threshold is less than or equal to 0, the first color adjustment weight is equal to 0; if the product of the ratio and the intensity control threshold is greater than or equal to 1, the first color adjustment weight is equal to 1; if the product of the ratio and the intensity control threshold is greater than 0 and less than 1, the first color adjustment weight is equal to the product of the ratio and the intensity control threshold.

[0056] In one possible implementation of the third aspect, when the aforementioned computer instructions are executed by the processor, the electronic device further performs the following steps: calculating the absolute differences between the three component pixel values ​​of the first pixel and the pixel value range, respectively; wherein the three component pixel values ​​are the pixel values ​​corresponding to the three RGB color components; and summing the absolute differences corresponding to the three component pixel values ​​to obtain the color overflow value of the first pixel.

[0057] In one possible implementation of the third aspect, when the aforementioned computer instructions are executed by the processor, the electronic device further performs the following steps: calculating the RGB color ratio of the first pixel, determining the second color adjustment weight corresponding to the first pixel based on the RGB color ratio; using the product of the second color adjustment weight and the first color adjustment weight as a new first color adjustment weight; using the sum of the first product and the second product as a second color correction matrix; wherein the first product is the product of the new first color adjustment weight and the standard identity matrix; and the second product is the product of the first color correction matrix and a first difference, the first difference being the difference between the value 1 and the second color adjustment weight.

[0058] In one possible implementation of the third aspect, when the aforementioned computer instructions are executed by the processor, the electronic device further performs the following steps: determining the RG adjustment weight based on the RG ratio corresponding to the first pixel, and determining the BG adjustment weight based on the BG ratio corresponding to the first pixel; and using the product of the RG adjustment weight and the BG adjustment weight as the second color adjustment weight corresponding to the first pixel.

[0059] In one possible implementation of the third aspect, when the aforementioned computer instructions are executed by the processor, the electronic device further performs the following steps: when the RG ratio is greater than or equal to 10 or less than or equal to 0.1, the RG adjustment weight is equal to 1; when the RG ratio is greater than or equal to 1 and less than 10, the RG adjustment weight is equal to the ratio of the second difference to the value 9; the second difference is the difference between the RG ratio and the value 1; when the RG ratio is less than or equal to 1 and greater than 0.1, the RG adjustment weight is equal to the product of the value 10 / 9 and the third difference; the third difference is the difference between the value 1 and the RG ratio.

[0060] In one possible implementation of the third aspect, when the aforementioned computer instructions are executed by the processor, the electronic device further performs the following steps: when the BG ratio is greater than or equal to 10 or less than or equal to 0.1, the BG adjustment weight is equal to 1; when the BG ratio is greater than or equal to 1 and less than 10, the BG adjustment weight is equal to the ratio of the fourth difference to the value 9; the fourth difference is the difference between the BG ratio and the value 1; when the BG ratio is less than or equal to 1 and greater than 0.1, the BG adjustment weight is equal to the product of the value 10 / 9 and the fifth difference; the fifth difference is the difference between the value 1 and the BG ratio.

[0061] In one possible implementation of the third aspect, when the aforementioned computer instructions are executed by the processor, the electronic device further performs the following steps: detecting a first pixel in the first image that has color overflow, including: comparing the component pixel values ​​of each pixel in the first image with the pixel value range respectively; wherein, the component pixel values ​​include the pixel values ​​corresponding to the three color components of RGB; and identifying pixels in the first image whose component pixel values ​​are not within the pixel value range as the first pixel with color overflow.

[0062] In one possible implementation of the third aspect, when the aforementioned computer instructions are executed by the processor, the electronic device further performs the following steps: calculating the second pixel using the following formula:

[0063] R out = a 11 × R in + a 12 × G in + a 13 × B in

[0064] G out = a 21 × R in + a 22 × G in + a 23 × B in

[0065] B out = a 31 × R in +a 32 × G in + a 33 × B in

[0066] in, R in , G in , B in It is the RGB value of the first pixel; R out , G out , B out It is the RGB value of the second pixel; a 11 , a 12 , a 13 , a 21 , a 22 , a 23 , a 31 , a 32 , a 32 It is an element in the second color correction matrix.

[0067] In one possible implementation of the third aspect, when the aforementioned computer instructions are executed by the processor, the electronic device further performs the following steps: discarding the first pixel when it is determined from the neighboring pixels of the first pixel to be a discrete noise pixel.

[0068] In one possible implementation of the third aspect, when the aforementioned computer instructions are executed by the processor, the electronic device further performs the following steps: converting the first image into a binary image, wherein the pixel value of the first pixel in the binary image is 1; dividing the first pixel and its neighboring pixels into pixel blocks according to a preset block size in the binary image; if there are n pixels with a pixel value of 1 in the pixel block, the first pixel is not a discrete noise pixel; if there are no n pixels with a pixel value of 1 in the pixel block, the first pixel is a discrete noise pixel.

[0069] Fourthly, this application provides an electronic device, comprising: one or more processors and a memory, the memory being coupled to the processor; the memory storing one or more computer program codes, the computer program codes including computer instructions; and when the processor executes the computer instructions, causing the electronic device to perform an image capturing method as described in the second aspect and any possible implementation thereof.

[0070] Fifthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor in an electronic device, causes the electronic device to perform an image capturing method as described in the first aspect and any possible implementation thereof, or the second aspect and any possible implementation thereof.

[0071] Sixthly, this application provides a computer program product that, when run on a computer, causes the computer to execute an image capturing method as described in the first aspect and any possible implementation thereof, or the second aspect and any possible implementation thereof. The computer may be the aforementioned electronic device.

[0072] Understandably, the beneficial effects that can be achieved by the electronic device of any possible implementation of the third aspect, the electronic device of any possible implementation of the fourth aspect, the computer-readable storage medium of the fifth aspect, and the computer program product of the sixth aspect can be referred to as the beneficial effects in the first aspect and any possible implementation of the second aspect and any possible implementation of the third aspect, which will not be repeated here. Attached Figure Description

[0073] Figure 1 An image illustration showing color overflow phenomenon provided in this application embodiment. Figure 1 ;

[0074] Figure 2 An image illustration showing color overflow phenomenon provided in this application embodiment. Figure 2 ;

[0075] Figure 3 This application provides an example of a scene that triggers the HDR mode. Figure 1 ;

[0076] Figure 4 This application provides an example of a scene that triggers the HDR mode. Figure 2 ;

[0077] Figure 5 This application provides an example of a scene that triggers the HDR mode. Figure 3 ;

[0078] Figure 6A flowchart illustrating an image capturing method provided in this application embodiment. Figure 1 ;

[0079] Figure 7 A schematic diagram of a first interface provided in an embodiment of this application;

[0080] Figure 8 An image illustration of a color overflow-free phenomenon provided in this application embodiment. Figure 1 ;

[0081] Figure 9 An image illustration of a color overflow-free phenomenon provided in this application embodiment. Figure 2 ;

[0082] Figure 10 A schematic diagram of a second interface provided in an embodiment of this application;

[0083] Figure 11 A schematic diagram of the structure of an electronic device 100 supporting HDR mode provided in an embodiment of this application;

[0084] Figure 12 A flowchart illustrating an image capturing method provided in this application embodiment. Figure 2 ;

[0085] Figure 13 A flowchart illustrating the generation of a second CCM provided in this application embodiment. Figure 1 ;

[0086] Figure 14 A schematic diagram illustrating the principle of an image capturing method provided in an embodiment of this application;

[0087] Figure 15 A flowchart illustrating an image capturing method provided in this application embodiment. Figure 3 ;

[0088] Figure 16 A flowchart illustrating the generation of a second CCM provided in this application embodiment. Figure 2 ;

[0089] Figure 17 This is a schematic diagram of a binary image provided in an embodiment of this application;

[0090] Figure 18 A schematic diagram of a pixel block provided in an embodiment of this application;

[0091] Figure 19 This is a schematic diagram of a chip system provided in an embodiment of this application. Detailed Implementation

[0092] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to limit the scope of this application.

[0093] Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" are not necessarily different. Also, in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0094] With the development of image capture technology and the increasing demands of users for shooting functions and image effects, the shooting modes that cameras in electronic devices can offer users are becoming more and more numerous. For example, the shooting modes that existing electronic devices can offer users include, but are not limited to, portrait mode, panorama mode, time-lapse photography, night scene mode, and high dynamic range imaging (HDR or HDRI) mode.

[0095] HDR is an imaging technique used to achieve a wider dynamic range (i.e., greater difference between light and dark areas) than ordinary digital imaging techniques. Its main purpose is to accurately represent the wide range of brightness in the real world, from direct sunlight to the darkest shadows. Simply put, compared to existing Low Dynamic Range (LDR) or Standard Dynamic Range (SDR), HDR can display a wider range of brightness, making the image more realistically present the brightness and color details of the actual scene, and making the overall image appear clearer in terms of brightness and contrast.

[0096] In other words, images and videos shot in HDR mode have higher brightness, color depth (bit depth), and a wider color gamut. Color depth is also called bit depth. For example, the color depth of HDR images can typically reach 10 bits, 12 bits, or 14 bits. A color depth of 10 bits means that the computer uses 10 bits as a unit of counting; the computer can represent 2^3 ... 10 There are (1024, 0-1023) colors. In SDR images, the bit depth representing a color is 8 bits, and a computer can represent 2^32 colors using these 8 bits. 8 (256, 0-255) colors.

[0097] However, most electronic devices currently equipped with displays can only discretize color channels to 8 bits, with only 255 levels of color space, resulting in a limited range of colors that can be displayed (i.e., grayscale from 0 to 255). Therefore, in order to adapt to the display capabilities of electronic devices and enable HDR images and videos to be displayed compatiblely on electronic devices, HDR tone mapping is required.

[0098] The purpose of HDR tone mapping can be simply understood as: to adapt high dynamic range (HDR) images to LDR or SDR display devices. Currently, to achieve this, HDR images need to be mapped to the standard range of LDR or SDR images, thereby compressing the dynamic range. In other words, when an electronic device captures an image in HDR mode, the image generation requires compressing the scene's dynamic range.

[0099] However, during compression, colors of highly saturated (vibrantly colored) objects outside the color gamut (0 to 255) are inevitably cropped out, resulting in minimal differences between the RGB channels of pixels. This leads to unclear textures and difficulty in distinguishing details. In other words, when electronic devices capture images in HDR mode, highly saturated objects are prone to severe color overflow due to scene dynamic range compression, making the corresponding image content difficult to discern, resulting in loss of detail and blurriness.

[0100] For example, such as Figure 1 and Figure 2 As shown, this is a schematic diagram of an image exhibiting color overflow.

[0101] Figure 1 The highly saturated object in the image shown is a dashboard. Because the lighting on this dashboard is a highly saturated red (red is not shown in the attached image), color bleeding occurs, causing the dashboard's content, such as speed, time, and gear position, to be unclear in the image.

[0102] Figure 2 The image shown is due to color overflow caused by a blue LED light source (the glowing cylinder in the image), resulting in color clipping throughout the scene. Figure 2 The entire scene shown in the image suffers from a loss of detail and is displayed as blurry.

[0103] Therefore, to avoid the problem of color overflow in high-saturation objects under HDR mode, which leads to indistinguishable image content and loss of detail, this application provides an image capturing method. The image capturing method provided in this application is applied to electronic devices that support HDR mode shooting. For example, mobile phones, tablets, cameras, camcorders, etc., that support HDR mode shooting. This application does not impose any special limitations on the specific type of electronic device.

[0104] When HDR mode is enabled, the electronic device can respond to the user's shooting operation by triggering the image shooting method provided in this application embodiment. The shooting operation can be in response to the user opening a camera application or clicking the shutter button.

[0105] The principle of the image capture method provided in this application embodiment is mainly as follows: After the electronic device performs conventional color correction processing on the image captured by the camera, for pixels in the color-corrected image that exhibit color overflow, a secondary color correction is performed based on the color overflow value corresponding to that pixel. This is because if a highly saturated object in the image exhibits color overflow, its corresponding pixel will also show color overflow. That is, the color overflow value of the pixel corresponding to the highly saturated object will exceed the pixel value range corresponding to the color depth in HDR mode. Therefore, this application embodiment performs a secondary color correction on this pixel based on its color overflow value, which can prevent color overflow in this pixel and thus ensure that the content of the highly saturated object corresponding to the pixel is displayed clearly.

[0106] In this application, the HDR mode in the electronic device can be configured to be enabled by default, based on actual product requirements, thus eliminating the need for user intervention. Therefore, the electronic device can trigger the image capture method provided in this embodiment as soon as it detects a user's shooting action or the opening of a camera application.

[0107] Optionally, the HDR mode on an electronic device can also be activated by the user through corresponding controls. For example, the user can activate the HDR mode in the shooting interface of the camera app on the electronic device, or in the settings interface of the camera app on the electronic device, or in the settings interface of the electronic device.

[0108] For example, an electronic device, such as a mobile phone, Figures 3-5 The images show different scenarios that trigger HDR mode.

[0109] Figure 3 This illustrates a scenario where a user triggers HDR mode from the camera app's shooting interface on their phone. (Reference) Figure 3When phone 1 is powered on, it can display something like this: Figure 3 The main interface 300 shown can include application icons for applications such as "Clock", "Calendar", "Camera", "Gallery", "Notes", and "Settings".

[0110] In response to the user's triggering operation on the "Camera" application icon 301 on the main interface 300, the mobile phone 1 enters the camera application and displays the shooting interface 302.

[0111] The shooting interface 302 includes multiple shooting options, such as "AI", "HDR", "Settings", "Aperture", "Night Scene", "Portrait", "Photo", "Video", and "Pro". Figure 3 The "HDR" shooting option in the shooting interface 302 shown is in the off state. Therefore, in response to the user's trigger operation on the "HDR" shooting option 303 in the shooting interface 302, the mobile phone 1 turns on the HDR shooting mode.

[0112] Figure 4 This illustrates a scenario where a user triggers HDR mode in the camera app's settings on their phone. (Reference) Figure 4 When phone 1 is powered on, it can display something like this: Figure 4 The main interface 400 shown can include icons for applications such as "Clock", "Calendar", "Camera", "Gallery", "Notes", etc., as well as application icons for applications such as "Settings" and "Camera X".

[0113] In this context, the "Camera" option on the main interface (400) can be understood as the camera application that comes pre-installed on the phone's operating system, while "[Camera Name]" can be understood as a third-party camera application installed on the phone. The scenario where a user triggers HDR mode in the shooting interface within the "Camera" application can be found in [reference needed]. Figure 3 As shown.

[0114] In the scenario where the user takes a picture using a third-party camera application, the mobile phone 1 responds to the user's trigger operation on the "Camera X" application icon 401 on the main interface 400, enters the third-party camera application, and displays the Camera X interface 402.

[0115] The camera interface 402 includes multiple function options, such as "Beauty," "Collage," "Background Cutout," "Home," "Camera," and "Me." In response to the user's triggering action on the "Me" option 403 within the camera interface 402, the mobile phone 1 enters the camera user interface 404.

[0116] The camera user interface 404 includes multiple function options, such as "Login," "Messages," "Settings," and "Feedback." In response to a user's triggering of the "Settings" option 405 within the camera user interface 404, the mobile phone 1 enters the camera settings interface 406.

[0117] The camera settings interface 406 includes multiple settings options, such as "Photo Settings," "Language Settings," and "Clear Cache." In response to the user's triggering of the "Photo Settings" option 407 in the camera user interface 406, the mobile phone 1 enters the photo settings interface 408.

[0118] The camera settings interface 408 includes multiple settings options, such as "Save Original Image," "Watermark," "Auto Mirror," and "HDR." In response to the user's triggering of the "HDR" option 409 in the camera settings interface 408, the phone 1 enables HDR shooting mode.

[0119] Figure 5 This illustrates a scene where a user triggers HDR mode in the settings interface of an electronic device. (Reference) Figure 5 When phone 1 is powered on, it can display something like this: Figure 5 The main interface 500 shown can include application icons for applications such as "Clock", "Calendar", "Camera", "Gallery", "Notes", and "Settings".

[0120] In response to the user's triggering operation on the "Settings" application icon 501 on the main interface 500, the mobile phone 1 enters the settings application and displays the settings application interface 502.

[0121] The settings application interface 502 can include multiple settings options, such as WLAN, Bluetooth, SIM card management, mobile network, music, camera, and other settings options. In response to the user selecting the "Camera" option 503 in the settings application interface 502, the mobile phone 1 enters the camera settings interface 504.

[0122] The camera settings interface 504 may include multiple camera settings options, such as HDR, grid, and level. In response to the user's triggering action on the "HDR" option 505 in the camera settings interface 504, the mobile phone 1 activates HDR shooting mode.

[0123] It should be noted that the above Figures 3-5 The HDR mode activation scenario shown is for example use in this application. The ways a user triggers HDR mode activation include, but are not limited to, those described above. Figures 3-5 The scenario shown is not limited to the specific implementation details in this application.

[0124] Since electronic devices typically capture preview images in response to a user opening a camera application, these preview images primarily serve to provide a preliminary view of the image. Therefore, compared to capturing an actual image in response to the shutter release, capturing a preview image involves less image information, resulting in a simpler and less intensive image processing flow. Consequently, highly saturated objects generally do not exhibit severe color clipping in the preview image, thus color overflow issues are unlikely to occur. Even if color overflow does occur due to clipping, its severity is relatively low. Therefore, the following embodiments of this application will primarily use the user's shutter release action as an example to illustrate the provided image capturing method.

[0125] Figure 6 The following is a flowchart illustrating the image capturing method provided in an embodiment of this application. The method will be described below in conjunction with electronic devices and... Figure 6 The flowchart shown provides a brief introduction to the image capturing method provided in the embodiments of this application.

[0126] refer to Figure 6 First, the electronic device displays a first interface, which includes a preview image captured by the electronic device and a shutter button. The shutter button triggers the electronic device to take a picture; the first interface can be the shooting interface of a camera application. After detecting a user's click on the shutter button on the first interface, the electronic device responds to that click by taking a picture.

[0127] For example, an electronic device, such as a mobile phone, Figure 7 A schematic diagram of a first interface is shown, including a shutter button 701. After the mobile phone detects a user's tap operation 702 on the shutter button 701, it takes an image in response to the tap operation 702. The preview image taken by the mobile phone can be displayed in the viewfinder 703.

[0128] Currently, whether electronic devices are responding to a shutter click to capture an image or responding to a user opening a camera app to preview an image, most of the process involves first using the camera to capture the raw image, and then processing the raw image according to the corresponding Image Signal Processing (ISP) process to obtain a displayable image.

[0129] If a preview image is captured, the electronic device will process the image according to the corresponding preview procedure to obtain a displayable preview image. However, if an actual image is captured, the electronic device will process the image according to the actual imaging procedure.

[0130] The main function of the ISP is to perform post-processing on the signal output from the front-end image sensor. It can be understood that in the entire process of camera shooting and imaging, the ISP is the first step in the camera's processing flow. After receiving the raw signal data (i.e., the raw image) collected by the photosensitive element (i.e., the image sensor) on the camera, the ISP unit (e.g., the ISP processor) processes this raw signal data to obtain a visible image.

[0131] The existing ISP process generally includes: image rasterization (bayer) → black level compensation → lens shading correction → bad pixel correction → color interpolation (demosaic) → bayer noise removal → automatic white balance (AWB) → color correction → gamma correction → color space conversion (e.g., RGB to YUV) → color noise removal and edge enhancement in YUV color space → color and contrast enhancement → outputting data in RGB (or YUV) format.

[0132] It should be noted that the above ISP process is used as an example in this application. The various processing steps in this ISP process can be added, subtracted or adjusted according to the actual situation, and this application embodiment does not limit this.

[0133] In this embodiment of the application, when the HDR mode is not enabled, the electronic device can process the raw image captured by the camera according to the existing ISP process to obtain an image that can be displayed.

[0134] However, with HDR mode enabled, the electronic device is triggered to further detect the first pixel in the first image that exhibits color overflow. The first image is the image output after the color correction step in the ISP process. In other words, the first image is the color-corrected version of the image captured by the camera.

[0135] Currently, the color correction step in the ISP process is mainly implemented using a 3×3 matrix, which is called the color correction matrix (CCM).

[0136] Each element in the CCM represents the linear relationship between a color channel (R, G, and B) in the input image and the corresponding color channel in the output image. By adjusting the elements in the CCM, conversion between different color spaces can be achieved, thereby correcting the image colors. In other words, the core of the CCM lies in the elements of the matrix, which can generally be obtained through pre-calibration. For example, the CMM matrix can be calculated using the least squares method; the specific implementation can refer to any existing publicly available method, and this application does not limit this approach.

[0137] A CCM obtained according to existing known methods can be understood as a standard CCM used for color correction in this technical field. For ease of distinction, the embodiments of this application are hereinafter referred to as the first CCM. Therefore, in some embodiments, the first image in the embodiments of this application may be an image after color correction by the first CCM.

[0138] After obtaining the first image, the electronic device generates a second image based on it. Specifically, the electronic device first detects first pixels in the first image that exhibit color overflow; these first pixels are those in the first image where color overflow occurs. Then, the electronic device performs color correction on the first pixel individually, based on the degree of color overflow, to obtain a second pixel corresponding to the first pixel. This can be understood as the second pixel being a pixel obtained by performing a second color correction on the first pixel.

[0139] After the electronic device re-corrects the color of each first pixel in the first image that exhibits color overflow based on its corresponding degree of color overflow, a second image including the second pixel can be obtained. In other words, the electronic device performs color correction on each first pixel in the first image to obtain the second image, thus avoiding the problem of unclear image content caused by subsequent color overflow.

[0140] Generally speaking, since color overflow is more likely to occur in HDR mode, which mainly involves high-saturation objects, it can be determined that these first pixels are the pixels corresponding to high-saturation objects (i.e., brightly colored objects).

[0141] In this embodiment, the degree of color overflow of the first pixel can be determined based on the color overflow value of the first pixel. The color overflow value refers to the total pixel value exceeding the pixel's value range. Since this application targets image capture in HDR mode, the pixel value range should correspond to the color depth of HDR supported by the electronic device.

[0142] For example, if an electronic device is configured to support 10-bit HDR color depth, then the corresponding pixel value range is (2... 0 ,2 10That is, (0, 1023). In other words, the electronic device can re-correct the color of the first pixel based on the color overflow value of the first pixel to obtain the corresponding second pixel.

[0143] In some embodiments, since color correction is generally implemented based on a 3×3 Color Filtering Mechanism (CCM), the electronic device can generate a second CCM corresponding to the first pixel based on the color overflow value of the first pixel, and then use the second CCM to re-correct the color of the first pixel to obtain the second pixel. That is, the electronic device adaptively generates a new CCM corresponding to each first pixel in the first image based on the degree of color overflow, and then uses the new CCM to perform secondary color correction on the first pixel. In other words, the second CCM can be understood as a CCM generated based on the degree of color overflow of the first pixel and suitable for color correction of that first pixel.

[0144] Specifically, after acquiring the first image output by the first CCM correction, the electronic device can generate a second CCM corresponding to each first pixel based on the color overflow value of the first pixel in the first image. Then, the electronic device uses these second CCMs to re-correct the color of the corresponding first pixels in the first image, thereby obtaining a second image including the second pixels.

[0145] Understandably, each first pixel in the first image has a corresponding second CCM. The second pixel is obtained by performing secondary color correction on the first pixel using the corresponding second CCM.

[0146] Therefore, the second image is composed of the second pixel and the pixels in the first image that do not exhibit color overflow (i.e., all pixels in the first image other than the first pixel). The second pixel is obtained by color correction of the first pixel.

[0147] Therefore, after the electronic device obtains the second image including the second pixel, it indicates that the electronic device has completed color correction for the pixels (i.e., the first pixel) in the captured image that exhibit color overflow. It can be understood that the second image in this embodiment refers to the image obtained after the electronic device performs color correction to address the color overflow phenomenon. This is equivalent to performing color correction on the first image exhibiting color overflow to obtain the corresponding second image.

[0148] Subsequently, the electronic device can continue to perform subsequent image processing based on the second image according to the ISP process to generate a visual image. Since the color overflow problem has been dealt with in the color correction stage, high-saturation objects in the final generated visual image will not exhibit color overflow.

[0149] For example, in the ISP process, the processing after color correction is usually gamma correction. After the electronic device re-corrects the first pixel in the first image using the second CCM, it inputs the resulting second image, including the second pixel, into the gamma correction to continue the subsequent image processing flow, ultimately generating a visual image. In the embodiments of this application, the visual image can be understood as a displayable third image generated based on the second image including the second pixel.

[0150] It should be noted that after obtaining the second image including the second pixel in the embodiments of this application, the specific implementation of the subsequent image processing flow to obtain the third image based on the second image can be carried out in accordance with the image capture process of any known electronic device, and this application does not limit it in this regard.

[0151] like Figures 8-9 As shown, with Figures 1-2 Using the same shooting scene as shown, an image diagram without color overflow is provided, namely, a diagram of the third image.

[0152] refer to Figure 1 and Figure 8 , Figure 1 This can be understood as an image that has not been reprocessed by the second CCM of this application (equivalent to a third image obtained based on the first image), while Figure 8 It is the final output image after being reprocessed by the second CCM in the embodiment of this application (i.e., the third image obtained based on the second image above).

[0153] By comparison Figure 1 and Figure 8 As can be seen, after the second CCM in this application embodiment performs color correction again, Figure 8 The contents of the dashboard shown are compared to Figure 1 This makes the display clearer.

[0154] Similarly, refer to Figure 2 and Figure 9 By comparison Figure 2 and Figure 9 It can be seen that after the processing of the second CCM in the embodiments of this application, Figure 9 The content of the scene shown is compared to Figure 2 It also displays more clearly.

[0155] Furthermore, the electronic device can further display the visual third image obtained from the final imaging of the second image. For example, the third image can be displayed on a second interface. This second interface can be the shooting interface. Depending on the actual design of the camera application, the electronic device can display the third image as a large image on the shooting interface, or it can display the third image as a small image (thumbnail) on the shooting interface.

[0156] For example, Figure 10 A schematic diagram of a second interface including a third image is shown. The third image can be displayed as a large image in area 1001 (i.e., directly in the viewfinder of the shooting interface). Alternatively, the third image can also be displayed as a small image in area 1002 (i.e., the captured third image is directly stored in the image library).

[0157] It should be noted that, Figure 10 The way the third image is displayed is only for example in this application and does not constitute a limitation on the display of the second image.

[0158] In addition, if the electronic device executes the image capturing method of this application embodiment in response to the user opening the camera application, then the third image finally obtained by the electronic device is a preview image, and therefore the third image can be directly displayed in the viewfinder.

[0159] Therefore, in order to avoid the phenomenon of color overflow causing unclear image content, the embodiments of this application re-correct the color based on the actual overflow situation of the first pixel in the first image where color overflow exists. This can prevent color overflow in subsequent compression and ensure that the image content corresponding to high-saturation objects is clearly visible and easy to distinguish.

[0160] Furthermore, it should be emphasized that in this embodiment, the recoloring is performed only on the first pixel in the first image. In other words, this embodiment performs local color correction on pixels with overflowing colors (i.e., only correcting the overflowing areas without affecting the non-overflowing areas), thereby ensuring that the details of the highly saturated imaged object are clearly distinguishable while avoiding affecting other normal pixels.

[0161] Figure 11 A schematic diagram of the structure of an electronic device 100 that supports HDR mode is shown.

[0162] Electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) connector 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera module 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A and a touch sensor 180B.

[0163] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0164] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0165] The processor can generate operation control signals based on the instruction opcode and timing signals to control the instruction fetching and execution. For example, the image capturing method described above can be implemented by processor 110.

[0166] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 may be a cache memory. This memory can store instructions or data that the processor 110 has used or that are used frequently. If the processor 110 needs to use the instruction or data, it can directly retrieve it from this memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0167] In some embodiments, the processor 110 may include one or more interfaces. These interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc. The processor 110 can connect to modules such as touch sensors, audio modules, wireless communication modules, displays, and camera modules through at least one of these interfaces.

[0168] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0169] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card, or music, video, and other files can be transferred from the electronic device to the external memory card.

[0170] Internal memory 121 can be used to store computer executable program code, including instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phone book, etc.). In addition, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional methods or data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory disposed in the processor.

[0171] The electronic device 100 can implement display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information. For example, the GPU can be used to render and the display screen 194 can be used to display a second image generated according to the embodiments of this application.

[0172] Display screen 194 is used to display images, videos, etc. For example, it is used to display a third image obtained based on the second image described above. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or more display screens 194.

[0173] The electronic device 100 can implement camera functions through a camera module 193, ISP, video codec, GPU, display screen 194, and application processor (AP), neural network processor (NPU), etc. For example, the camera module 193 and ISP are used to generate the second image in the embodiments of this application.

[0174] The camera module 193 can be used to acquire color image data and depth data of the subject. The ISP can be used to process the color image data acquired by the camera module 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element (i.e., image sensor). The light signal is converted into an electrical signal, and the camera's photosensitive element transmits this electrical signal to the ISP for processing, converting it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene.

[0175] In some embodiments, the ISP may be located in the camera module 193.

[0176] In some embodiments, the camera module 193 may consist of a color camera module and a 3D sensing module.

[0177] In some embodiments, the photosensitive element of the camera in the color camera module can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats.

[0178] In some embodiments, the 3D sensing module can be a time-of-flight (TOF) 3D sensing module or a structured light 3D sensing module. Structured light 3D sensing is an active depth sensing technology, and its basic components may include an infrared emitter, an IR camera module, etc. The working principle of a structured light 3D sensing module is to first emit a specific pattern of light onto the object being photographed, then receive the light coding on the object's surface, compare it with the original projected light pattern, and calculate the object's three-dimensional coordinates using triangulation principles. These three-dimensional coordinates include the distance between the electronic device 100 and the object being photographed. Similarly, TOF 3D sensing can be an active depth sensing technology, and its basic components may include an infrared emitter, an IR camera module, etc. The working principle of a TOF 3D sensing module is to calculate the distance (i.e., depth) between the TOF 3D sensing module and the object being photographed by measuring the infrared reflection time to obtain a 3D depth map.

[0179] Structured light 3D sensing modules can also be applied to fields such as facial recognition, motion-sensing game consoles, and industrial machine vision inspection. Time-of-flight (TOF) 3D sensing modules can also be applied to fields such as game consoles, augmented reality (AR) / virtual reality (VR).

[0180] In other embodiments, the camera module 193 may also consist of two or more cameras. These two or more cameras may include a color camera, which can be used to acquire color image data of the object being photographed. These two or more cameras may employ stereo vision technology to acquire depth data of the object being photographed. Stereo vision technology is based on the principle of human parallax. Under natural light, two or more cameras capture images of the same object from different angles, and then triangulation and other calculations are performed to obtain the distance information, i.e., depth information, between the electronic device 100 and the object being photographed.

[0181] In some embodiments, the electronic device 100 may include one or more camera modules 193. Specifically, the electronic device 100 may include one front-facing camera module 193 and one rear-facing camera module 193. The front-facing camera module 193 is typically used to capture color image data and depth data of the photographer facing the display screen 194, while the rear-facing camera module is used to capture color image data and depth data of the subject (such as a person, landscape, etc.) being photographed by the photographer.

[0182] In some embodiments, the CPU, GPU, or NPU in processor 110 can process the color image data and depth data acquired by camera module 193. In some embodiments, the NPU can identify the skeletal points of the subject by using neural network algorithms based on skeletal point recognition technology, such as convolutional neural network algorithms (CNN). The CPU or GPU can also run neural network algorithms to determine the skeletal points of the subject based on the color image data.

[0183] Digital signal processors (DSPs) are used to process digital signals, and can also process other digital signals. For example, when electronic device 100 selects a frequency point, the DSP is used to perform Fourier transforms on the frequency energy, etc.

[0184] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0185] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs can enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0186] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the intensity of the touch operation based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0187] Touch sensor 180B, also known as a "touch device," can be located on display screen 194. The touch sensor 180B and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K is used to detect touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0188] Specifically, in the embodiments of this application, the electronic device 100 can detect the user's click operation on the shutter button and the user's opening operation of the camera application through the pressure sensor 180A and the touch sensor 180B.

[0189] USB connector 130 is a USB standard compliant interface used to connect electronic device 100 and peripheral devices, specifically a Mini USB connector, Micro USB connector, USB Type-C connector, etc. Charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. Power management module 141 connects to battery 142, and charging management module 140 connects to processor 110.

[0190] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0191] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0192] Button 190 may include a power button, volume buttons, etc. Motor 191 can generate vibration feedback. Indicator 192 may be an indicator light, used to indicate charging status, battery level changes, and also to indicate messages, missed calls, notifications, etc. SIM card interface 195 is used to connect a SIM card.

[0193] It should be noted that the image capturing methods described in the following embodiments can all be implemented in the electronic device 100 with the above-described hardware structure.

[0194] The following, combined with Figure 12 The flowchart shown below provides a detailed explanation of the image capturing method proposed in the embodiments of this application. (Refer to...) Figure 12 The image capturing method provided in this application mainly includes steps S1201-S1207.

[0195] S1201, the electronic device acquires a first image that has been color-corrected using a first CCM.

[0196] The electronic device takes an image previewing the picture in response to the user's action of opening the camera application. Alternatively, the electronic device opens the camera application and displays a first screen including the preview image and the shutter button (the first screen may be...). Figure 7 After the shooting interface shown, if the user clicks the shutter button on the first interface, the electronic device will respond to the user's click operation and start the actual image capture.

[0197] During image capture, the electronic device first uses the camera to acquire the raw image. Then, the electronic device processes the raw image acquired by the camera according to the ISP (Image Signal Processor) procedure.

[0198] When the ISP process reaches the color correction step, the electronic device acquires the image output after color correction, that is, the electronic device acquires the first image that has been color corrected using the first CCM.

[0199] S1202, The electronic device detects a first pixel in the first image that has color overflow.

[0200] After acquiring the first image, the electronic device performs color overflow detection on each pixel in the first image to determine the first pixel in the first image that has color overflow.

[0201] In some embodiments, color overflow can be understood as a pixel value exceeding the pixel value range. That is, if the pixel value of a certain pixel exceeds the pixel value range, then this pixel can be identified as the first pixel exhibiting color overflow.

[0202] Meanwhile, image pixels typically include three color components: R, G, and B, and each color component corresponds to a pixel value, hereinafter referred to as the component pixel value. Therefore, electronic devices can determine whether a pixel is the first pixel with color overflow by judging whether the three component pixel values ​​corresponding to each pixel in the first image exceed the pixel value range. If the component pixel values ​​of a pixel exceed the pixel value range, then this pixel is the first pixel with color overflow.

[0203] Based on this, in S1202, the electronic device detects a first pixel in the first image that has color overflow, which may include: the electronic device comparing the component pixel value of each pixel in the first image with the pixel value range respectively; the electronic device taking the pixel in the first image whose component pixel value is not within the pixel value range as the first pixel with color overflow.

[0204] It should be noted that the pixel value range in this embodiment corresponds to the color depth of HDR mode. That is, the pixel value range is not the commonly used 8-bit range (0, 255). This is because, although the output image after imaging processing conforms to the 8-bit color depth (0 to 255) that the display can show, if the electronic device captures an image in HDR mode, the color depth of the image during imaging processing is still the HDR color depth. In other words, when capturing an image in HDR mode, the color depth of the input image is usually higher than the color depth of the output image.

[0205] The color depth in HDR mode is usually higher than the normal 8-bit, for example, it can be 10-bit, 12-bit or 14-bit, and the specific bit value depends on the actual configuration of the electronic device.

[0206] Therefore, if the electronic device supports 10-bit color depth HDR, the corresponding pixel value range is (0, 1023). Similarly, if the electronic device supports 14-bit color depth, the corresponding pixel value range is (0, 16383).

[0207] In this embodiment of the application, the pixel value range can be expressed as ( thre 1, thre2) thre 1 is the minimum pixel value within the range of pixel values. thre 2 is the maximum pixel value in the range of pixel values.

[0208] Then, the electronic device compares the component pixel values ​​(R value, G value, and B value) of the three color components of the pixel with the pixel value range, as shown below:

[0209] R value > thre 2 or R value < thre 1;

[0210] G value > thre 2 or G value < thre 1;

[0211] B value > thre 2 or B value < thre 1;

[0212] Furthermore, if a pixel's RGB values ​​satisfy any one or more of the above conditions, then this pixel can be determined as the first pixel to exhibit color overflow. That is, if any pixel in the first image has a component pixel value that is outside the range of pixel values, then this pixel is the first pixel to exhibit color overflow.

[0213] In addition, since images are captured in HDR mode, the colors of highly saturated objects are usually overflowing, so the first pixel detected by the electronic device can be identified as the pixel corresponding to the highly saturated object.

[0214] S1203, the electronic device calculates the color overflow value of the first pixel.

[0215] Color overflow refers to a pixel value exceeding the range of pixel values. Therefore, color overflow value refers to a value that exceeds the range of pixel values. thre 1, thre 2) Total pixel value. Since color overflow occurs because pixel colors are truncated during dynamic range compression, the color overflow value of a pixel can actually be understood as the truncated color of that pixel if compressed. That is, the larger the color overflow value of the first pixel, the greater the color truncation intensity of that first pixel will be if dynamic range compression is performed.

[0216] Therefore, after the electronic device detects the first pixel in the first image that has color overflow, in order to accurately adjust the color of the first pixel and avoid the problem of unclear and indistinguishable image content caused by color overflow due to color truncation of the first pixel, the electronic device first calculates and determines the color overflow value of each first pixel.

[0217] In some embodiments, since a pixel typically corresponds to three color components: R, G, and B, and each color component has a corresponding pixel value (i.e., component pixel value), the electronic device can first calculate the absolute difference between the three component pixel values ​​corresponding to the first pixel and the pixel value range. Then, the electronic device sums these three absolute differences to obtain the color overflow value (also known as the RGB color overflow value) of the first pixel.

[0218] That is, the electronic device first calculates the pixel values ​​that exceed the three color components corresponding to the first pixel, and then sums the pixel values ​​that exceed the three color components to obtain the RGB color overflow value.

[0219] RGB color overflow value sumN The following formula can be used for calculation:

[0220]

[0221] S1204, the electronic device generates the second CCM corresponding to the first pixel based on the color overflow value of the first pixel.

[0222] Once the electronic device determines the color overflow value of the first pixel, it can adaptively generate a new color clipping mechanism (CCM) corresponding to that first pixel based on the color overflow value of each subsequent first pixel. In other words, a second CCM can be generated based on the color overflow value of the first pixel, which can be precisely used to adjust the color of that first pixel.

[0223] Since a corresponding second CCM is generated for each first pixel, it can be understood that the number of second CCMs generated is equal to the number of first pixels in the first image, and these second CCMs correspond one-to-one with the first pixels.

[0224] Specifically, the larger the color overflow value of the first pixel, the more color overflows from the first pixel. Therefore, in order to achieve a good color correction effect, the color correction intensity of the first pixel can be increased accordingly.

[0225] Therefore, electronic devices can compare the color overflow value of the first pixel with the maximum pixel value. thre The ratio of 2 is considered as the adjustment level required for color correction of this first pixel. Furthermore, the electronic device generates a second CCM suitable for this first pixel using a determined adjustment level.

[0226] For example, the color overflow value of the first pixel can be compared with the maximum pixel value. thre The ratio of 2 to the standard identity matrix I Multiply them, and the resulting matrix is ​​used as the second CCM.

[0227] In some embodiments, such as Figure 13As shown, in S1204, the electronic device generates a second CCM based on the color overflow value of the first pixel, which may include: the electronic device determining the ratio of the color overflow value to the maximum pixel value in the pixel value range; the electronic device determining a first color adjustment weight based on the product of the ratio and the intensity control threshold; and the electronic device using the product of the first color adjustment weight and the standard identity matrix as the second color correction matrix.

[0228] Electronic devices based on the color overflow value of the first pixel sumN When generating the second CCM, the color overflow value of the first pixel can be calculated first. sumN With the range of pixel values ​​( thre 1, thre 2) Maximum pixel value thre The ratio of 2 sumN / thre 2. This ratio sumN / thre 2 refers to the adjustment level required to correct the color of the first pixel to prevent color overflow in the first pixel.

[0229] Then, the electronic device calculates this ratio. sumN / thre 2 and the preset intensity control threshold rio The product of 3 is used to determine the first color adjustment weight.

[0230] Among them, intensity control threshold rio 3 is a preset value based on experience, used to adjust the intensity of color correction. In the embodiments of this application, 10 ≤ rio 3≤100, rio A higher value of 3 indicates greater intensity. The specific intensity can be adjusted according to the actual business needs, and this application does not impose any limitations on it. The first color adjustment weight is the adjustment level for color correction of the first pixel ultimately determined by the electronic device.

[0231] That is, the electronic device determines the adjustment level (i.e., ratio) required for the first pixel. sumN / thre 2) After that, it can be further adjusted with the set intensity control threshold. rio Multiplying by 3 yields the first color adjustment weight that ultimately satisfies the adjustment requirements.

[0232] In some embodiments, determining the first color adjustment weight based on the product of the ratio and the intensity control threshold may include: when the product of the ratio and the intensity control threshold is less than or equal to 0, the first color adjustment weight is equal to 0; when the product of the ratio and the intensity control threshold is greater than or equal to 1, the first color adjustment weight is equal to 1; when the product of the ratio and the intensity control threshold is greater than or equal to 0 and less than or equal to 1, the first color adjustment weight is equal to the product of the ratio and the intensity control threshold.

[0233] Because the typical range of weight values ​​is [0,1]. Therefore, in sumN / thre 2* rio When 3≤0, the weight of the first color is adjusted. Weight clip =0.

[0234] exist sumN / thre 2* rio When 3 ≥ 1, the weight of the first color is adjusted. Weight clip =1.

[0235] And 0 < sumN / thre 2* rio When 3 < 1, the actual value can be used, and the weight of the first color will be adjusted. Weight clip = sumN / thre 2* rio 3.

[0236] Electronic devices receive the first color adjustment weight Weight clip Next, the first color weight Weight clip With the standard identity matrix I The product of these is the second CCM. That is, the second CCM = Weight clip * I Since existing CCMs are typically 3×3 matrices, the standard identity matrix in this embodiment is used. I The expression is as follows:

[0237]

[0238] Therefore, it can be seen that due to the standard identity matrix I It is a matrix where the diagonal elements are 1s and the other elements are 0s. Therefore, the weight of the first color is adjusted. Weight clip With the standard identity matrix I The product of these two values ​​serves as the second CCM, ensuring that the resulting second CCM simultaneously satisfies both the adjustment intensity required for the first pixel and the adjustment strength required by the actual business requirements.

[0239] Therefore, the electronic device can achieve the best color correction effect by using this second CCM to re-correct the color of the first pixel.

[0240] S1205, the electronic device uses the second CCM to re-correct the color of the first pixel to obtain the second pixel.

[0241] After the electronic device obtains the new second CCM, it can use this second CCM to recalibrate the corresponding first pixel using the traditional color calibration method to obtain the second pixel.

[0242] The RGB value of the second pixel obtained after recoloring can be calculated using the following formula:

[0243] R out = a 11 × R in + a 12 × G in + a 13 × B in

[0244] G out = a 21 × R in + a 22 × G in + a 23 × B in

[0245] B out = a 31 × R in + a 32 × G in + a 33 × B in

[0246] in, R in , G in , B in It is the RGB value of the pixel input before recoloring (i.e., the RGB value of the first pixel). R out ,G out , B out It is the RGB value of the pixel output after recoloring (i.e., the RGB value of the second pixel). a 11 , a 12 , a 13 , a 21 , a 22 , a 23 , a 31 , a 32 , a 32 These are elements in the second CCM, expressed as follows:

[0247]

[0248] Since the second CCM is the color adjustment weight and standard identity matrix I The matrix obtained by multiplication, and the standard identity matrix. I All elements outside the diagonal are 0, so the element a 12 , a 13 , a 21 , a 23 , a 31 , a 32 It can actually be 0.

[0249] S1206, the electronic device obtains a second image including the second pixel.

[0250] S1207, the electronic device obtains a third image based on the second image and displays the third image.

[0251] After the electronic device re-corrects the color of the first pixel in the first image that has color overflow using the second CCM, the adjustment of the color-overflowing pixel is completed. Therefore, the electronic device can obtain an image based on the second pixel that will not cause color overflow in high-saturation imaging objects due to subsequent imaging compression, which is the second image of this application embodiment. That is, the second image includes the second pixel and all other pixels in the first image except for the first pixel.

[0252] Understandably, the second image is an image obtained by performing secondary color correction on local pixels in the first image, and the local pixels that have undergone secondary color correction are the first pixels that were detected to have color overflow.

[0253] After obtaining the second image, the electronic device can continue to perform subsequent imaging processing based on the second image to generate a third image that can be displayed.

[0254] Therefore, in order to avoid the phenomenon of color overflow causing unclear image content, this application embodiment generates a new second CCM based on the actual overflow situation of the detected color overflow pixels, and then uses this second CCM to re-correct the color of the color overflow pixels, thereby avoiding color overflow in subsequent compression and ensuring that the image content corresponding to high-saturation objects is clearly visible and easy to distinguish.

[0255] Furthermore, it should be emphasized that in this embodiment, the second CCM is an adaptively generated CCM for the first pixel, and the second CCM is only used to recalibrate the corresponding first pixel. Other pixels in the first image, after undergoing color correction processing by the first CCM, will not undergo further color correction. In other words, this embodiment performs local color correction on pixels with color overflow (i.e., only corrects the areas with color overflow, without affecting the areas without color overflow), thereby ensuring that the details of the highly saturated imaged object are clearly distinguishable while avoiding affecting other normal pixels.

[0256] Figure 14 A schematic block diagram illustrating the image capturing method provided in this application is shown. Combined with... Figure 14 As can be seen from the above embodiments, the principle of the image capturing method provided in this application can be briefly summarized as follows:

[0257] When an electronic device responds to a user's action of opening the camera application or clicking the shutter button to capture an image, it takes a second color correction step after the color correction step in the ISP process (the first CCM performs color correction). In the process of acquiring the original image according to the conventional image capture principle and generating and displaying the image according to the conventional ISP process, a second color correction step is added.

[0258] For example, the ISP process in the image capture process of this application embodiment may include: image rasterization (bayer) → black level compensation → lens shading correction → bad pixel correction → color interpolation (demosaic) → bayer noise removal → automatic white balance (AWB) → color correction → secondary color correction → gamma correction → color space conversion (e.g., RGB to YUV) → color noise removal and edge enhancement in YUV color space → color and contrast enhancement → outputting data in RGB (or YUV) format.

[0259] Therefore, it can be seen that, compared with the above-mentioned ISP process, the ISP process in this application embodiment has an additional secondary color correction processing step.

[0260] In this embodiment, the principle of secondary color correction is to perform secondary color correction on pixels with color overflow. That is, the electronic device performs local color correction on pixels with color overflow separately, thereby avoiding the problem of unclear image content caused by color overflow, while avoiding affecting other normal pixels in the first image.

[0261] The secondary color correction process mainly includes the following steps: First, detecting overflow pixels, that is, detecting the first pixel with color overflow in the first image output after regular color correction. Second, calculating the color overflow value, that is, calculating the RGB color overflow value of the first pixel. Next, determining the color adjustment weight, that is, determining the first color adjustment weight based on the RGB color overflow value. Then, generating a new CCM, that is, generating a second CCM based on the first color adjustment weight. Finally, performing color correction using the new CCM, that is, using the second CCM to perform secondary color correction on the first pixel in the first image to obtain a second image including the second pixel.

[0262] Therefore, performing a color correction again to address pixel color overflow can prevent color overflow during subsequent dynamic range compression, thus ensuring that the image content corresponding to highly saturated objects is clearly visible.

[0263] In some embodiments, higher pixel values ​​for the R (red) and B (blue) color components of a pixel indicate a higher degree of color vividness. However, the more vivid the imaged object (the higher its saturation), the higher the probability of color overflow.

[0264] Therefore, electronic devices can further combine the RGB color ratio of the first pixel to generate a second CCM, thereby improving the correction effect for color overflow areas. The RGB color ratio refers to the ratio between the pixel values ​​corresponding to the R, G, and B color components of a pixel, i.e., the ratio of the pixel values ​​of the three components.

[0265] Figure 15 A flowchart illustrating the image capturing method provided in this application embodiment is shown, including steps S1501-S1507. By comparison... Figure 12 and Figure 15 As can be seen, in the embodiments of this application Figure 15 The provided image capture method is in Figure 12 Based on the image capture method shown, an additional processing step is added to generate a second CCM based on the RGB color ratio of the first pixel.

[0266] That is, in the embodiments of this application, the electronic device generates the second CCM corresponding to the first pixel by comprehensively considering the color overflow value and RGB color ratio of the first pixel.

[0267] Therefore, the specific implementations of S1501-S1503 and S1505-S1507 in the embodiments of this application can be referred to Figure 12 The corresponding embodiments are described. For example, referring to the description of S1201-S1203 and S1205-S1207 in the above embodiments, the principles are the same, and the embodiments of this application will not be repeated here.

[0268] Hereinafter, this application embodiment will describe in detail how the electronic device generates a second CCM based on the color overflow value and RGB color ratio of the first pixel in S1504.

[0269] Figure 16 This diagram illustrates the process of generating the second CCM based on the color overflow value and RGB color ratio of the first pixel. (Refer to...) Figure 16 This includes steps S1601-S1604.

[0270] S1601, the electronic device calculates the RGB color ratio of the first pixel.

[0271] Since the embodiments of this application mainly consider the influence of the R color component and the B color component on the vividness of colors, and since by selecting a reference value from the three values ​​and calculating its ratio with the remaining two values, the relationship between the remaining two values ​​can also be derived from the known ratio.

[0272] Therefore, in some embodiments, the RGB color ratios mainly include the RG ratio and the BG ratio. That is, with the G color component as the reference, the ratios of the R color component, B color component, and G color component are calculated separately. Thus, the RG ratio (R value / G value) is the ratio of the pixel values ​​corresponding to the R color component and the G color component, while the BG ratio (B value / G value) is the ratio of the pixel values ​​corresponding to the B color component and the G color component.

[0273] S1602, the electronic device determines the second color adjustment weight based on the RGB color ratio.

[0274] The second color adjustment weight is determined based on the RGB color ratio.

[0275] In some embodiments, the electronic device determines a second color adjustment weight based on the RGB color ratio, which may include: determining an RG adjustment weight based on the RG ratio and a BG adjustment weight based on the BG ratio; and using the product of the RG adjustment weight and the BG adjustment weight as the second color adjustment weight.

[0276] In other words, the electronic device first determines the corresponding adjustment weights based on the RG ratio and BG ratio, respectively, to obtain the RG adjustment weight and BG adjustment weight. Then, the electronic device combines the adjustment weights of the two color ratios and uses the product of the RG adjustment weight and BG adjustment weight as the second color adjustment weight.

[0277] Among them, the RG ratio rio RG Corresponding RG Adjustment Weight Weight rioRG and BG ratio rio BG Corresponding BG adjustment weight Weight rioBG The weights are set based on experience, and electronic devices can obtain the weight parameters by querying a preset weight table. Weight rioRG and Weight rioBG In other words, this weighting table is an empirical parameter table, and the specific settings can be configured according to actual business needs. Generally speaking, the greater the difference between the RG ratio and the BG ratio and 1, the greater the difference between the R, B, and G color components, and thus the higher the color vibrancy. Therefore, the weights should be adjusted accordingly. Weight rioRG and Weight rioBG It can be as big as possible.

[0278] In the embodiments of this application, the RG ratio rio RG Ratio to BG rioBG The closer the value is to 0.1 or 10, the higher the weighting parameter. Weight rioRG and Weight rioBG The closer it is to 1, the better. And the RG ratio... rio RG Ratio to BG rio BG The closer to 1, the higher the weight parameter. Weight rioRG and Weight rioBG The closer it is to 0.

[0279] In some embodiments, at the RG ratio rio RG When the value is greater than or equal to 10 or less than or equal to 0.1, RG adjusts its weights. Weight rioRG Equals 1. In the RG ratio rio RG When RG is greater than or equal to 1 and less than 10, the weights are adjusted. Weight rioRG It equals the ratio of the second difference to the value 9, where the second difference is the RG ratio. rio RG The difference from the value 1. In the RG ratio rio RG When RG is less than or equal to 1 and greater than 0.1, the weights are adjusted. Weight rioRG It equals the product of the value 10 / 9 and the third difference, which is the ratio of the value 1 to RG. rio RG difference.

[0280] The RG ratio can be calculated using the following formula. rio RG Corresponding RG Adjustment Weight Weight rioRG :

[0281]

[0282] Similarly, in the BG ratio rio BG When the value is greater than or equal to 10 or less than or equal to 0.1, the BG adjusts the weight. Weight rioBG Equals 1. (BG ratio) rio BG When the value is greater than or equal to 1 and less than 10, the weight of BG is adjusted. Weight rioBG It equals the ratio of the fourth difference to the value 9, where the fourth difference is the BG ratio. rio BG The difference from the value 1. In the BG ratio rio BG When the value is less than or equal to 1 and greater than 0.1, the BG adjusts the weight. Weight rioBG It equals the product of the value 10 / 9 and the fifth difference, where the fifth difference is the ratio of the value 1 to BG. rio BG difference.

[0283] The BG ratio can be calculated using the following formula. rio BG Corresponding BG adjustment weight Weight rioBG :

[0284]

[0285] Electronic devices obtain weight parameters Weight rioRG and Weight rioBG Next, the weight parameters Weight rioRG and Weight rioBG The product of these is used as the second color adjustment weight. Weight rio .Right now, Weight rio = Weight rioRG * Weight rioBG .

[0286] S1603, the electronic device uses the product of the first color adjustment weight and the second color adjustment weight as the new first color adjustment weight.

[0287] That is, after the electronic device obtains the second color adjustment weight determined according to the RGB color ratio, it combines the two color adjustment weights to further adjust the first color adjustment weight. Weight clip Second color adjustment weight Weight rio The product of these values ​​is used as the new first color adjustment weight. Weight total .Right now, Weight total = Weight clip * Weight rio .

[0288] S1604, the electronic device uses the sum of the first product and the second product as the second CCM. The first product is the product of the new first color adjustment weight and the standard identity matrix; the second product is the product of the first color correction matrix and the first difference, where the first difference is the difference between the value 1 and the second color adjustment weight.

[0289] Specifically, the second CCM can be calculated using the following formula:

[0290] CCM out = Weight total × I +( 1-Weight rio )× CCM in

[0291] in, CCM out It is the second CCM. CCM in It is the first CCM. Weight rio The first color is used to adjust the weight. Weight total The new first color has its weight adjusted. I It is a standard identity matrix. Finally, electronic devices can utilize the output of the embodiments of this application. CCM out Recalibrate the color of the first pixel.

[0292] Therefore, it can be seen that the embodiment of this application combines the RGB color ratio to generate the second CCM, which can further improve the color correction effect and ensure that the image content corresponding to the high-saturation imaging object is clearly distinguishable.

[0293] In some embodiments, discrete noise pixels may exist in the image. Therefore, to avoid the influence of discrete noise pixels, in this embodiment, after the electronic device detects the first pixel in the first image, it can further detect the first pixel that belongs to discrete noise pixels. If there is a first pixel that is a discrete noise pixel, then this first pixel is discarded.

[0294] Discrete noise pixels can be obtained through neighboring pixel detection. In the embodiments of this application, the neighboring pixels of the first pixel can be obtained in the form of 4-neighborhood, 8-neighborhood, or D-neighborhood, and this application does not limit this.

[0295] In some embodiments, determining that the first pixel is a discrete noise pixel based on its neighboring pixels may include: converting the first image into a binary image, where the pixel value of the first pixel is 1; dividing the first pixel and its neighboring pixels into pixel blocks in the binary image; if there are n pixels with a pixel value of 1 in a pixel block, the first pixel is not a discrete noise pixel; if there are no n pixels with a pixel value of 1 in a pixel block, the first pixel is a discrete noise pixel. Since the first pixel corresponds to a highly saturated imaging object, after binarization of the first image, the pixel value of the first pixel is 1, and it appears as white in the image.

[0296] For example, such as Figure 17 The diagram illustrates a first image corresponding to a binary image. (Reference) Figure 17 In the left image, the brightly colored red flower 1701 appears as white in the right binary image after binarization.

[0297] Then, the electronic device divides the first pixel and its neighboring pixels into pixel blocks in the binary image. For example, when the neighboring pixels of the first pixel are obtained in an 8-neighborhood manner, the pixel block formed by the first pixel and its neighboring pixels can be a 3x3 grid. Figure 18 The diagram illustrates a pixel block. Here, P represents the first pixel, and the eight surrounding pixels Q are the neighboring pixels of P.

[0298] Finally, the electronic device determines whether there are n pixels with a value of 1 in the pixel block. If there are n pixels with a value of 1 in the pixel block, it means that the first pixel is not a discrete noise pixel, and the first pixel is retained. If there are no n pixels with a value of 1 in the pixel block, it means that the first pixel is a discrete noise pixel, and the first pixel is discarded. Taking a pixel block containing 9 pixels as an example, that is, if there are fewer than n pixels with a value of 1 in the 9 pixels, the first pixel is discarded. If there are equal to or more than n pixels with a value of 1 in the 9 pixels, the first pixel is retained. Here, n is a number set based on practical experience and the situation of the neighboring pixels of the actual discrete noise, and can be configured according to specific circumstances. This application embodiment does not limit this.

[0299] Therefore, in this embodiment of the application, discarding the first pixel that belongs to discrete noise pixels based on neighboring pixels can avoid the influence of discrete noise pixels on color correction, thereby improving the effect of color correction.

[0300] Another embodiment of this application provides an electronic device, including: one or more processors and a memory. The memory is coupled to the processor; the memory stores one or more computer program codes, the computer program codes including computer instructions; when the processor executes the computer instructions, the electronic device implements the image capturing method described in any of the above embodiments.

[0301] Another embodiment of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor in an electronic device, causes the electronic device to implement the image capturing method described in any of the above embodiments.

[0302] This application also provides a computer program product that, when run on a computer, causes the computer to perform the various functions or steps described in the above method embodiments.

[0303] This application also provides a chip system, such as... Figure 19 As shown, the chip system 1900 includes at least one processor 1901 and at least one interface circuit 1902. The processor 1901 and the interface circuit 1902 are interconnected via lines. For example, the interface circuit 1902 can be used to receive signals from other devices (e.g., a computer's memory). As another example, the interface circuit 1902 can be used to send signals to other devices (e.g., the processor 1901).

[0304] For example, interface circuit 1902 can read instructions stored in memory and send those instructions to processor 1901. When the instructions are executed by processor 1901, the computer can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, and this application embodiment does not specifically limit this.

[0305] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0306] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0307] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0308] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0309] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0310] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image capturing method, characterized in that, Applied to an electronic device, the electronic device including a camera, the electronic device supporting high dynamic range (HDR) imaging mode, the method includes: The first interface is displayed; wherein, the first interface includes a preview image captured by the electronic device and the shutter speed. In response to the user's click operation on the shutter button, a first image is obtained in the HDR mode; wherein, the first image includes a first pixel with color overflow, and the first image is an image after color correction of the image captured by the camera through a first color correction matrix; wherein, the first color correction matrix is ​​a standard color correction matrix for color correction. Calculate the color overflow value of the first pixel; the color overflow value is the total pixel value that exceeds the pixel value range, and the pixel value range corresponds to the color depth of the HDR mode; Determine the ratio of the color overflow value to the maximum pixel value in the pixel value range; The first color adjustment weight of the first pixel is determined by multiplying the ratio by the intensity control threshold. The product of the first color adjustment weight and the standard identity matrix is ​​used as the second color correction matrix; Using each of the second color correction matrices, the corresponding first pixel in the first image is re-color corrected to obtain a second image; wherein, the second image includes a second pixel corresponding to the first pixel, and the second pixel is obtained by color correction based on the color overflow value of the first pixel.

2. The method according to claim 1, characterized in that, Determining the first color adjustment weight of the first pixel based on the product of the ratio and the intensity control threshold includes: If the product of the ratio and the intensity control threshold is less than or equal to 0, the first color adjustment weight is equal to 0; If the product of the ratio and the intensity control threshold is greater than or equal to 1, the first color adjustment weight is equal to 1; If the product of the ratio and the intensity control threshold is greater than 0 and less than 1, the first color adjustment weight is equal to the product of the ratio and the intensity control threshold.

3. The method according to claim 1 or 2, characterized in that, The calculation of the color overflow value of the first pixel includes: Calculate the absolute difference between the three component pixel values ​​of the first pixel and the pixel value range; wherein, the three component pixel values ​​are the pixel values ​​corresponding to the three RGB color components; The color overflow value of the first pixel is obtained by summing the absolute differences corresponding to the three component pixel values.

4. The method according to claim 1 or 2, characterized in that, The method further includes: calculating the RGB color ratio of the first pixel, and determining the second color adjustment weight corresponding to the first pixel based on the RGB color ratio; wherein, the RGB color ratio is the ratio between the pixel values ​​corresponding to the three RGB color components; The step of multiplying the first color adjustment weight by the standard identity matrix to obtain the second color correction matrix includes: The product of the second color adjustment weight and the first color adjustment weight is used as the new first color adjustment weight; The sum of the first product and the second product is used as the second color correction matrix; wherein, the first product is the product of the new first color adjustment weight and the standard identity matrix; the second product is the product of the first color correction matrix and the first difference, the first difference being the difference between the value 1 and the second color adjustment weight.

5. The method according to claim 4, characterized in that, The RGB color ratio includes the RG ratio and the BG ratio; the RG ratio is the ratio of the pixel values ​​corresponding to the R color component and the G color component, and the BG ratio is the ratio of the pixel values ​​corresponding to the B color component and the G color component. Determining the second color adjustment weight corresponding to the first pixel based on the RGB color ratio includes: The RG adjustment weight is determined based on the RG ratio corresponding to the first pixel, and the BG adjustment weight is determined based on the BG ratio corresponding to the first pixel. The product of the RG adjustment weight and the BG adjustment weight is used as the second color adjustment weight corresponding to the first pixel.

6. The method according to claim 5, characterized in that, The step of determining the RG adjustment weight based on the RG ratio corresponding to the first pixel includes: When the RG ratio is greater than or equal to 10 or less than or equal to 0.1, the RG adjustment weight is equal to 1; When the RG ratio is greater than or equal to 1 and less than 10, the RG adjustment weight is equal to the ratio of the second difference to the value 9; the second difference is the difference between the RG ratio and the value 1. When the RG ratio is less than or equal to 1 and greater than 0.1, the RG adjustment weight is equal to the product of the value 10 / 9 and the third difference; the third difference is the difference between the value 1 and the RG ratio.

7. The method according to claim 5 or 6, characterized in that, The step of determining the BG adjustment weight based on the BG ratio corresponding to the first pixel includes: When the BG ratio is greater than or equal to 10 or less than or equal to 0.1, the BG adjustment weight is equal to 1; When the BG ratio is greater than or equal to 1 and less than 10, the BG adjustment weight is equal to the ratio of the fourth difference to the value 9; the fourth difference is the difference between the BG ratio and the value 1. When the BG ratio is less than or equal to 1 and greater than 0.1, the BG adjustment weight is equal to the product of the value 10 / 9 and the fifth difference; the fifth difference is the difference between the value 1 and the BG ratio.

8. The method according to any one of claims 1-2, characterized in that, The method further includes detecting a first pixel in the first image that exhibits color overflow, including: The component pixel value of each pixel in the first image is compared with the pixel value range; wherein, the component pixel value includes the pixel value corresponding to the three color components of RGB; In the first image, the pixel whose component pixel value is not within the range of pixel values ​​is taken as the first pixel of color overflow.

9. The method according to any one of claims 1-2, characterized in that, The step of re-correcting the color of the corresponding first pixel in the first image using each of the second color correction matrices to obtain a second image including the second pixel includes: The second pixel is calculated using the following formula: R out = a 11 × R in + a 12 × G in + a 13 × B in G out = a 21 × R in + a 22 × G in + a 23 × B in B out = a 31 × R in + a 32 × G in + a 33 × B in in, R in , G in , B in It is the RGB value of the first pixel; R out , G out , B out It is the RGB value of the second pixel; a 11 , a 12 , a 13 , a 21 , a 22 , a 23 , a 31 , a 32 , a 32 It is an element in the second color correction matrix.

10. The method according to any one of claims 1-2, characterized in that, The method further includes: discarding the first pixel when it is determined that the first pixel is a discrete noise pixel based on the neighboring pixels of the first pixel.

11. The method according to claim 10, characterized in that, The step of determining that the first pixel is a discrete noise pixel based on the neighboring pixels of the first pixel includes: The first image is converted into a binary image, and the pixel value of the first pixel in the binary image is 1; In the binary image, the first pixel and the neighboring pixels are divided into pixel blocks according to a preset block size; If there are n pixels with a value of 1 in the pixel block, the first pixel is not a discrete noise pixel; If there are no n pixels with a value of 1 in the pixel block, the first pixel is a discrete noise pixel.

12. An image capturing method, characterized in that, Applied to an electronic device, the electronic device including a camera, the electronic device supporting high dynamic range (HDR) imaging mode, the method includes: Receive user commands to open the camera application; In response to the activation operation, a first image is obtained in the HDR mode; wherein the first image includes a first pixel with color overflow; the first image is an image obtained by color correction of the image captured by the camera using a first color correction matrix; wherein the first color correction matrix is ​​a standard color correction matrix for color correction. Calculate the color overflow value of the first pixel and calculate the RGB color ratio of the first pixel; wherein, the RGB color ratio is the ratio between the pixel values ​​corresponding to the three color components of RGB; A second color correction matrix corresponding to the first pixel is generated based on the color overflow value and the RGB color ratio value of the first pixel. Using each of the second color correction matrices, the corresponding first pixel in the first image is re-color corrected to obtain a second image including the second pixel. Display a preview interface including the second image; The second image includes a second pixel corresponding to the first pixel, and the second pixel is obtained by color correction based on the color overflow value of the first pixel; the color overflow value is the total pixel value that exceeds the pixel value range, and the pixel value range corresponds to the color depth of the HDR mode.

13. The method according to claim 12, characterized in that, The step of generating a second color correction matrix corresponding to the first pixel based on the color overflow value and the RGB color ratio value of the first pixel includes: Calculate the ratio of the color overflow value to the maximum pixel value in the pixel value range, and determine the first color adjustment weight of the first pixel based on the product of the ratio and the intensity control threshold; Calculate the second color adjustment weight of the first pixel based on the RGB color ratio corresponding to the first pixel; The product of the second color adjustment weight and the first color adjustment weight is used as the new first color adjustment weight; The sum of the first product and the second product is used as the second color correction matrix; wherein, the first product is the product of the new first color adjustment weight and the standard identity matrix; the second product is the product of the first color correction matrix and the first difference, the first difference being the difference between the value 1 and the second color adjustment weight.

14. An electronic device, characterized in that, include: One or more processors and a memory, the memory being coupled to the processor; the memory storing one or more computer program codes, the computer program codes including computer instructions; when the processor executes the computer instructions, the electronic device causes the electronic device to perform the image capturing method as described in any one of claims 1-11.

15. An electronic device, characterized in that, include: One or more processors and a memory, the memory being coupled to the processor; the memory storing one or more computer program codes, the computer program codes including computer instructions; when the processor executes the computer instructions, the electronic device causes the electronic device to perform the image capturing method as described in any one of claims 12-13.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor of the electronic device, the electronic device performs the image capturing method as described in any one of claims 1-11.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor of the electronic device, the electronic device performs the image capturing method as described in any one of claims 12-13.

Images