Image generation method, control device, electronic device, vehicle, and storage medium

Abstract

The invention discloses an image generation method, a control device, an electronic device, a vehicle and a computer readable storage medium, and the method comprises the steps: obtaining a plurality of second images corresponding to a first image, determining the first weight of the second image according to the exposure of the second image, and performing synthesis processing on the plurality of second images according to the first weight of each second image and the pixel value of the pixel point in each second image, determining a third image, and generating a target image according to the first image and the third image. Therefore, the first weights of the second images can be determined according to the exposure quantities of the second images, and the plurality of second images are synthesized into the third image, so that information such as image details and light and shade relationships of the plurality of second images can be reserved in the third image; and supplementing information such as image details and light and shade relationships in the third image to the first image to generate a target image, thereby ensuring that the target image is matched with the visual habits of the user.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image generating method, a control device, an electronic device, a vehicle and a computer-readable storage medium. Background Art

[0002] HDR (High Dynamic Range) has the advantages of large image information content and rich light and shadow details, which can accurately reflect the real environment. Therefore, some vehicles are equipped with the function of generating and displaying HDR images so that drivers can accurately know the surrounding road conditions through HDR images.

[0003] However, when the dynamic range supported by the in-vehicle display device is low, the in-vehicle display device cannot display HDR normally, so in related technologies, the dynamic range of the HDR image is usually compressed before display. However, when compressing the dynamic range of the HDR image, details in the HDR image may be lost, the contrast between light and dark may be distorted, etc., making the HDR image inconsistent with the user's visual habits and easily causing a strong sense of disharmony. Summary of the invention

[0004] The present application provides an image generating method, a control device, an electronic device, a vehicle and a computer-readable storage medium.

[0005] The present application provides an image generation method, including:

[0006] Acquire a plurality of second images corresponding to the first image, wherein the first image is determined by the plurality of second images, and the plurality of second images correspond to a plurality of exposure values;

[0007] determining a first weight of the second image according to an exposure amount of the second image;

[0008] Performing synthesis processing on the plurality of second images according to the first weight of each second image and the pixel value of a pixel point in each second image to determine a third image;

[0009] A target image is generated according to the first image and the third image.

[0010] In the image generation method provided in the embodiment of the present application, multiple second images corresponding to the first image can be obtained, and the first weight of the second image can be determined based on the exposure of the second image. Then, based on the first weight of each second image and the pixel value of the pixel point in each second image, the multiple second images are synthesized to determine the third image, and the target image is generated based on the first image and the third image.

[0011] Thus, in the implementation mode of the present application, the first weight of each second image can be determined according to the exposure amount of each second image, and the multiple second images can be synthesized into a third image according to the first weight of each second image and the pixel value of the pixel point in each second image, so that the image details, light and dark relationship and other information of the multiple second images can be retained in the third image, and then, the image details, light and dark relationship and other information in the third image can be supplemented to the first image to generate a target image, thereby ensuring to a certain extent that the target image matches the user's visual habits.

[0012] In certain embodiments of the present application, the method further comprises:

[0013] synthesizing the plurality of second images according to the pixel value of each pixel in each of the second images to obtain a fourth image, wherein a dynamic range of the fourth image is higher than a dynamic range of the second image;

[0014] According to the predetermined compression curve parameters and tone mapping function, tone mapping processing is performed on the fourth image to obtain the first image.

[0015] Thus, in an embodiment of the present application, pixel-level synthesis processing can be performed on multiple second images through the pixel value of each pixel in each second image to obtain a fourth image with a higher dynamic range than the second image, and then tone mapping processing can be performed on the fourth image according to predetermined compression curve parameters and tone mapping function to obtain the first image.

[0016] In some embodiments of the present application, the synthesizing process of the plurality of second images according to the pixel value of each pixel point in each of the second images to obtain the fourth image includes:

[0017] Determining a second weight for each pixel in the second image according to a pixel value of each pixel in the second image;

[0018] Determining a maximum image exposure according to the exposure of each of the second images;

[0019] determining an exposure weight of the second image according to a ratio of the exposure of the second image to the maximum exposure of the image;

[0020] The fourth image is obtained by synthesizing the plurality of second images according to the pixel value of each pixel in the second image, the second weight of each pixel in the second image, and a predetermined first image synthesis function.

[0021] Thus, in the implementation mode of the present application, the second weight of each pixel in the second image can be determined according to the pixel value of each pixel in the second image, and the maximum exposure of each second image can be determined according to the exposure of each second image, and then the exposure proportion of the second image can be determined according to the ratio of the exposure of the second image to the maximum exposure of the image, and based on the pixel value of the pixel in each second image, the second weight of the pixel in each second image and the predetermined first image synthesis function, the multiple second images are synthesized at the pixel level to obtain a fourth image.

[0022] In some embodiments of the present application, determining the second weight of each pixel in the second image according to the pixel value of each pixel in the second image includes:

[0023] When the pixel value of a pixel point in the second image is less than a predetermined first preset value, determining the second weight of the pixel point according to the pixel value of the pixel point; and / or

[0024] When the pixel value of a pixel point in the second image is greater than or equal to the first preset value, the second weight of the pixel point is determined according to a difference between a predetermined second preset value and the pixel value of the pixel point.

[0025] Thus, in an embodiment of the present application, when the pixel value of the pixel point in the second image is less than a predetermined first preset value, the second weight of the pixel point can be determined according to the pixel value of the pixel point, and / or when the pixel value of the pixel point in the second image is greater than or equal to the first preset value, the second weight of the pixel point can be determined according to the difference between the predetermined second preset value and the pixel value of the pixel point.

[0026] In some embodiments of the present application, the acquiring of a plurality of second images corresponding to the first image includes:

[0027] acquiring a plurality of fifth images with the same exposure;

[0028] determining a sixth image corresponding to the second image among the plurality of third images according to the exposure amount of the fifth image and the predetermined exposure amount of the second image;

[0029] The plurality of second images are determined according to the sixth image corresponding to each of the second images.

[0030] Thus, in an embodiment of the present application, when a plurality of fifth images with the same exposure amount are acquired, a fourth image corresponding to the second image among the plurality of third images can be determined according to the exposure amount of the third image and the exposure amount of the predetermined second image, and a plurality of second images can be determined according to the fourth image corresponding to each second image, so that the acquisition of the second image can be achieved through the plurality of fifth images with the same exposure amount, thereby ensuring the robust acquisition of the second image to a certain extent.

[0031] In some embodiments of the present application, determining a sixth image corresponding to the second image among the plurality of third images according to the exposure amount of the fifth image and the predetermined exposure amount of the second image includes:

[0032] determining a required number of fifth images corresponding to the second image according to a ratio of an exposure amount of the second image to an exposure amount of the fifth image;

[0033] The sixth image corresponding to the second image in the plurality of third images is determined according to the required number of the fifth images corresponding to the second image.

[0034] Thus, in an embodiment of the present application, the required number of fifth images corresponding to the second image can be determined based on the exposure amount of the third image and the predetermined exposure amount of the second image, and the sixth image corresponding to the second image in multiple third images can be determined based on the required number of fifth images corresponding to the second image.

[0035] In some embodiments of the present application, the plurality of fifth images are a plurality of frames of images taken continuously, and determining the sixth image corresponding to the second image in the plurality of third images according to the required number of the fifth images corresponding to the second image includes:

[0036] Determining a starting frame image and an ending frame image among the plurality of fifth images according to a required number of fifth images corresponding to the second image;

[0037] The sixth image corresponding to the second image is determined according to all images from the start frame image to the end frame image in the plurality of third images.

[0038] Thus, in the implementation mode of the present application, the starting frame image and the ending frame image in the multiple fifth images taken continuously can be determined according to the required number of fifth images corresponding to the second image, and then the multiple consecutive fifth images from the starting frame image to the ending frame image can be determined as the sixth image corresponding to the second image, thereby ensuring the reliable determination of the sixth image to a certain extent.

[0039] In some embodiments of the present application, determining the plurality of second images according to the sixth image corresponding to each of the second images includes:

[0040] In the case that there are multiple sixth images corresponding to the second image, the multiple sixth images are synthesized according to the pixel values ​​of the pixels in each of the sixth images and a predetermined second image synthesis function to determine the second image.

[0041] Thus, in the embodiment of the present application, when there are multiple sixth images corresponding to the second image, the multiple sixth images may be synthesized at the pixel level according to a predetermined second image synthesis function to determine the second image.

[0042] In some embodiments of the present application, determining the plurality of second images according to the sixth image corresponding to each second image includes:

[0043] When the sixth image corresponding to the second image is one image, the sixth image is determined as the second image.

[0044] Thus, in the embodiment of the present application, when the sixth image corresponding to the second image is one, the sixth image corresponding to the second image may be determined as the second image.

[0045] In some embodiments of the present application, the acquiring of a plurality of second images corresponding to the first image includes:

[0046] Acquire a sixth image, wherein a dynamic range of the sixth image is higher than a dynamic range of the second image;

[0047] The sixth image is decomposed according to the pixel values ​​of the pixels in the sixth image, the predetermined image coefficients of the second image and the predetermined image decomposition function to determine the second image.

[0048] Thus, in the embodiment of the present application, when a sixth image with a higher dynamic range is acquired, the sixth image may be decomposed according to predetermined image coefficients of each second image to determine a plurality of second images with a lower dynamic range.

[0049] In some embodiments of the present application, determining the first weight of the second image according to the exposure amount of the second image includes:

[0050] Determine the maximum brightness of a pixel of the second image according to the brightness of each pixel in the second image;

[0051] The first weight of the second image is determined according to a maximum brightness value of pixels of the second image.

[0052] Thus, in an embodiment of the present application, the maximum brightness of the pixels of the second image can be determined according to the brightness of each pixel in the second image, and the first weight of the second image can be determined according to the maximum brightness of the pixels of the second image.

[0053] In some embodiments of the present application, determining the first weight of the second image according to the maximum brightness of the pixels of the second image includes:

[0054] The first weight of the second image is determined according to the maximum brightness value of the pixel points of the second image and a predetermined piecewise function, wherein the piecewise function is used to associate brightness with weight.

[0055] Thus, in the embodiment of the present application, the first weight of the second image is determined according to the maximum brightness value of the pixel points of the second image and the piecewise function.

[0056] In some embodiments of the present application, the synthesizing process of the plurality of second images according to the first weight of each second image and the pixel value of a pixel point in each second image to determine the third image includes:

[0057] The plurality of second images are synthesized according to the first weight of each second image, the pixel value of each pixel in each second image, and a predetermined third image synthesis function to determine the third image.

[0058] Thus, in an embodiment of the present application, the electronic device can obtain a third image by performing weighted summation on each second image at the pixel level using the first weight of each second image, the pixel value of each pixel in each second image, and a predetermined fourth image synthesis function.

[0059] In some embodiments of the present application, generating a target image according to the first image and the third image includes:

[0060] The first image and the third image are synthesized according to the pixel value of each pixel in the first image, the predetermined weight of the first image, the pixel value of each pixel in the third image, the predetermined weight of the third image and a predetermined fourth image synthesis function to generate the target image.

[0061] Thus, in an embodiment of the present application, the first image and the third image can be synthesized at the pixel level to generate a target image through a fourth image synthesis function, the pixel value of each pixel in the first image, the predetermined weight of the first image, the pixel value of each pixel in the third image, and the predetermined weight of the third image.

[0062] The embodiment of the present application provides a control device, including a transceiver unit and a processing unit;

[0063] The transceiver unit is configured to obtain a plurality of second images corresponding to the first image, wherein the first image is determined by the plurality of second images, and the plurality of second images correspond to a plurality of exposure amounts;

[0064] The processing unit is configured to determine a first weight of the second image according to the exposure of the second image, and to synthesize the multiple second images according to the first weight of each second image and the pixel value of each pixel in the second image to determine a third image, and to generate a target image according to the first image and the third image.

[0065] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned image generation method is implemented.

[0066] The present application provides a vehicle including the above-mentioned control device or the above-mentioned electronic device.

[0067] An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by one or more processors, the above-mentioned image generation method is implemented.

[0068] The control device, electronic device, vehicle and computer-readable storage medium provided in the embodiments of the present application can determine the first weight of each second image according to the exposure amount of each second image, and synthesize multiple second images into a third image according to the first weight of each second image and the pixel value of the pixel point in each second image, so that the image details, light and dark relationship and other information of the multiple second images can be retained in the third image, and then, the image details, light and dark relationship and other information in the third image can be supplemented to the first image to generate a target image, thereby ensuring that the target image matches the user's visual habits to a certain extent.

[0069] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0071] Figure 1 Schematic diagram of global tone mapping;

[0072] Figure 2 Schematic diagram of local tone mapping;

[0073] Figure 3 A schematic diagram of a flow chart of an image generation method in certain embodiments of the present application;

[0074] Figure 4 A schematic diagram of a first image in some embodiments of the present application;

[0075] Figure 5 A schematic diagram of a second image in some embodiments of the present application;

[0076] Figure 6 A schematic diagram of a second image in some embodiments of the present application;

[0077] Figure 7 A schematic diagram of a second image in some embodiments of the present application;

[0078] Figure 8 A schematic diagram of a second image in some embodiments of the present application;

[0079] Fig. 9 A schematic diagram of application scenarios in certain embodiments of the present application;

[0080] Fig.10 A schematic diagram of a third image in some embodiments of the present application;

[0081] Fig.11 A schematic diagram of a target image in some embodiments of the present application;

[0082] Fig.12 A schematic diagram of an image after dynamic range compression in some embodiments of the present application;

[0083] Fig.13 A schematic diagram of application scenarios in certain embodiments of the present application;

[0084] Fig.14 A schematic diagram of a flow chart of an image generation method in certain embodiments of the present application;

[0085] Fig.15 A schematic diagram of a flow chart of an image generation method in certain embodiments of the present application;

[0086] Fig.16A schematic diagram of a flow chart of an image generation method in certain embodiments of the present application;

[0087] Fig.17 A schematic diagram of a flow chart of an image generation method in certain embodiments of the present application;

[0088] Fig.18 A schematic diagram of a flow chart of an image generation method in certain embodiments of the present application;

[0089] Fig.19 A schematic diagram of application scenarios in certain embodiments of the present application;

[0090] Fig. 20 A schematic diagram of a flow chart of an image generation method in certain embodiments of the present application;

[0091] Fig.21 A schematic diagram of application scenarios in certain embodiments of the present application;

[0092] Fig. 22 A schematic diagram of a flow chart of an image generation method in certain embodiments of the present application;

[0093] Fig.23 A schematic diagram of a piecewise function in certain embodiments of the present application;

[0094] Fig.24 A schematic diagram of application scenarios in certain embodiments of the present application;

[0095] Fig.25 Schematic diagram of the flow of the image generation method in certain embodiments of the present application. DETAILED DESCRIPTION

[0096] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0097] In the related art, HDR (High Dynamic Range) images are usually synthesized by multiple frames of LDR (Low Dynamic Range) images with different exposures. However, when the dynamic range supported by the image display device is low (such as the maximum display brightness of a cathode ray tube display of about 80cd / m 2 , the actual dynamic range is usually lower than 100:1), and the image display device may not be able to display the HDR image normally, such as obvious distortion of brightness, texture and color.

[0098] Therefore, when the dynamic range supported by the image display device is low, the HDR image can usually be tone mapped with the help of a preset tone mapping function to compress the dynamic range of the HDR image. According to the scope of the tone mapping function, tone mapping can be divided into global tone mapping and local tone mapping.

[0099] Specifically, see Figure 1 and Figure 2 , Figure 1 is a schematic diagram of global tone mapping. Figure 2 is a schematic diagram of local tone mapping. More specifically, Figure 1 As shown, for the image to be tone mapped (or the HDR image to be tone mapped), a single mapping function can be used to achieve compression mapping of the entire HDR image to the LDR image. It can be understood that the advantages of this method are simple implementation, low resource consumption, and a good overall light-dark relationship for the image, but the details of the highlight part are seriously lost.

[0100] like Figure 2 As shown, for the image to be tone mapped (or the HDR image to be tone mapped), the image can be divided into multiple small blocks, and each block of the image can be compressed and mapped using different mapping functions, and then the block effect can be eliminated by using methods such as bilinear interpolation, and finally a LDR image with rich details can be obtained. It can be understood that the advantage of this method is that the details of each part of the image can be well preserved, but the implementation steps are more complicated and consume more resources, so it is difficult to apply in real-time HDR synthesis scenarios. At the same time, because local tone mapping overemphasizes the retention of details of each part, the contrast of the light and dark relationship between the image blocks is weakened, which does not conform to people's visual habits and may make users feel a strong sense of disobedience.

[0101] In addition, it is understandable that when the HDR image is compressed and mapped through the tone mapping function, since the tone mapping function mainly focuses on the compression mapping of the HDR image to the LDR image, it is difficult to pay attention to the multiple frames of LDR images with different exposures used when synthesizing the HDR image, resulting in the details of the multiple frames of LDR images with different exposures that can be normally displayed on the low dynamic range display device being ignored.

[0102] Furthermore, when the exposure of the LDR image is high, the LDR image can show the details of the shadow part, but the rest of the part outside the shadow part is usually overexposed. When the exposure of the LDR image is small, the LDR image can show the details of the highlight part, but the rest of the part is usually dark. Therefore, if multiple frames of LDR images with different exposures used to synthesize HDR images are mixed, a synthetic image can be obtained in which the details of the low-light part and the highlights can be well preserved, and the light-dark relationship of the image is appropriate, but details will be lost in some dynamic range transition areas.

[0103] Based on the above problems you may encounter, please refer to Figure 3 , the present application embodiment provides an image generation method, including:

[0104] 01: Acquire multiple second images corresponding to the first image, wherein the first image is determined by the multiple second images, and the multiple second images correspond to multiple exposure values;

[0105] 02: determining a first weight of the second image according to the exposure amount of the second image;

[0106] 03: synthesizing the plurality of second images according to the first weight of each second image and the pixel value of a pixel point in each second image to determine a third image;

[0107] 04: Generate a target image according to the first image and the third image.

[0108] The embodiment of the present application provides a control device. The image generation method of the embodiment of the present application can be implemented by the control device of the embodiment of the present application. Specifically, the control device includes a transceiver unit and a processing unit. The transceiver unit is configured to obtain multiple second images corresponding to the first image, the first image is determined by the multiple second images, and the multiple second images correspond to multiple exposure amounts. The processing unit is configured to determine the first weight of the second image according to the exposure amount of the second image, and synthesize the multiple second images according to the first weight of each second image and the pixel value of the pixel point in each second image, determine the third image, and generate the target image according to the first image and the third image.

[0109] The embodiment of the present application also provides an electronic device, which includes a memory and a processor. The image generation method of the embodiment of the present application can be implemented by the electronic device of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to obtain multiple second images corresponding to the first image, and to determine the first weight of the second image according to the exposure of the second image, and to synthesize the multiple second images according to the first weight of each second image and the pixel value of the pixel point in each second image to determine the third image, and to generate a target image according to the first image and the third image, wherein the first image is determined by the multiple second images, and the multiple second images correspond to multiple exposures.

[0110] Specifically, in an embodiment of the present application, an electronic device (or a vehicle, or a control device) can obtain multiple second images corresponding to the first image and corresponding to multiple exposure amounts, and determine the second weight of each second image through the exposure amount of each second image, and then synthesize the multiple second images into a second image based on the first weight of each second image and the pixel value of the pixel point in each second image, and synthesize the first image and the third image to generate a final target image.

[0111] It is understandable that in the implementation manner of the present application, both the first image and the second image can be set according to actual conditions. For example, in one example, the first image refers to an HDR (High Dynamic Range) image obtained by taking multiple second images with different exposures, and the second image refers to multiple LDR (Low Dynamic Range) images with different exposures.

[0112] For example, in another example, the first image refers to an image after tone mapping of "an HDR image obtained by multiple second images with different exposures", and the second image refers to multiple LDR images with different exposures. Figure 4 , Figure 5 , Figure 6 , Figure 7 and Figure 8 , Figure 4 This is a schematic diagram of a first image in some embodiments of the present application. Figures 5 to 8 are schematic diagrams of the second image in certain embodiments of the present application. Specifically, the second image may refer to Figures 5 to 8 Four LDR images with decreasing exposure are shown.

[0113] Furthermore, the first image may be Figure 4 The image shown, that is, through Figures 5 to 8 After generating an HDR image from four LDR images with successively lower exposures, tone mapping is performed on the HDR image to obtain the following: Figure 4 The first image is shown.

[0114] It can also be understood that in the implementation manner of the present application, the specific method for obtaining the second image can be set according to actual conditions. Fig. 9 , Fig. 9 Schematic diagram of application scenarios in certain embodiments of the present application. That is, in the embodiments of the present application, the electronic device can control sensors such as staggered HDR, DCG (Dual Conversion Gain) HDR, Split-diode pixel HDR, etc. to perform a shooting action for a target shooting object, so that the sensor directly outputs multiple frames of different exposure LDR images, that is, multiple frames of second images. In addition, it can be understood that in Fig. 9 In , F represents the number of second images.

[0115] Furthermore, if Figures 5 to 8 Therefore, under different exposure conditions, the clearly visible parts of the image are also different, or in other words, the image details and brightness attributes of each LDR image are different. The image details and brightness attributes in an LDR image with a lower exposure may not be presented in an LDR image with a lower exposure.

[0116] Therefore, in order to retain the image details and brightness attributes of each LDR image, in the implementation manner of the present application, the electronic device can determine the first weight of each LDR image through the exposure amount of each LDR image, and then synthesize the various LDR images into a third image through the pixel values ​​of the pixel points in each LDR image, so that the image details and brightness attributes in each LDR image can be retained in the third image.

[0117] In one example, the acquisition process of the third image can be expressed by formula (1), that is:

[0118]

[0119] In formula (1), LDR_Blend(x,y) represents the pixel value of the pixel at (x,y) in the third image, Weight(f) represents the first weight of the second image of the f-th second image, LDR(x,y,f) represents the pixel value of the pixel at (x,y) in the f-th second image, and Weight(f) represents the first weight of the second image.

[0120] In an example, see Fig.10 , Fig.10 This is a schematic diagram of the third image in some embodiments of the present application. It can be understood that Fig.10 is through Figures 5 to 8 The four LDR images shown in FIG. 1 are synthesized with successively lower exposures, so that the low-light details and highlight details of the four LDR images are all in Fig.10 is better preserved, and Fig.10 The image shown has an appropriate relationship between light and dark, but there is a loss of detail in some areas of dynamic range transition.

[0121] In addition, it can also be understood that when the third image is generated by multiple second images so that the image details and light and dark properties of the multiple second images can be reflected by the third image, the electronic device can supplement the image details and light and dark properties in the first image through the third image to generate an image with image details preserved, a sufficiently obvious light and dark relationship, and a sufficiently natural dynamic range transition area, that is, the target image in the embodiment of the present application. Alternatively, the electronic device can generate an image with image details preserved, a sufficiently obvious light and dark relationship, and a sufficiently natural dynamic range transition area through the first image and the third image, that is, the target image in the embodiment of the present application.

[0122] To more clearly illustrate the implementation of this application, please refer to Fig.11 and Fig.12 , Fig.11 is a schematic diagram of a target image in some embodiments of the present application, Fig.12 This is a schematic diagram of an image after dynamic range compression in some embodiments of the present application. It should be noted that: Fig.11 is through Figure 4 and Fig.10 The image obtained, Fig.12 To pass Figures 5 to 8 After generating an HDR image from the four LDR images with decreasing exposure, the tonemap function in MATLAB is called to perform tone mapping on the HDR image to achieve a dynamic range compressed image. Fig.12 and Fig.10 All are "through Figures 5 to 8 After generating an HDR image from four LDR images with successively lower exposures, tone mapping is performed on the HDR image to obtain an image", but Fig.12 and Fig.10 Different methods are used for tone mapping.

[0123] Furthermore, if Figure 4 , Fig.10 and Fig.11 As shown, Figure 4 (i.e. the first image) missing details (such as the hole in the lamp) can be Fig.10 (i.e. the third image) complements, so in the target image (i.e. Fig.11 ) was observed.

[0124] Furthermore, if Fig.11 and Fig.12 , Fig.12 Although it can be well preserved Figures 5 to 8 However, the overall image contrast is seriously lost, and the brightness of each area of ​​the image is consistent, which does not conform to human visual habits. Fig.11 Keep Figures 5 to 8 While maintaining the local details in the image, it also achieves a natural sense of brightness in each area of ​​the image, avoiding Fig.12 In the case of "the brightness of each area of ​​the image is high", the specific example is Fig.11 The area framed by the white line and Fig.12 As shown in the area framed by white lines.

[0125] In summary, in the implementation manner of the present application, the first weight of each second image can be determined according to the exposure amount of each second image, and multiple second images can be synthesized into a third image according to the first weight of each second image and the pixel value of the pixel point in each second image, so that the image details, light and dark relationship and other information of the multiple second images can be retained in the third image, and then, the image details, light and dark relationship and other information in the third image can be supplemented to the first image to generate a target image, thereby ensuring that the target image matches the user's visual habits to a certain extent.

[0126] In addition, to more clearly illustrate the various images in the embodiments of this application, please refer to Fig.13 , Fig.13 Schematic diagram of application scenarios in certain embodiments of the present application. That is, in the embodiments of the present application, an image or a frame of an image can be understood as Fig.13 The Bayer diagram of size C*R is shown. Wherein, C represents the width of the image, and R represents the height of the image. And, in the implementation mode of the present application, Bayer(x,y) represents the pixel value of the pixel point located at the (x,y) coordinate in the image. In addition, for the convenience of understanding later, let the number of second images in the implementation mode of the present application be F, so Bayer(x,y,f) represents the pixel value of the pixel point located at the (x,y) coordinate in the f-th second image among the F second images. Correspondingly, in the implementation mode of the present application, the value range of x is [1, C], the value range of y is [1, R], and the value range of f is [1, F].

[0127] See also Fig.14 In some embodiments of the present application, the image generation method further includes:

[0128] 05: synthesizing the plurality of second images according to the pixel value of each pixel point in each second image to obtain a fourth image, wherein a dynamic range of the fourth image is higher than a dynamic range of the second image;

[0129] 06: Perform tone mapping processing on the fourth image according to the predetermined compression curve parameters and tone mapping function to obtain the first image.

[0130] The processing unit of the embodiment of the present application is configured to synthesize multiple second images according to the pixel value of each pixel in each second image to obtain a fourth image, and to perform tone mapping on the fourth image according to predetermined compression curve parameters and tone mapping function to obtain the first image, and the dynamic range of the fourth image is higher than the dynamic range of the second image.

[0131] The processor of the embodiment of the present application is also used to synthesize multiple second images according to the pixel value of each pixel in each second image to obtain a fourth image, and to perform tone mapping on the fourth image according to predetermined compression curve parameters and tone mapping function to obtain the first image. The dynamic range of the fourth image is higher than the dynamic range of the second image.

[0132] Specifically, in the embodiment of the present application, the determination process of the first image may include: performing pixel-level synthesis processing on multiple second images according to the pixel value of each pixel point in each second image, so as to obtain an image with a higher dynamic range, that is, the fourth image. Then, according to the predetermined compression curve parameters and tone mapping function, tone mapping processing is performed on the fourth image to compress the dynamic range of the four images, so as to obtain the first image.

[0133] The process of tone mapping the fourth image can be expressed by formulas (2), (3) and (4), namely:

[0134]

[0135] K=ER(F) (3)

[0136] L max =max(HDRmap(x,y)) (4)

[0137] Wherein, formula (2) is the tone mapping function, ToneMap(x, y) represents the pixel value of the pixel at (x, y) in the first image, HDRMap(x, y) represents the pixel value of the pixel at (x, y) in the fourth image, K and L max are all pre-determined compression curve parameters,

[0138] Thus, in an embodiment of the present application, pixel-level synthesis processing can be performed on multiple second images through the pixel value of each pixel point in each second image to obtain a fourth image with a higher dynamic range than the second image, and then tone mapping processing can be performed on the fourth image according to predetermined compression curve parameters and tone mapping function to obtain the first image.

[0139] See also Fig.15 In certain embodiments of the present application, step 05 includes:

[0140] 050: Determine a second weight of each pixel in the second image according to a pixel value of each pixel in the second image;

[0141] 051: Determine the maximum exposure of the image according to the exposure of each second image;

[0142] 052: determining the exposure weight of the second image according to the ratio of the exposure of the second image to the maximum exposure of the image;

[0143] 053: Perform synthesis processing on the plurality of second images according to the pixel value of each pixel in the second image, the second weight of each pixel in the second image, and a predetermined first image synthesis function to obtain a fourth image.

[0144] The processing unit of the embodiment of the present application is configured to determine the second weight of each pixel in the second image according to the pixel value of each pixel in the second image, determine the maximum exposure of the image according to the exposure of each second image, determine the exposure proportion of the second image according to the ratio of the exposure of the second image to the maximum exposure of the image, and synthesize the multiple second images according to the pixel value of the pixel in each second image, the second weight of the pixel in each second image and a predetermined first image synthesis function to obtain a fourth image.

[0145] The processor of the embodiment of the present application is also used to determine the second weight of each pixel in the second image according to the pixel value of each pixel in the second image, and determine the maximum exposure of the image according to the exposure of each second image, and determine the exposure proportion of the second image according to the ratio of the exposure of the second image to the maximum exposure of the image, and synthesize multiple second images according to the pixel value of the pixel in each second image, the second weight of the pixel in each second image and a predetermined first image synthesis function to obtain a fourth image.

[0146] Specifically, in the embodiment of the present application, the electronic device can determine the second weight of each pixel in the second image according to the pixel value of each pixel in the second image. At the same time, according to the exposure of each second image, the maximum value of the exposure of each second image, that is, the maximum exposure of the image, is determined. And, according to the ratio of the exposure of the second image to the maximum exposure of the image, the exposure proportion of the second image is determined. Finally, according to the pixel value of the pixel in each second image, the second weight of the pixel in each second image, and the predetermined first image synthesis function, the plurality of second images are synthesized at the pixel level to obtain a fourth image.

[0147] In one example, since the product of the LDR image (i.e., the second image) and the corresponding exposure ratio can represent the real brightness of the current image scene, the maximum exposure of an image Exposure(1) can be determined according to the exposure of each LDR image, and the ratio ER between the maximum exposure of the image Exposure(1) and the exposure of each frame of the LDR image can be determined, as shown in formula (5), that is:

[0148] ER(f)=Exposure(1) / Exposure(f) (5)

[0149] In formula (5), ER(f) represents the ratio of the maximum image exposure Exposure(1) to the exposure Exposure(f) of the f-th second image.

[0150] In one example, the process of acquiring the fourth image may be as shown in formula (6), that is:

[0151]

[0152] In formula (6), HDR(x,y) represents the pixel value of the pixel at (x,y) in the fourth image, W(x,y,f) represents the second weight of the pixel at (x,y) in the f-th second image, and LDR(x,y,f) represents the pixel value of the pixel at (x,y) in the f-th second image.

[0153] Thus, in the implementation mode of the present application, the second weight of each pixel in the second image can be determined according to the pixel value of each pixel in the second image, and the maximum exposure of each second image can be determined according to the exposure of each second image, and then the exposure proportion of the second image can be determined according to the ratio of the exposure of the second image to the maximum exposure of the image, and based on the pixel value of the pixel in each second image, the second weight of the pixel in each second image and the predetermined first image synthesis function, the multiple second images are synthesized at the pixel level to obtain a fourth image.

[0154] In some embodiments of the present application, step 050 includes:

[0155] When the pixel value of a pixel point in the second image is less than a predetermined first preset value, determining a second weight of the pixel point according to the pixel value of the pixel point; and / or

[0156] When the pixel value of a pixel point in the second image is greater than or equal to the first preset value, a second weight of the pixel point is determined according to a difference between the predetermined second preset value and the pixel value of the pixel point.

[0157] The processing unit of the embodiment of the present application is configured to determine the second weight of the pixel point according to the pixel value of the pixel point when the pixel value of the pixel point in the second image is less than the predetermined first preset value, and / or determine the second weight of the pixel point according to the difference between the predetermined second preset value and the pixel value of the pixel point when the pixel value of the pixel point in the second image is greater than or equal to the first preset value.

[0158] The processor of the embodiment of the present application is also used to determine the second weight of the pixel point according to the pixel value of the pixel point when the pixel value of the pixel point in the second image is less than the predetermined first preset value. And / or when the pixel value of the pixel point in the second image is greater than or equal to the first preset value, determine the second weight of the pixel point according to the difference between the predetermined second preset value and the pixel value of the pixel point.

[0159] It can be understood that in an LDR image, the closer the pixel value of a pixel is to 128, the greater the credibility of the actual brightness value of the scene represented by the pixel. Conversely, if the pixel value of a pixel is closer to 0 or 255, the credibility of the actual brightness value of the scene represented by the pixel is smaller. Therefore, in an example, the first preset value is 128, the second preset value is 255, and the second weight can be obtained by formula (7), that is:

[0160]

[0161] In formula (7), W(x, y, f) represents the second weight of the pixel at (x, y) in the second image, and LDR(x, y) represents the pixel value of the pixel at (x, y) in the second image.

[0162] Thus, in an embodiment of the present application, when the pixel value of the pixel point in the second image is less than a predetermined first preset value, the second weight of the pixel point can be determined according to the pixel value of the pixel point, and / or when the pixel value of the pixel point in the second image is greater than or equal to the first preset value, the second weight of the pixel point can be determined according to the difference between the predetermined second preset value and the pixel value of the pixel point.

[0163] See also Fig.16 In certain embodiments of the present application, step 01 includes:

[0164] 010: Acquire multiple fifth images with the same exposure;

[0165] 011: determining a sixth image corresponding to the second image among the plurality of third images according to the exposure amount of the fifth image and the predetermined exposure amount of the second image;

[0166] 012: Determine a plurality of second images according to a sixth image corresponding to each second image.

[0167] The transceiver unit of the embodiment of the present application is configured to obtain a plurality of fifth images with the same exposure. The processing unit of the embodiment of the present application is configured to determine a sixth image corresponding to the second image among the plurality of third images according to the exposure of the fifth image and the exposure of the second image determined in advance, and to determine a plurality of second images according to the sixth image corresponding to each second image.

[0168] The processor of the embodiment of the present application is also used to obtain multiple fifth images with the same exposure, and to determine the sixth image corresponding to the second image among the multiple third images based on the exposure of the fifth image and the exposure of the predetermined second image, and to determine multiple second images based on the sixth image corresponding to each second image.

[0169] Specifically, in the embodiment of the present application, the second image in the embodiment of the present application can be generated by photographing a plurality of fifth images of the same object at a fixed exposure.

[0170] For example, suppose Figure 5 The exposure of the LDR image shown is Exposure(1), and Exposure(1) is 4 times the exposure of the fifth image, so Figure 5 The LDR image shown can be obtained by synthesizing four fifth images. Furthermore, assuming that there are four third images in total, the four third images can be superimposed at the pixel level to obtain Figure 5 It can be understood that in this example, Figure 5 The sixth image corresponding to the LDR image shown is four fifth images.

[0171] Furthermore, assuming Figure 6 The exposure of the LDR image shown is Exposure(2), and Exposure(2) is three times the exposure of the fifth image. Figure 6The LDR image shown can be obtained by synthesizing three fifth images. Furthermore, when there are four third images in total, three of the four third images can be extracted to perform pixel-level superposition, thereby obtaining Figure 6 It can be understood that in this example, Figure 6 The fourth images corresponding to the LDR images shown are three of the four third images.

[0172] Thus, in an embodiment of the present application, when a plurality of fifth images with the same exposure amount are acquired, a fourth image corresponding to the second image among the plurality of third images can be determined according to the exposure amount of the third image and the exposure amount of the predetermined second image, and a plurality of second images can be determined according to the fourth image corresponding to each second image, so that the acquisition of the second image can be achieved through the plurality of fifth images with the same exposure amount, thereby ensuring the robust acquisition of the second image to a certain extent.

[0173] See also Fig.17 In the implementation mode of the present application, step 011 includes:

[0174] 0110: determining the required number of fifth images corresponding to the second image according to the ratio of the exposure amount of the second image to the exposure amount of the fifth image;

[0175] 0111: Determine a sixth image corresponding to the second image in the plurality of third images according to the required number of fifth images corresponding to the second image.

[0176] The processing unit of the embodiment of the present application is configured to determine the required number of fifth images corresponding to the second image based on the ratio of the exposure amount of the second image to the exposure amount of the fifth image, and to determine the sixth image corresponding to the second image in multiple third images based on the required number of fifth images corresponding to the second image.

[0177] The processor of the embodiment of the present application is also used to determine the required number of fifth images corresponding to the second image based on the ratio of the exposure amount of the second image to the exposure amount of the fifth image, and to determine the sixth image corresponding to the second image in multiple third images based on the required number of fifth images corresponding to the second image.

[0178] It should be noted that, in the implementation manner of the present application, the exposure amount of any fourth image is lower than or equal to the exposure amount of any second image.

[0179] Further, let Figures 5 to 8 In the four LDR images shown, the exposure of the fth LDR image is Exposure(f), and the exposure of the fifth image is LowExposure, then formula (8) can be obtained, that is:

[0180]

[0181] In formula (8), Num(f) represents the number of fifth images required for the f-th second image. It is understandable that the operation of rounding down is only used for the convenience of explaining the implementation mode of the present application, and the operation of rounding down can be set according to actual conditions.

[0182] For example, suppose Figure 5 The exposure of the LDR image is Exposure(1), and Exposure(1) is 4 times of LowExposure, so Num(1) is 4. There are 4 third images in total, so the 4 fifth images can be superimposed to obtain Figure 5 The LDR image shown. In other words, Figure 5 The sixth image corresponding to the LDR image shown is the entire fifth image.

[0183] For example, suppose Figure 6 The exposure of the LDR image is Exposure(2), and Exposure(2) is three times of LowExposure, so Num(2) is 3. There are a total of four third images obtained. Then, three of the four fifth images can be extracted and superimposed to obtain Figure 6 The LDR image shown. In other words, Figure 6 The sixth image corresponding to the LDR image shown is three of the fifth images.

[0184] Thus, in an embodiment of the present application, the required number of fifth images corresponding to the second image can be determined based on the exposure amount of the third image and the predetermined exposure amount of the second image, and the sixth image corresponding to the second image in multiple third images can be determined based on the required number of fifth images corresponding to the second image.

[0185] See also Fig.18 In some embodiments of the present application, the plurality of fifth images are a plurality of frames of images taken continuously, and step 0111 includes:

[0186] 01110: determining a start frame image and an end frame image among the plurality of fifth images according to the required number of fifth images corresponding to the second image;

[0187] 01111: Determine a sixth image corresponding to the second image according to all images from a start frame image to an end frame image in the plurality of third images.

[0188] The transceiver unit of the embodiment of the present application is configured to determine the starting frame image and the ending frame image among multiple fifth images based on the required number of fifth images corresponding to the second image, and to determine the sixth image corresponding to the second image based on all images from the starting frame image to the ending frame image in the multiple third images.

[0189] The processor of the embodiment of the present application is also used to determine the starting frame image and the ending frame image among multiple fifth images based on the required number of fifth images corresponding to the second image, and to determine the sixth image corresponding to the second image based on all images from the starting frame image to the ending frame image in multiple third images.

[0190] Specifically, in the embodiment of the present application, the second image can be obtained by extracting one or more consecutive frames of images from the multiple frames of the fifth image that are continuously captured. Therefore, in order to ensure the accurate extraction of the frame image, the embodiment of the present application can determine the starting frame image and the ending frame image in the multiple fifth images.

[0191] To more clearly illustrate the implementation of this application, please refer to Fig.19 , Fig.19 Schematic diagram of application scenarios in certain embodiments of the present application. Fig.19 In the above formula, StartN(1) indicates the sequence number of the start frame image of the first second image among all fifth images, StartN(2) indicates the sequence number of the start frame image of the second second image among all fifth images, StopN(1) indicates the sequence number of the end frame image of the first second image among all fifth images, StopN(2) indicates the sequence number of the end frame image of the second second image among all fifth images, Num(1) indicates the required number of fifth images of the first second image, and Num(2) indicates the required number of fifth images of the second second image. Fig.16 The same can be said for other variables.

[0192] And, in one example, the required number of frames can be determined by the aforementioned formula (8), the sequence number StartN(f) of the start frame image corresponding to the f-th second image can be determined by the following formula (9), and the sequence number StopN(f) of the end frame image corresponding to the f-th second image can be determined by the following formula (10), that is:

[0193]

[0194] StopN(f)=StartN(f)+Num(f)-1 (10)

[0195] Thus, in the implementation mode of the present application, the starting frame image and the ending frame image in the multiple fifth images taken continuously can be determined according to the required number of fifth images corresponding to the second image, and then the multiple consecutive fifth images from the starting frame image to the ending frame image can be determined as the sixth image corresponding to the second image, thereby ensuring the reliable determination of the sixth image to a certain extent.

[0196] In some embodiments of the present application, step 012 includes:

[0197] In the case that there are multiple sixth images corresponding to the second image, the multiple sixth images are synthesized according to the pixel values ​​of the pixels in each sixth image and a predetermined second image synthesis function to determine the second image.

[0198] The processing unit of the embodiment of the present application is configured to, when there are multiple sixth images corresponding to the second image, synthesize the multiple sixth images according to the pixel values ​​of the pixels in each sixth image and a predetermined second image synthesis function to determine the second image.

[0199] The processor of the embodiment of the present application is also used to, when there are multiple sixth images corresponding to the second image, synthesize the multiple sixth images according to the pixel values ​​of the pixels in each sixth image and a predetermined second image synthesis function to determine the second image.

[0200] Specifically, in an embodiment of the present application, when the sixth image corresponding to the second image is a continuous plurality of frames, the electronic device may synthesize the continuous plurality of sixth images at the pixel level according to a predetermined second image synthesis function to obtain the second image.

[0201] In one example, the synthesis method of the second image can refer to the following formula (11), that is:

[0202]

[0203] In formula (11), LDR(x,y,f) represents the pixel value of the pixel at (x,y) in the f-th second image. LE_Bayer(x,y,f) represents the pixel value of the pixel at (x,y) in the n-th fifth image, and the value of n ranges from StartN(f) to StopN(f).

[0204] It can be understood that StartN(f) in formula (11) can be obtained by the aforementioned formula (9), and StopN(f) can be obtained by the aforementioned formula (10).

[0205] It is also understood that the bit width of a general LDR image can be any one of 8 bits, 10 bits and 12 bits. Further, in the embodiment of the present application, when the bit width of the LDR image is 8 bits, that is, when the image dynamic range is 0 to 255, the value greater than 255 in LDR (x, y, f) can be forcibly assigned to 255.

[0206] Thus, in the embodiment of the present application, when there are multiple sixth images corresponding to the second image, the multiple sixth images may be synthesized at the pixel level according to a predetermined second image synthesis function to determine the second image.

[0207] In some embodiments of the present application, step 012 includes:

[0208] When the sixth image corresponding to the second image is one, the sixth image is determined as the second image.

[0209] The processing unit in the embodiment of the present application is configured to determine the sixth image as the second image when the second image corresponds to only one sixth image.

[0210] The processor of the embodiment of the present application is further configured to determine the sixth image as the second image when the second image corresponds to only one sixth image.

[0211] Specifically, in the embodiment of the present application, when the required number of fifth images is 1 and thus the start frame image and the end frame image are the same image, the sixth image corresponding to the second image is one.

[0212] Furthermore, in the case where the second image corresponds to a sixth image, the electronic device according to the embodiment of the present application may directly determine the sixth image as the second image.

[0213] Thus, in the embodiment of the present application, when the sixth image corresponding to the second image is one, the sixth image corresponding to the second image may be determined as the second image.

[0214] See also Fig. 20 In certain embodiments of the present application, step 01 includes:

[0215] 013: Acquire a sixth image, wherein a dynamic range of the sixth image is higher than that of the second image;

[0216] 014: Decompose the sixth image according to the pixel values ​​of the pixels in the sixth image, the predetermined image coefficients of the second image, and the predetermined image decomposition function to determine the second image.

[0217] The transceiver unit of the embodiment of the present application is configured to obtain a sixth image, and decompose the sixth image according to the pixel values ​​of the pixels in the sixth image, the predetermined image coefficients of the second image, and the predetermined image decomposition function to determine the second image, and the dynamic range of the sixth image is higher than the dynamic range of the second image.

[0218] The processor of the embodiment of the present application is also used to acquire a sixth image, and to decompose the sixth image according to the pixel values ​​of the pixels in the sixth image, the predetermined image coefficients of the second image, and the predetermined image decomposition function to determine the second image, and the dynamic range of the sixth image is higher than the dynamic range of the second image.

[0219] Specifically, considering certain scenarios, the image output by the sensor is an HDR image (i.e., the sixth image). Therefore, the embodiment of the present application can decompose the HDR image to obtain multiple frames of different-exposure LDR images, i.e., obtain multiple second images, when the HDR image output by the sensor is obtained.

[0220] To more clearly illustrate the implementation of this application, please refer to Fig.21 , Fig.21 Schematic diagram of application scenarios in certain embodiments of the present application. Fig.21 In , Wink(1) represents the image coefficient of the first second image, Wink(2) represents the image coefficient of the second second image, and other variable parameters can be deduced accordingly or refer to the above description.

[0221] It is understandable that the bit width of a common HDR image can be one of 16 bits, 18 bits and 24 bits. For the convenience of explaining the implementation of the present application, the bit width of the HDR image (i.e., the sixth image) is 16 bits, and the dynamic range of the corresponding HDR image is 0 to 2 16 -1.

[0222] It is also understandable that Fig.21 As shown, for the single frame HDRin(x, y) with high dynamic range to be decomposed, since the HDR image basically contains the dynamic range of each current image scene, the HDR image can be windowed to decompose and obtain an LDR image with a certain dynamic range. That is, when the dynamic range of each second image has been predetermined, a corresponding window coefficient can be set for each second image according to the dynamic range of each second image, that is, the image coefficient in the embodiment of the present application. Then, according to the predetermined image decomposition function, the sixth image is decomposed. Among them, the image decomposition function can be expressed by the following formula (12), that is:

[0223]

[0224] In formula (12), Represents the maximum limit function.

[0225] In one example, It can be expressed by the following formula (13):

[0226]

[0227] In formula (13), i represents the pixel value.

[0228] Based on formula (12) and formula (13), in one example, the method for acquiring the second image can refer to the following formula (14), that is:

[0229]

[0230] In addition, since the dynamic range transition between two adjacent frames of images needs to be appropriate, F frames of LDR images can be input into the automatic exposure module of the camera, so that the automatic exposure module adjusts WinK(f) according to the brightness relationship between the images, thereby obtaining a set of more appropriate LDR images again. After continuous cyclic adjustment, a set of most appropriate LDR images (i.e., multiple second images) will be obtained.

[0231] Thus, in the embodiment of the present application, when a sixth image with a higher dynamic range is acquired, the sixth image may be decomposed according to predetermined image coefficients of each second image to determine a plurality of second images with a lower dynamic range.

[0232] See also Fig. 22 In certain embodiments of the present application, step 02 includes:

[0233] 020: Determine the maximum brightness of the pixels of the second image according to the brightness of each pixel in the second image;

[0234] 021: Determine a first weight of the second image according to the maximum brightness value of the pixels of the second image.

[0235] The processing unit of the embodiment of the present application is configured to determine the maximum brightness of the pixels of the second image according to the brightness of each pixel in the second image, and determine the first weight of the second image according to the maximum brightness of the pixels of the second image.

[0236] The processor of the embodiment of the present application is also used to determine the maximum brightness of the pixels of the second image according to the brightness of each pixel in the second image, and to determine the first weight of the second image according to the maximum brightness of the pixels of the second image.

[0237] Specifically, in an embodiment of the present application, the maximum brightness of each pixel in the second image can be determined according to the brightness of each pixel in the second image, and the first weight of the second image can be determined according to the maximum brightness of the pixels in the second image.

[0238] In one example, the brightness of each pixel in the second image can be calculated by formula (15), that is:

[0239]

[0240] Wherein, Y(x, y, f) represents the brightness of the pixel at (x, y) in the f-th second image (or LDR image).

[0241] In one example, the maximum value of the pixel brightness of the second image can be calculated by formula (16), that is:

[0242] Y max (f)=Max(Y(x,y,f)) (16)

[0243] Where Y max (f) represents the maximum brightness value of the pixel in the f-th second image.

[0244] Thus, in an embodiment of the present application, the maximum brightness of the pixels of the second image can be determined according to the brightness of each pixel in the second image, and the first weight of the second image can be determined according to the maximum brightness of the pixels of the second image.

[0245] In certain embodiments of the present application, step 05 includes:

[0246] A first weight of the second image is determined according to a maximum brightness value of a pixel point of the second image and a piecewise function, wherein the piecewise function is used to associate brightness with weight.

[0247] The processing unit of the embodiment of the present application is configured to determine a first weight of the second image according to a maximum brightness value of a pixel point of the second image and a piecewise function, wherein the piecewise function is used to associate brightness with weight.

[0248] The processor of the embodiment of the present application is also used to determine the first weight of the second image according to the maximum brightness of the pixels of the second image and the piecewise function, wherein the piecewise function is used to associate the brightness with the weight.

[0249] Specifically, in the implementation manner of the present application, the first weight of each second image may be determined by a preset piecewise function.

[0250] For example, assuming that the second image includes 5 images with successively lower exposures, the last 4 of the 5 images are Figures 5 to 8As shown in FIG. 4, namely LDR(1), LDR(2), LDR(3) and LDR(4), and assuming that the first second image LDR(0) among the five images is a fully exposed image, the maximum pixel brightness Y of the second image LDR(0) is max (0) is 255.

[0251] Further, in the embodiment of the present application, the shape of the piecewise function can be determined by the four parameters Ay, Bx, Cx, and Cy. It can be understood that the four parameters Ay, Bx, Cx, and Cy can be set according to actual conditions. In an example, the piecewise function can be as follows: Fig.23 As shown, Fig.23 This is a schematic diagram of a piecewise function in certain embodiments of the present application.

[0252] Furthermore, since LDR(0) is a fully exposed image and LDR(1) is an incompletely exposed LDR image with the maximum exposure, it can characterize the details of the dark part of the LDR image. Therefore, Ay(1)=255, Bx(1)=255, Cx(1)=255, Cy(1)=255 can be set, and the first weight weight(1) of LDR(1) is 255.

[0253] For the second images in the latter order, the settings of Ay, Bx, Cx, and Cy for each second image can be as shown in formula (17), that is:

[0254]

[0255] Based on the maximum brightness value Y of each second image max , and the piecewise function with completed parameter setting, the first weight Weight of each second image can be determined by formula (18), that is:

[0256]

[0257] Wherein, Weight(f) represents the first weight of the f-th second image.

[0258] Thus, in the embodiment of the present application, the first weight of the second image is determined according to the maximum brightness value of the pixel points of the second image and the piecewise function.

[0259] In addition, it can be understood that the piecewise function can be set according to actual conditions. The piecewise function determined by the four parameters Ay, Bx, Cx, and Cy proposed in the implementation of the present application is only a feasible example. The piecewise function can also be set to other forms according to actual conditions.

[0260] In certain embodiments of the present application, step 03 includes:

[0261] The plurality of second images are synthesized according to the first weight of each second image, the pixel value of each pixel in each second image, and a predetermined third image synthesis function to determine a third image.

[0262] The processing unit of the embodiment of the present application is configured to synthesize multiple second images according to the first weight of each second image, the pixel value of each pixel in each second image and a predetermined third image synthesis function to determine the third image.

[0263] The processor of the embodiment of the present application is also used to synthesize multiple second images according to the first weight of each second image, the pixel value of each pixel in each second image and a predetermined third image synthesis function to determine a third image.

[0264] That is, in an embodiment of the present application, the electronic device can obtain a third image by performing weighted summation on each second image at the pixel level using the first weight of each second image, the pixel value of each pixel in each second image, and a predetermined third image synthesis function.

[0265] In one example, the third image synthesis function is as shown in the aforementioned formula (1), that is:

[0266]

[0267] In formula (1), LDR_Blend(x,y) represents the pixel value of the pixel at (x,y) in the third image, Weight(f) represents the first weight of the second image, LDR(x,y,f) represents the pixel value of the pixel at (x,y) in the f-th second image, and Weight(f) represents the first weight of the f-th second image.

[0268] Thus, in an embodiment of the present application, the electronic device can obtain a third image by performing weighted summation on each second image at the pixel level using the first weight of each second image, the pixel value of each pixel in each second image, and a predetermined fourth image synthesis function.

[0269] In certain embodiments of the present application, step 04 includes:

[0270] The first image and the third image are synthesized to generate a target image according to the pixel value of each pixel in the first image, the predetermined weight of the first image, the pixel value of each pixel in the third image, the predetermined weight of the third image and a predetermined fourth image synthesis function.

[0271] The processing unit of the embodiment of the present application is configured to synthesize the first image and the third image to generate a target image based on the pixel value of each pixel in the first image, a predetermined weight of the first image, the pixel value of each pixel in the third image, a predetermined weight of the third image, and a predetermined fourth image synthesis function.

[0272] The processor of the embodiment of the present application is also used to synthesize the first image and the third image to generate a target image based on the pixel value of each pixel in the first image, a predetermined weight of the first image, the pixel value of each pixel in the third image, a predetermined weight of the third image, and a predetermined fourth image synthesis function.

[0273] Specifically, in the implementation manner of the present application, a weighted summation at the pixel level may be performed on the first image and the third image according to a predetermined fourth image synthesis function to obtain a target image.

[0274] In one example, the fourth image synthesis function can be expressed by formula (19), that is:

[0275]

[0276] Wherein, BlendBayer(x,y) represents the pixel value of the pixel at (x,y) in the target image, LDR_Blend(x,y) represents the pixel value of the pixel at (x,y) in the third image, LBW represents the weight of the third image, ToneMap(x,y) represents the pixel value of the pixel at (x,y) in the first image, and TWM represents the weight of the first image.

[0277] It is understandable that both LBW and TWM can be set according to actual conditions. For example, in one example, LBW and TWM are equal. In another example, LBW is less than TWM.

[0278] Thus, in an embodiment of the present application, the first image and the third image can be synthesized at the pixel level to generate a target image through a fourth image synthesis function, the pixel value of each pixel in the first image, the predetermined weight of the first image, the pixel value of each pixel in the third image, and the predetermined weight of the third image.

[0279] In certain embodiments of the present application, please refer to Fig.24 and Fig.25 , Fig.24 This is a schematic diagram of application scenarios in certain embodiments of the present application. Fig.25 Schematic diagram of the flow of the image generation method in certain embodiments of the present application.

[0280] Specifically, if Fig.24 and Fig.25 As shown, in an embodiment of the present application, the electronic device (or control device, or vehicle) can obtain a third image through the F-frame second images and the above-mentioned formulas (1), (15)-(18) after obtaining a series of F-frame second images (or LDR images) with successively decreasing exposure amounts.

[0281] At the same time, the fourth image (or HDR) is obtained through the F frames of second images and the above formulas (5) to (7). The fourth image is then dynamically compressed through the above formulas (2) to (4) to obtain the first image.

[0282] Finally, according to the above formula (19), the first image and the third image are weightedly summed at the pixel level to obtain the target image.

[0283] The embodiment of the present application further provides a vehicle, wherein the vehicle includes the above-mentioned control device or the above-mentioned electronic device.

[0284] The embodiment of the present application further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned image generation method when the computer program is executed by one or more processors.

[0285] In the description of this specification, the descriptions with reference to the terms "specifically", "further", "particularly", "understandably", etc. are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0286] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0287] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An image generation method, characterized in that: include: Acquire a plurality of second images corresponding to the first image, wherein the first image is determined by the plurality of second images, and the plurality of second images correspond to a plurality of exposure values; determining a first weight of the second image according to an exposure amount of the second image; Performing synthesis processing on the plurality of second images according to the first weight of each second image and the pixel value of a pixel point in each second image to determine a third image; A target image is generated according to the first image and the third image.

2. The method according to claim 1, characterized in that The method further comprises: synthesizing the plurality of second images according to the pixel value of each pixel in each of the second images to obtain a fourth image, wherein a dynamic range of the fourth image is higher than a dynamic range of the second image; According to the predetermined compression curve parameters and tone mapping function, tone mapping processing is performed on the fourth image to obtain the first image.

3. The method according to claim 2, characterized in that The synthesizing process of the plurality of second images according to the pixel value of each pixel point in each of the second images to obtain a fourth image comprises: Determining a second weight for each pixel in the second image according to a pixel value of each pixel in the second image; Determining a maximum image exposure according to the exposure of each of the second images; determining an exposure weight of the second image according to a ratio of the exposure of the second image to the maximum exposure of the image; The fourth image is obtained by synthesizing the plurality of second images according to the pixel value of each pixel in the second image, the second weight of each pixel in the second image, and a predetermined first image synthesis function.

4. The method according to claim 3, characterized in that The determining, according to the pixel value of each pixel in the second image, a second weight of each pixel in the second image comprises: When the pixel value of a pixel point in the second image is less than a predetermined first preset value, determining the second weight of the pixel point according to the pixel value of the pixel point; and / or When the pixel value of a pixel point in the second image is greater than or equal to the first preset value, the second weight of the pixel point is determined according to a difference between a predetermined second preset value and the pixel value of the pixel point.

5. The method according to claim 1, characterized in that The acquiring of a plurality of second images corresponding to the first image comprises: acquiring a plurality of fifth images with the same exposure; determining a sixth image corresponding to the second image among the plurality of third images according to the exposure amount of the fifth image and the predetermined exposure amount of the second image; The plurality of second images are determined according to a sixth image corresponding to each of the second images.

6. The method according to claim 5, characterized in that Determining a sixth image corresponding to the second image among the plurality of third images according to the exposure amount of the fifth image and the predetermined exposure amount of the second image includes: determining a required number of fifth images corresponding to the second image according to a ratio of an exposure amount of the second image to an exposure amount of the fifth image; The sixth image corresponding to the second image in the plurality of third images is determined according to the required number of the fifth images corresponding to the second image.

7. The method according to claim 6, characterized in that The plurality of fifth images are a plurality of frames of images taken continuously, and determining the sixth image corresponding to the second image in the plurality of third images according to the required number of the fifth images corresponding to the second image includes: Determining a starting frame image and an ending frame image among the plurality of fifth images according to a required number of fifth images corresponding to the second image; The sixth image corresponding to the second image is determined according to all images from the start frame image to the end frame image in the plurality of third images.

8. The method according to claim 5, characterized in that The determining the plurality of second images according to the sixth image corresponding to each of the second images includes: In the case that there are multiple sixth images corresponding to the second image, the multiple sixth images are synthesized according to the pixel values ​​of the pixels in each of the sixth images and a predetermined second image synthesis function to determine the second image.

9. The method according to claim 5, characterized in that The determining the plurality of second images according to the sixth image corresponding to each of the second images includes: When the sixth image corresponding to the second image is one image, the sixth image is determined as the second image.

10. The method according to claim 1, characterized in that The acquiring of a plurality of second images corresponding to the first image comprises: Acquire a sixth image, wherein a dynamic range of the sixth image is higher than a dynamic range of the second image; The sixth image is decomposed according to the pixel values ​​of the pixels in the sixth image, the predetermined image coefficients of the second image and the predetermined image decomposition function to determine the second image.

11. The method according to claim 1, characterized in that: The determining the first weight of the second image according to the exposure amount of the second image includes: Determine the maximum brightness of a pixel of the second image according to the brightness of each pixel in the second image; The first weight of the second image is determined according to a maximum brightness value of pixels of the second image.

12. The method according to claim 11, characterized in that The determining the first weight of the second image according to the maximum value of the brightness of the pixels of the second image includes: The first weight of the second image is determined according to the maximum brightness value of the pixel points of the second image and a predetermined piecewise function, wherein the piecewise function is used to associate brightness with weight.

13. The method according to claim 1, characterized in that The synthesizing process of the plurality of second images according to the first weight of each second image and the pixel value of each pixel in the second image to determine the third image includes: The plurality of second images are synthesized according to the first weight of each second image, the pixel value of each pixel in each second image, and a predetermined third image synthesis function to determine the third image.

14. The method according to claim 1, characterized in that Generating a target image according to the first image and the third image comprises: The first image and the third image are synthesized according to the pixel value of each pixel in the first image, the predetermined weight of the first image, the pixel value of each pixel in the third image, the predetermined weight of the third image and a predetermined fourth image synthesis function to generate the target image.

15. A control device, characterized in that: including a transceiver unit and a processing unit; The transceiver unit is configured to obtain a plurality of second images corresponding to the first image, wherein the first image is determined by the plurality of second images, and the plurality of second images correspond to a plurality of exposure amounts; The processing unit is configured to determine a first weight of the second image according to the exposure of the second image, and to synthesize the multiple second images according to the first weight of each second image and the pixel value of each pixel in the second image to determine a third image, and to generate a target image according to the first image and the third image.

16. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 14 is implemented.

17. A vehicle, characterized in that: Comprising the device of claim 15 or 16.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 14 is implemented.

Images