Exposure control method and device, HDR image fusion method and product

Abstract

The invention provides an exposure control method and device, an HDR image fusion method and a product, the method is applied to the technical field of image processing, and the method comprises the following steps: obtaining multiple frames of images and brightness information of the multiple frames of images; the brightness information, the underexposure point threshold value and the overexposure point threshold value of the multiple frames of images are utilized to determine a region of interest of each frame of image, and the region of interest represents a region needing to be fully exposed in the image; determining a brightness change coefficient of each frame of image according to a difference value between the brightness of the region of interest and target brightness; according to the brightness change coefficient, adjusting the current exposure of each frame of image to obtain the target exposure of each frame of image; and shooting according to the target exposure to obtain a new multi-frame image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an exposure control method, device, HDR image fusion method and product. Background Art

[0002] High Dynamic Range Imaging (HDR) image fusion technology requires the use of multiple frames of images with different exposures to obtain details in the highlights or shadows, and to expand the dynamic range of the image after fusion. Therefore, controlling the good exposure of multiple frames of images has become an important prerequisite for the success of HDR image fusion.

[0003] However, the existing image exposure control methods are conservative and often use fixed exposure ratio parameters to achieve exposure control for a single frame of image. The fixed parameter exposure control method cannot adapt to scenes with different image dynamic ranges and has poor flexibility. It is difficult to achieve appropriate exposure for each frame of image, which limits the dynamic range of the composite image. Therefore, how to achieve good exposure control for multiple frames of image and improve the accuracy of exposure control has become a problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0004] In view of the above problems, the embodiments of the present application provide an exposure control method, device, HDR image fusion method and product to overcome the above problems or at least partially solve the above problems.

[0005] A first aspect of an embodiment of the present application provides an exposure control method, the method comprising:

[0006] Acquire multiple frames of images and brightness information of the multiple frames of images;

[0007] Determine a focus area of ​​each frame of image by using brightness information of the multiple frames of image, an underexposure point threshold, and an overexposure point threshold, wherein the focus area represents an area in the image that needs to be fully exposed;

[0008] Determining a brightness variation coefficient of each frame of the image according to a difference between the brightness of the focus area and the target brightness;

[0009] According to the brightness variation coefficient, the current exposure of each frame of image is adjusted to obtain the target exposure of each frame of image;

[0010] Shooting is performed according to the target exposure to obtain new multi-frame images.

[0011] A second aspect of an embodiment of the present application provides an exposure control device, the device comprising:

[0012] An image acquisition module, used to acquire multiple frames of images and brightness information of the multiple frames of images;

[0013] A focus region determination module, used to determine a focus region of each frame of image by using brightness information of the multiple frames of image, an underexposure point threshold and an overexposure point threshold, wherein the focus region indicates a region in the image that needs to be fully exposed;

[0014] A brightness variation coefficient determination module, used to determine the brightness variation coefficient of each frame of the image according to the difference between the brightness of the focus area and the target brightness;

[0015] A target exposure determination module, used to adjust the current exposure of each frame of image according to the brightness variation coefficient to obtain a target exposure of each frame of image;

[0016] The shooting module is used to shoot according to the target exposure to obtain new multiple-frame images.

[0017] A third aspect of the embodiments of the present application provides an HDR image fusion method, the method comprising:

[0018] Acquire a plurality of frames of images, where the plurality of frames of images are obtained by the exposure control method according to any one of the first aspects of the embodiments of the present application;

[0019] The multiple frames of images are used to perform HDR image fusion to obtain a fused image.

[0020] A fourth aspect of the embodiments of the present application provides a computing and processing device, including:

[0021] a memory having computer readable code stored therein; and

[0022] One or more processors, when the computer readable code is executed by the one or more processors, the computing and processing device executes the exposure control method as described in any one of the first aspects of the embodiments of the present application, or the computing and processing device executes the HDR image fusion method as described in the third aspect of the embodiments of the present application.

[0023] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program / instruction stored thereon, the computer program / instruction being executed by a processor as the exposure control method described in any one of the first aspect of the embodiment of the present application, or the computing processing device executing the HDR image fusion method described in the third aspect of the embodiment of the present application.

[0024] The embodiments of the present application provide an exposure control method, device, HDR image fusion method and product, the exposure control method comprising: obtaining multiple frames of images and brightness information of the multiple frames of images; using the brightness information of the multiple frames of images, underexposure point thresholds and overexposure point thresholds to determine the focus area of ​​each frame of image, the focus area represents the area in the image that needs to be fully exposed; according to the difference between the brightness of the focus area and the target brightness, the brightness variation coefficient of each frame of image is determined; according to the brightness variation coefficient, the current exposure of each frame of image is adjusted to obtain the target exposure of each frame of image; shooting according to the target exposure to obtain new multiple frames of images. Compared with performing exposure analysis and calculation on a single frame of image, the embodiments of the present application obtain multiple frames of images, determine the focus area in each frame of image in combination with the brightness information of the multiple frames of images, and can more accurately obtain the area in each frame of image that needs to be fully exposed. The embodiment of the present application further determines the brightness variation coefficient of each frame of the image, obtains the target exposure required for each frame of the image based on the brightness variation coefficient, and captures the image according to the target exposure to obtain new multi-frame images with appropriate exposure, thereby achieving good exposure control of the multi-frame images and being able to adapt to a variety of complex scenes with different image dynamic ranges. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Figure 1 is a schematic diagram of a method for controlling exposure of a single frame image provided by an embodiment of the present application;

[0027] Figure 2 It is a schematic diagram of a method for individually controlling the exposure of each frame of an image provided by an embodiment of the present application;

[0028] Figure 3 is a schematic flow chart of the steps of an exposure control method provided in an embodiment of the present application;

[0029] Figure 4 is a schematic diagram of the distribution of brightness values ​​of various image points in an image provided by an embodiment of the present application;

[0030] Figure 5 is a schematic diagram of exposure control of multiple frames of images provided in an embodiment of the present application;

[0031] Figure 6 is a schematic structural diagram of an exposure control device provided in an embodiment of the present application;

[0032] Figure 7 This is a flowchart of the steps of an HDR image fusion method provided in an embodiment of the present application;

[0033] Figure 8 This is a distribution diagram of a region of interest provided in an embodiment of the present application;

[0034] Fig. 9 This is a schematic diagram of the distribution of the focus area of ​​the first frame image provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The exemplary embodiments of the present application will be described in more detail below in conjunction with the accompanying drawings in the embodiments of the present application. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to enable the scope of the present application to be fully communicated to those skilled in the art.

[0036] HDR image fusion refers to using the camera to continuously output multiple frames of images from underexposure to overexposure in time, and then selecting the most valuable part of each frame of the multiple frames for image fusion to obtain an HDR image with a wider dynamic range. HDR image fusion technology requires the use of multiple frames of images with different exposures to obtain the details of the highlights and shadows, and to achieve the expansion of the image dynamic range after fusion. Therefore, controlling the good exposure of multiple frames has become an important prerequisite for the success of HDR image fusion. As for exposure (Exposure), in photography, exposure refers to the amount of light allowed to enter the lens and shine on the photosensitive medium (the film of a film camera or the image sensor of a digital camera) during the photography process. Exposure can be controlled by a combination of aperture, shutter and the sensitivity of the photosensitive medium.

[0037] In the related art, a method of automatically calculating the exposure of a single frame of a multi-frame image is mainly used, referring to Figure 1 , Figure 1 A schematic diagram of a method for controlling exposure of a single frame image is shown. Figure 1As shown, the method performs automatic exposure calculation on the first frame image to obtain the exposure amount that needs to be fed back for the frame image, and then performs exposure control on the other frames of images based on the exposure amount according to the preset exposure ratio. Such a control method is limited to the maximum dynamic range of a single frame image, and the exposure control is usually conservative. If the image dynamic range of the current scene is large, it may cause the long integral exposure to be not thorough enough and fail to obtain sufficient image details in the shadow area; the short integral exposure is not accurate enough and the details in the highlight area are lost. By increasing the set target brightness, removing the statistics of overexposed points or modifying the exposure ratio, the appropriate exposure of the current high dynamic range scene can be achieved, but in the low dynamic range scene, the problem of overexposure and dynamic range connection error will occur. By using a large number of different exposure images for HDR fusion, the above problems can be effectively reduced, but the real-time HDR image fusion process is limited by factors such as resources and frame rate, and a large number of images cannot be used for fusion. Therefore, the above method of performing exposure calculation on a single frame image and controlling the exposure of the remaining frames according to the preset ratio is heavily dependent on the control of the single frame image and the preset exposure ratio. A single parameter cannot adapt to scenes with different dynamic ranges, and the flexibility is poor. It is difficult to achieve appropriate exposure for each frame image, which limits the dynamic range of the composite image.

[0038] also, Figure 2 A schematic diagram of a method for individually controlling the exposure of each frame of an image is also shown. Figure 2 As shown, this method performs individual exposure control on each frame of image. Although flexible control of each frame of image is achieved, the dynamic range connection between the frames of image obtained in this way is prone to errors, and the details of the synthesized image are seriously lost in the transition area. In addition, there are also problems such as large differences in exposure control in different scenes and poor parameter adaptability, which makes the compatibility and robustness of this adjustment method still unable to meet the requirements.

[0039] In view of the above problems, the embodiments of the present application propose an exposure control method, device, HDR image fusion method and product to solve the problems of insufficient precision of the exposure control of the above-mentioned multi-frame images, poor parameter adaptability, and difficulty in compatibility with various complex scenes, so as to achieve a higher exposure effect. The exposure control method provided by the embodiments of the present application is described in detail below in conjunction with the accompanying drawings through some embodiments and their application scenarios.

[0040] This embodiment proposes an exposure control method, referring to Figure 3 , Figure 3 A schematic diagram of the steps of an exposure control method is shown in FIG. Figure 3 As shown, the system includes:

[0041] Step S301, obtaining multiple frames of images and brightness information of the multiple frames of images.

[0042] In a specific implementation, the multiple frames of images may be multiple frames of images obtained by continuously shooting the same scene or the same target. In order to use the multiple frames of images for HDR image fusion, the exposure amounts between the multiple frames of images are different. After the multiple frames of images are obtained by shooting, the brightness of the images may be analyzed and calculated to obtain the brightness information of each frame of the multiple frames of images, and the brightness information includes the brightness value of each image point in the frame of the image.

[0043] In an optional implementation, after obtaining multiple frames of images, brightness information of the multiple frames of images may be obtained in the following manner:

[0044] Reference Figure 4 , Figure 4 A schematic diagram showing the distribution of brightness values ​​of each image point in an image is shown in FIG. Figure 4 As shown in the figure, a single frame image is a Bayer diagram of size C*R, where C represents the width of the image and R represents the height of the image. The Bayer format is the original image inside the camera, and the image includes three basic colors: red, blue, and green. Where Bayer (x, y) represents an image point located at the (x, y) coordinate in the image. For example, Bayer (7, 8) represents an image point located at the coordinate of the 7th column and the 8th row.

[0045] In this embodiment, multiple frames of images with different exposures are obtained by continuous shooting. Let the total number of multiple frames be F, where F is a constant greater than 1. Correspondingly, Bayer (x, y, f) represents an image point located at the (x, y) coordinate in the f-th frame of the F-frame image. In the subsequent embodiments, unless otherwise specified, the restrictions on x, y, and f mentioned are: 1≤x≤C, 1≤y≤R, and 1≤f≤F. In addition, when F=1, the exposure control method proposed in the embodiment of the present application is a method for exposure control of a single frame image, which can also achieve good single-frame exposure control.

[0046] In this embodiment, the brightness value of each image point in each frame of image can be calculated according to the following formula:

[0047] (Formula 1)

[0048] Wherein, Y(x, y, f) represents the brightness value of the image point located at the (x, y) coordinate in the f-th frame image.

[0049] In addition, to facilitate the subsequent calculation and description of brightness, it is assumed that in addition to the F frame image, there is also a frame of fully exposed 0 image and a frame of insufficiently exposed F+1 image. The brightness of the image is:

[0050] Among them, Unex_th represents the underexposure point threshold, and the unit of the brightness value is the brightness level in the RGB color mode.

[0051] It should be noted that the method for calculating the brightness of an image is not limited to the arrangement and calculation methods shown in the above embodiments. Any calculation method that can correctly reflect the brightness value of an image point on the image space coordinates can be applied to the embodiments of the present application.

[0052] Step S302, using the brightness information of the multiple frames of images, the underexposure point threshold and the overexposure point threshold, determine the focus area of ​​each frame of image, where the focus area represents the area in the image that needs to be fully exposed.

[0053] In this embodiment, the underexposure point threshold and the overexposure point threshold are threshold points of the brightness value. If the brightness value of an image point is lower than the underexposure point threshold, it means that the image point is underexposed and the brightness is too low, and the amount of light entering needs to be increased; if the brightness value of the image point is higher than the overexposure point threshold, it means that the image point is overexposed and the brightness is too high, and the amount of light entering needs to be reduced.

[0054] In specific implementation, this embodiment can determine the size of the underexposure point threshold value Unex_th and the overexposure point threshold value Ovex_th in advance according to the number of frames of multiple frames of images and the image dynamic range of each frame of image. Specifically, if the number of frames of the image is large enough, it means that there are more valuable image points in the image. Correspondingly, the underexposure point threshold value is made higher and the overexposure point threshold value is made lower, so that the number of suitable image points between the two thresholds is further reduced. If the number of frames of the image is small, it means that there are relatively few valuable image points in the image. Correspondingly, the underexposure point threshold value is made lower and the overexposure point threshold value is made higher, so that the number of suitable image points between the two thresholds is larger. Image dynamic range refers to, similarly, for the image dynamic range of each frame of image, if the image dynamic range is wide enough, it means that there are more valuable image points in the image. Correspondingly, the underexposure point threshold value is made higher and the overexposure point threshold value is made lower, so that the number of suitable image points between the two thresholds is further reduced. If the dynamic range of the image is narrow, it means that there are relatively few valuable image points in the image. Correspondingly, the underexposure point threshold is made lower and the overexposure point threshold is made higher, so that more suitable image points can be accommodated between the two thresholds to ensure that there are enough image points that can be used for subsequent HDR image fusion. In specific implementation, it is necessary to make a judgment based on the number of image frames and the image dynamic range in actual conditions to determine the accurate underexposure point threshold Unex_th and overexposure point threshold Ovex_th.

[0055] In an optional implementation, the step S302, using the brightness information of the multiple frames of images, the underexposure point threshold and the overexposure point threshold, to determine the focus area of ​​each frame of image, includes:

[0056] Step S3021: for each frame of the multiple frames of images, according to the brightness information of the frame of image, an image point in the frame of image whose brightness value is less than the underexposure point threshold is determined as a first image point.

[0057] Specifically, according to the brightness value of the image point obtained in step S301, an image point in each frame of image that satisfies the following formula is determined as the first image point:

[0058] (Formula 2) Unex1 (x, y, f) = Y (x, y, f) < Unex_th, 0 ≤ f ≤ F + 1

[0059] Formula 2 indicates that for the brightness value Y(x, y, f) of the image point located at the (x, y) coordinate in the f-th frame image, if the brightness value is lower than the underexposure point threshold value Unex_th, the image point is the first image point Unex1(x, y, f). The first image point indicates an underexposed image point in the frame image that exceeds the range of [underexposure point threshold value, overexposure point threshold value] and has too low brightness.

[0060] Step S3022: For each frame of the multiple images, based on the brightness information of the frame, an image point in the frame whose brightness value is greater than or equal to the underexposure point threshold and less than m times the underexposure point threshold is determined as a second image point, where m is a constant greater than 1.

[0061] In a specific implementation, according to the brightness value of the image point obtained in step S301, an image point in each frame of image that satisfies the following formula is determined as the second image point:

[0062] (Formula 3) Unex2(x, y) = Unex_th ≤ Y(x, y, f) < m*Unex_th, 0 ≤ f ≤ F+1;

[0063] Formula 3 indicates that for the brightness value Y(x, y, f) of the image point located at the (x, y) coordinate in the f-th frame image, if the brightness value is greater than or equal to the underexposure point threshold Unex_th, and is lower than m times the underexposure point threshold, then the image point is the second image point Unex2(x, y). In this embodiment, the second image point represents an image point with lower brightness that meets the range of [underexposure point threshold, overexposure point threshold]. Correspondingly, the value of m cannot be too high, otherwise a larger range will be selected for the brightness of the second image point. Optionally, m can be 2.

[0064] Step S3023, for each frame of the multiple frames of images, according to the brightness information of the frame of image, an image point in the frame of image whose brightness value is greater than the overexposure point threshold is determined as a third image point, wherein m times the underexposure point threshold is less than the overexposure point threshold;

[0065] In a specific implementation, according to the brightness value of the image point obtained in step S301, an image point in each frame of image that satisfies the following formula condition is determined as the third image point:

[0066] (Formula 4) Ovex(x, y, f) = Y(x, y, f) > Ovex_th, 0≤f≤F+1;

[0067] Formula 4 indicates that for the brightness value Y(x, y, f) of the image point located at the (x, y) coordinate in the f-th frame image, if the brightness value is greater than the overexposure point threshold Ovex_th, then the image point is the third image point Ovex(x, y, f). The third image point represents an overexposed image point with excessive brightness that exceeds the range of [underexposure point threshold, overexposure point threshold]. In this embodiment, since the second image point represents an image point with a lower brightness value and the third image point represents an image point with an excessively high brightness value, the maximum brightness value of the second image point, i.e., m times the underexposure point threshold, should always be smaller than the minimum brightness value of the third image point, i.e., the underexposure point threshold.

[0068] Step S3024: determining a focus area of ​​each frame of image according to the first image point, the second image point and the third image point of the multiple frames of image.

[0069] In this embodiment, the first image point, the second image point and the third image point are determined for each frame of the multiple images, thereby obtaining the region of interest for each frame. The region of interest represents the region in the image that needs to be fully exposed, that is, the region that needs to be processed. This embodiment determines the region of interest based on multiple frames of images. Exemplarily, the previous frame of image, the current frame of image and the next frame of image can be selected, and the brightness information of the image points of the three frames of image can be combined to determine the region of interest for the current frame of image. Therefore, compared to performing exposure calculation based on a single frame of image, this embodiment combines the brightness information of multiple frames of image to determine the region of interest in each frame of image, thereby obtaining a more accurate image region that needs to be exposed, which is helpful for subsequent calculation of the exposure amount and improves the accuracy of exposure control.

[0070] In an optional implementation, the exposure amounts of the multiple frames of images are sequentially reduced according to the image shooting order, and the step S3024, determining the focus area of ​​each frame of image according to the first image point, the second image point, and the third image point of the multiple frames of images, includes:

[0071] Step S3024-a, the multiple frames of images include F frames of images, and according to the image shooting order, for each frame of images from the 2nd to F-1st frames of the F frames of images, the focus area of ​​the frame of image is determined according to the following steps:

[0072] Determine the first area of ​​the current frame image according to the image shooting sequence and based on the location information of the third image point in the previous frame image of the current frame image;

[0073] Determine a second area of ​​the current frame image according to location information of all image points except the third image point in the current frame image;

[0074] Determine a third area of ​​the current frame image according to the location information of the second image point in the frame image after the current frame image;

[0075] Determine a fourth area of ​​the current frame image according to the location information of the first image point in the frame image subsequent to the current frame image;

[0076] A focus area of ​​the current frame image is determined according to the first area, the second area, the third area, and the fourth area.

[0077] In this embodiment, the multiple frames of images are obtained by shooting in a certain image shooting sequence, and there are a total of F frames of images, and the exposure of the F frames of images decreases in sequence according to the image shooting sequence. Figure 5 , Figure 5 FIG. 4 shows a schematic diagram of exposure control of a multi-frame image. Figure 5 As shown, the camera sensor acquires multiple frames of images, and according to the image shooting order, the brightness of the multiple frames of images decreases from 1 to F. Assuming there is a 0th image, the exposure of the 0th image is 255, that is, the highest image exposure; assuming there is an F+1th image, the exposure of the F+1th image is Unex_th, that is, the lowest image exposure.

[0078] When determining the focus area of ​​the current frame image, the brightness information of the previous frame image, the current frame image and the next frame image are combined to determine the focus area. Figure 5As shown, the second frame image is taken as the current frame image. When determining the focus area of ​​the second frame image, the brightness information of the first frame image, the second frame image and the third frame image are combined to determine it. In this embodiment, the first area is determined according to the location information of the third image point in the previous frame image, and the first area indicates the location of the image point with too high brightness in the previous frame image; the second area is determined according to the location information of all image points except the third image point in the current frame image, and the second area indicates the location of the image point with brightness below the overexposure point threshold in the current frame image; the third area is determined according to the location information of the second image point in the next frame image, and the third area indicates the location of the point with lower brightness in the next frame image; the fourth area of ​​the current frame image is determined according to the location information of the first image point in the next frame image, and the fourth area indicates the location of the point with too low brightness in the next frame image. Finally, the focus area of ​​the current frame image is determined according to the determined first area, second area, third area and fourth area.

[0079] In an optional implementation, determining the focus area of ​​the current frame image according to the first area, the second area, the third area, and the fourth area includes:

[0080] Determine the intersection of the first region and the second region as a first candidate region;

[0081] Determine the intersection of the first candidate area and the third area as the second candidate area;

[0082] The union of the second candidate area and the fourth area is determined as the focus area of ​​the current frame image.

[0083] According to the image shooting order, for the 2nd to F-1 frames in the F frame images, the focus area of ​​any frame image can be determined according to the following formula:

[0084] Cal_Pixel(x, y, f)

[0085] =(Ovex(x,y,f-1)&&~Ovex(x,y,f)&&Unex2(x,y,f

[0086] +1))||Unex1(x,y,f+1)

[0087] (Formula 5);

[0088] Among them, Cal_Pixel(x, y, f) represents the focus area of ​​the f-th frame image, the symbol && represents the AND logical operator, indicating that if both areas are non-zero, the condition is true; the symbol || represents the OR logical operator, indicating that if any one of the two areas is non-zero, the condition is true.

[0089] In an optional implementation, determining the first area of ​​the current frame image according to the location information of the third image point in the previous frame image of the current frame image includes:

[0090] The vertical projection area of ​​the third image point in the previous frame image of the current frame image is determined as the first area of ​​the current frame image. The vertical projection area represents the area vertically projected in the current frame image through the parallel projection line perpendicular to the current frame image point when the image is parallel to the current frame image.

[0091] Ovex(x, y, f-1) represents the first area of ​​the current frame image, that is, the area in the current frame image corresponding to the position of the third image point in the previous frame image of the current frame image. For example, assuming that the overexposure point threshold is 240, Ovex(x, y, f-1) represents the area corresponding to the image point with brightness greater than 240 in the f-1 frame image. According to the image shooting order, the exposure of the previous frame image is greater than that of this frame image. Therefore, the area with too high brightness in the previous frame image (the area corresponding to the third image point) can be fully exposed in this frame image, so as to obtain a clearer image with a more appropriate exposure. Figure 8 , Figure 8 A distribution diagram of a focus area is shown, such as Figure 8 As shown, the A region in the f-1th frame image is the position of the third image point in the f-1th frame image, and Ovex(x, y, f-1) represents the A region in the f-1th frame image. In this case, the image points in the A region are set to 1, and the image points in the remaining regions are set to 0. Exemplarily, the position of the third image point in the previous frame image is set to (2, 2). Correspondingly, the image point in the current frame image that has a vertical projection relationship and is also located at (2, 2) is determined as the image point in the first region. In this embodiment, this region Ovex(x, y, f-1) is used as the basis to determine the focus region of the current frame image.

[0092] In an optional implementation, determining the second area of ​​the current frame image according to the location information of all image points except the third image point in the current frame image includes:

[0093] The positions of all image points except the third image point in the current frame image are determined as the second area of ​​the current frame image.

[0094] ~Ovex(x, y, f) represents the second area of ​​the current frame image, that is, the area corresponding to the third image point in the frame image is inverted. For example, assuming that the overexposure point threshold is 240, then Ovex(x, y, f) represents the area corresponding to the image point with brightness greater than 240 in the f-th frame image. According to the image shooting sequence, the exposure of this frame image is greater than that of the next frame image, so the area with too high brightness in this frame image (the area corresponding to the third image point) does not need to be processed in the current frame image, but can be exposed in the next frame image (the K-th frame image with lower brightness), thereby obtaining an image area with more appropriate exposure. Figure 8 As shown, Ovex(x, y, f) represents the area B where the complete triangle in the f-th frame image is located. It is inverted, so the values ​​of all image points in the B area change from 1 to 0, and the second area represents all areas outside the triangle B in the f-th frame image.

[0095] In this embodiment, the intersection of the first region and the second region is determined as the first candidate region. Specifically, according to Formula 5, according to the logic of 1&&0=0, Ovex(x, y, f-1)&&~Ovex(x, y, f) represents the intersection of the first region and the second region from the Ovex(x, y, f-1) region (such as Figure 8 In the circular area A in which the image point value is 1), remove the area corresponding to Ovex(x, y, f) (such as Figure 8 The B1 area in the triangle B area in , where the image point value is 0), and the remaining area is the shadow part in the circle A in the f-th frame image.

[0096] In an optional implementation, determining the third area of ​​the current frame image according to the location information of the second image point in the frame image subsequent to the current frame image includes:

[0097] The vertical projection area in the current frame image of the position of the second image point in the frame image after the current frame image is determined as the third area of ​​the current frame image. The vertical projection area represents the area vertically projected in the current frame image through the parallel projection line perpendicular to the current frame image point when the image is parallel to the current frame image.

[0098] Unex2(x, y, f+1) represents the third area of ​​the current frame image, that is, the area in the current frame image corresponding to the position of the second image point in the frame image after the frame image. For example, assuming that the underexposure point threshold is 32, Unex2(x, y, f+1) represents the area corresponding to the image points with brightness greater than or equal to 32 and less than or equal to 64 in the f+1 frame image. According to the image shooting sequence, the exposure of this frame image is greater than that of the next frame image. For the area with lower brightness in the next frame image (the area corresponding to the second image point), it can be exposed in this frame image (the image with higher brightness) to obtain an image area with more appropriate exposure. For the image points outside the Unex2(x, y, f+1) area in the next frame image, that is, the image points with relatively high brightness, they can be processed in the subsequent multiple frames of images (the images with lower brightness) to obtain clearer details, so they can be removed from the focus area of ​​the current frame image.

[0099] In this embodiment, the intersection of the first candidate area and the third area is determined as the second candidate area. Figure 8 As shown, Unex2(x, y, f+1) indicates that the complete rectangle in the f+1th frame image is located in the C region, and the image points in the region have a value of 1, and the image points outside the region have a value of 0. According to the operation logic of 1&&1=1, 1&&0=0, the region outside the C region is removed from the focus region, which is expressed as Figure 8 The intersection of the shadow area in the circle A (the first candidate area) and the rectangular area C (the third area) is the shadow area C1 in the f-th frame image.

[0100] In an optional implementation, determining the fourth area of ​​the current frame image according to the location information of the first image point in the frame image subsequent to the current frame image includes:

[0101] The vertical projection area of ​​the position of the first image point in the frame image after the current frame image is determined as the fourth area of ​​the current frame image. The vertical projection area represents the area vertically projected in the current frame image through parallel projection lines perpendicular to the current frame image points when the image is parallel to the current frame image.

[0102] Unex1(x, y, f+1) represents the fourth area, that is, the area in the f-th frame image corresponding to the position of the first image point in the next frame image of this frame image, so that the image points in this area take the value of 1. Exemplarily, assuming that the underexposure point threshold is 32, then Unex1(x, y, f+1) represents the area corresponding to the image points with brightness less than 32 in the f+1-th frame image. According to the image shooting order, the exposure of this frame image is greater than that of the next frame image. For the area with too low brightness in the next frame image (the area corresponding to the first image point), that is, the area with a brightness value lower than the underexposure point threshold, it can be exposed in this frame image (the image with higher brightness) to obtain an image area with a more appropriate exposure. Figure 8 As shown, Unex1(x, y, f+1) represents the D region where the complete rhombus in the f+1th frame image is located.

[0103] In this embodiment, the union of the second candidate area and the fourth area is determined as the focus area of ​​the current frame image. Figure 8 As shown, the focus area of ​​the f-th frame image finally determined is the C1 area (the second candidate area) of the shaded part of the f-th frame image plus the D area (the fourth area).

[0104] In this embodiment, the area Ovex(x, y, f-1) where the overexposed point of the previous frame of image is located is used as the base ( Figure 8 A area in the current frame image), remove the overexposed area of ​​the current frame image ~ Ovex (x, y, f) ( Figure 8 B1 area in the next frame), remove the area Unex2 (x, y, f+1) with high brightness in the next frame (get Figure 8 C1 area in the next frame), add the area where the underexposed point is located in the next frame image (such as Figure 8 The D area in the image is obtained, and the focus area of ​​the f-th frame image is finally determined (such as Figure 8 If the current frame image is the 1st frame, then the 0th frame is fully exposed and can be omitted here; if the current frame image is the Fth frame, then there is no image with lower exposure to take over the overexposed area of ​​the Fth frame, so the Fth frame image only needs to consider the overexposed area of ​​the F-1th frame image. The following embodiments respectively illustrate the above two special cases (the 1st frame and the Fth frame).

[0105] In an optional implementation, in step S3024-b, for the first frame of the F frames, the focus area of ​​the frame is determined according to the following steps:

[0106] Determine a fifth area of ​​the current frame image according to the image shooting sequence and based on the location information of all image points except the third image point in the current frame image;

[0107] Determine a sixth area of ​​the current frame image according to the location information of the second image point in the frame image after the current frame image;

[0108] Determine a seventh region of the current frame image according to the location information of the first image point in the frame image subsequent to the current frame image;

[0109] The focus area of ​​the current frame image is determined according to the fifth area, the sixth area and the seventh area.

[0110] In this embodiment, for the first frame of image, since the previous frame of image is a fully exposed image and has no reference value, only the brightness information of the next frame of image can be used to determine the focus area of ​​the current frame of image. Specifically, according to the area corresponding to all image points except the third image point in the current frame of image, the fifth area is determined, and the fifth area indicates the location of the image point whose brightness does not exceed the overexposure point threshold in the frame of image; according to the area corresponding to the second image point in the next frame of image, the sixth area is determined, and the sixth area indicates the location of the image point with lower brightness in the next frame of image; according to the area corresponding to the first image point in the next frame of image, the seventh area is determined, and the seventh area indicates the location of the image point with too low brightness in the next frame of image.

[0111] In an optional implementation, determining the focus area of ​​the current frame image according to the fifth area, the sixth area, and the seventh area includes:

[0112] Determine the intersection of the fifth region and the sixth region as a third candidate region;

[0113] The union of the third candidate region and the seventh region is determined as the focus region of the current frame image.

[0114] For the first frame image, that is, the image with f=1, the focus area of ​​the frame image can be determined according to the following formula:

[0115] (Formula 6)Cal_Pixel(x,y,f)=(~Ovex(x,y,f)&&Unex2(x,y,f+1))||Unex1(x,y,f+1);

[0116] In an optional implementation, determining the fifth area of ​​the current frame image according to the location information of all image points except the third image point in the current frame image includes:

[0117] The positions of all image points except the third image point in the current frame image are determined as the fifth area of ​​the current frame image.

[0118] ~Ovex(x, y, f) represents the fifth area of ​​the current frame image, that is, the area corresponding to the third image point in the frame image is inverted. If the current frame image is the first frame, then the zeroth frame is all exposed, and the analysis can be omitted in this embodiment, so the focus area is determined directly from the current frame. According to the image shooting order, the exposure of this frame image is greater than that of the second frame image, so the area with too high brightness in this frame image (the area corresponding to the third image point) can be exposed in the second frame image to obtain an image area with a more appropriate exposure. Therefore, when determining the focus area of ​​the frame image, select the area corresponding to all image points except the third image point, and refer to Fig. 9 , Fig. 9 FIG. 4 shows a schematic diagram of the distribution of the focus area of ​​the first frame image, such as Fig. 9 As shown, the third image point is Fig. 9 The circular E area in , ~Ovex(x, y, f) represents all shadow areas (fifth areas) outside the E area in the first frame image.

[0119] In an optional implementation, determining the sixth area of ​​the current frame image according to the location information of the second image point in the frame image subsequent to the current frame image includes:

[0120] The vertical projection area of ​​the second image point in the next frame image of the current frame image is determined as the sixth area of ​​the current frame image, and the vertical projection area represents the area vertically projected in the current frame image through the parallel projection line perpendicular to the current frame image point when the image is parallel to the current frame image.

[0121] Unex2(x, y, f+1) represents the sixth region, that is, the position region in the current frame image corresponding to the position of the second image point in the next frame image of the frame image, such as Fig. 9 The triangle F area of ​​the second frame image in . According to the image shooting order, the exposure of this frame image is greater than that of the second frame image (i.e., the next frame image). Therefore, the area with lower brightness in the next frame image (the area corresponding to the second image point) can be exposed in this frame image (the image with higher brightness) to obtain an image area with more appropriate exposure. For the image points outside the Unex2 (x, y, f+1) area in the next frame image, that is, the image points with higher brightness, they can be processed in the subsequent multiple frames of images (the images with lower brightness) to obtain clearer details. Therefore, they can be removed from the focus area of ​​the current frame image.

[0122] In this embodiment, the intersection of the fifth region and the sixth region is determined as the third candidate region. Fig. 9As shown, the sixth region Unex2 (x, y, f+1) represents the F region where the triangle of the second frame image is located. The image points in this region have a value of 1, and the image points outside this region have a value of 0. According to the operation logic of 1&&1=1, 1&&0=0, the region outside the F region is removed from the focus region, which is expressed as Fig. 9 The area of ​​shadow F1 in the f+1th frame image (the third candidate area).

[0123] In an optional implementation manner, determining the seventh area of ​​the current frame image according to the location information of the first image point in the frame image subsequent to the current frame image includes:

[0124] The vertical projection area of ​​the first image point in the frame image after the current frame image point is determined as the seventh area of ​​the current frame image, and the vertical projection area represents the area vertically projected in the current frame image through the parallel projection line perpendicular to the current frame image point when the image is parallel to the current frame image.

[0125] |Unex1(x, y, f+1) represents the seventh region, that is, the region in the current frame image corresponding to the first image point position in the next frame image of the frame image, such as Fig. 9 According to the image shooting order, the exposure of this frame image is greater than that of the next frame image. For the area with too low brightness in the next frame image (the area corresponding to the first image point), that is, the area with a brightness value lower than the underexposure point threshold, it can be used as the focus area of ​​this frame image, and exposure processing is performed in this frame image (the image with higher brightness), so as to obtain an image area with more appropriate exposure.

[0126] In this embodiment, the union of the third candidate region and the seventh region is determined as the focus region of the current frame image. Fig. 9 As shown in FIG. 1 , according to the operation logic of 1||1=1, 0||1=1, the area where the rectangle H is located is added as the focus area. The focus area of ​​the first frame is finally determined as follows: Fig. 9 As shown, the area where the shadow F1 in the first frame image is located (the third candidate area) and the area where the rectangle H is located (the seventh area).

[0127] In an optional implementation, step S3024-c, for the Fth frame image in the F frame images, the focus area of ​​the current frame image is determined according to the following steps:

[0128] According to the image shooting sequence, the focus area of ​​the current frame image is determined based on the location information of the third image point in the frame image before the current frame image.

[0129] For the F-th frame image, that is, the image with f=F, the focus area of ​​the frame image can be determined according to the following formula:

[0130] (Formula 7) Cal_Pixel(x, y, f) = Ovex(x, y, f-1);

[0131] In an optional implementation, determining the focus area of ​​the current frame image according to the location information of the third image point in the previous frame image of the current frame image includes:

[0132] The vertical projection area of ​​the third image point in the previous frame image of the current frame image is determined as the focus area of ​​the current frame image. The vertical projection area represents the area vertically projected in the current frame image through a parallel projection line perpendicular to the current frame image point when the image is parallel to the current frame image.

[0133] In this embodiment, according to the image shooting sequence, the exposure of this frame image is less than that of the previous frame image, so for the area with too high brightness in the previous frame image (the area corresponding to the third image point), the exposure processing can be performed in this frame image, so as to obtain an image area with a more appropriate exposure. For the last frame F frame image, there is no image with a lower exposure to take over the overexposed area of ​​the F frame, so the F frame image only needs to consider the overexposed area of ​​the F-1 frame image. Therefore, the area corresponding to the third image point in the previous frame image can be directly determined as the focus area of ​​the frame image.

[0134] In addition, in the above embodiment of calculating the focus area, the brightness calculated for the image points on the right edge and the image points on the bottom edge of each frame image may be wrong. Therefore, when determining the focus area, the image points in the last column and the last row of each frame image may be removed, which can be expressed as the following formula:

[0135] (Formula 8) Wherein, Cal_Pixel(x, y, f)=0 indicates that the area corresponding to the image point is not the focus area of ​​the f-th frame image.

[0136] Step S303: determining a brightness variation coefficient of each frame of the image according to a difference between the brightness of the focus area and the target brightness.

[0137] The target brightness indicates the brightness that each image point needs to reach after the exposure of the frame image is adjusted. In this embodiment, the target brightness can be a preset value. The brightness variation coefficient indicates the coefficient value of the frame image from the current brightness to the target brightness, that is, the coefficient value of the exposure that needs to be adjusted. The larger the brightness variation coefficient, the greater the difference between the current brightness of the frame image and the target brightness, and the greater the exposure that needs to be adjusted; the smaller the brightness variation coefficient, the smaller the difference between the current brightness of the frame image and the target brightness, and the smaller the exposure that needs to be adjusted.

[0138] In an optional implementation, in step S303, the brightness information of the multiple frames of images includes brightness information of image points of each frame of images, and determining the brightness variation coefficient of each frame of images according to the difference between the brightness of the focus area and the target brightness includes:

[0139] Step S3031 : determining the brightness of the focus area of ​​the current frame image according to the image points contained in the focus area of ​​the current frame image and the brightness information of each image point.

[0140] In this embodiment, in order to avoid the erroneous influence of the image points with brightness 0 on the image brightness adjustment, the image points with brightness 0 need to be set to 1, as shown in the following formula:

[0141] (Formula 9)

[0142] According to the focus area Cal_Pixel (x, y, f) of the current frame image determined in step S302, all image points in the focus area can be obtained. Combined with the brightness information of the current frame image, the brightness of the focus area of ​​the current frame image can be determined according to the following formula.

[0143] (Formula 10) Yf(x, y, f) = Yc(x, y, f)*Cal_Pixel(x, y, f); wherein Yf(x, y, f) represents the brightness of the focus area of ​​the f-th frame image, which is the brightness of the focus area finally obtained after calculation, correction and other processing based on the brightness information of the original image.

[0144] Step S3032, determining a loss function according to the target brightness and the brightness of the focus area of ​​the current frame image.

[0145] Step S3033: determine the brightness variation coefficient of the current frame image according to the minimum value of the loss function.

[0146] In this embodiment, the loss function represents the difference between the brightness of the focus area of ​​the current frame image and the target brightness. The smaller the value of the loss function, the closer the brightness of the focus area is to the target brightness. Since the brightness distribution of each frame image is not completely equal, that is, the brightness values ​​of the image points in each frame image are not exactly the same, in order to make the brightness value of each image point in the frame image closer to the target brightness, the loss value of each frame image needs to reach the minimum value, so as to find the brightness change coefficient that minimizes the loss function value.

[0147] In an optional implementation, the loss function is determined according to the following mean square error formula:

[0148] (Formula 11) In this embodiment, the mean square error formula is used as the loss function for calculation, wherein Ytarget is the target brightness, Cal_Pixel (x, y, f) is the focus area of ​​the f-th frame image, and Yf ​​(x, y, f) is the brightness of the focus area of ​​the f-th frame image, that is, the brightness of the focus area finally determined.

[0149] The step S3033, determining the brightness variation coefficient of the current frame image according to the minimum value of the loss function, includes:

[0150] The loss function is derived according to the following formula to determine the brightness change coefficient of the current frame image:

[0151] (Formula 12) Among them, a and b are parameters greater than 0.

[0152] In this embodiment, the goal of automatic exposure adjustment is to minimize the mean square error of each frame. Considering that the loss function is a convex function and that the exposure is proportional to the brightness within a certain range, the coefficient of variation that minimizes the loss function can be directly found by deriving the above loss function. In addition, in Formula 12, parameter b is added to the numerator and parameter a is added to the denominator to avoid abnormal exposure changes caused by the absence of calculation points in the current frame image. Optionally, the value of a can be 2 and the value of b can be 1.

[0153] Step S304: adjusting the current exposure of each frame of image according to the brightness variation coefficient to obtain a target exposure of each frame of image.

[0154] In this embodiment, the current exposure value indicates the exposure value used for capturing the frame image, and the target exposure value indicates the exposure value required for capturing the image next time.

[0155] In an optional embodiment, the method further includes:

[0156] Acquire parameter values ​​used for capturing the multiple frames of images;

[0157] The current exposure value of each frame of the multiple frames of images is calculated according to the parameter value.

[0158] In this embodiment, in order to calculate the current exposure of each frame of image, the parameter values ​​used to shoot the multiple frames of image can be obtained while acquiring the multiple frames of image. The specific parameter values ​​can be integration time, analog gain, aperture size, pixel photosensitive area size, superposition of multiple frames of images with the same exposure, digital gain, high conversion gain (High Comversion Gain, HCG) mode, dual conversion gain (Dual Comversion Gain, DCG) mode and other parameters. Different parameters correspond to different exposure calculation methods, which are not limited in this embodiment.

[0159] In an optional implementation, the parameter value includes an integration time and an analog gain for capturing the multiple frames of images, and calculating the current exposure of each frame of the multiple frames of images according to the parameter value includes:

[0160] The product of the integration time of the current frame image and the analog gain is used as the current exposure amount of the current frame image.

[0161] In this embodiment, the current exposure amount of the current frame image can be calculated according to the following formula using the product of the integration time and the analog gain when the current frame image is shot.

[0162] (Formula 13) Exposure_pre(f)=Int_time_pre(f)*Ana_gain_pre(f); wherein Exposure_pre(f) represents the current exposure of the f-th frame image, Int_time_pre(f) represents the integration time of the f-th frame image, and Int_time_pre(f) represents the analog gain of the f-th frame image.

[0163] In an optional implementation, the parameter value further includes a light source operating frequency and an image maximum exposure time, and the method further includes:

[0164] The maximum exposure step length is determined according to the maximum exposure time of the image and the operating frequency of the light source. Specifically, in order to more accurately calculate the current exposure, this embodiment also takes into account the three variables of the operating frequency of the light source, the image exposure time limit and the analog gain limit. The operating frequency of the light source is the operating frequency of the light source in the scene environment, the image exposure time limit is the maximum exposure time that can be achieved by shooting multiple frames of images, and the analog gain limit is the maximum analog gain that can be achieved by shooting multiple frames of images. Based on the pre-acquired parameters, the maximum exposure step length is determined according to the following formula:

[0165] (Formula 14) Max_step(f)=Int_time_max(f) / Int_time_step(f); wherein the maximum exposure step length Max_step(f) is equal to the quotient of the maximum exposure time of the image Int_time_max(f) and the operating frequency of the light source Int_time_step(f).

[0166] The actual maximum integration time is determined according to the maximum exposure step length, the maximum image exposure time and the light source operating frequency. Specifically, the actual maximum integration time is determined according to the following formula:

[0167] (Formula 15)

[0168] As shown in Formula 15, when the maximum exposure step Max_step(f) is 0, the actual maximum integration time Int_time_max_act(f) is equal to the value of the image maximum exposure time Int_time_max(f); when the maximum exposure step Max_step(f) is not 0, the actual maximum integration time Int_time_max_act(f) is equal to the product of the light source operating frequency Int_time_step(f) and the maximum exposure step Max_step(f).

[0169] The actual maximum exposure is determined according to the actual maximum integration time and the analog gain. Specifically, the actual maximum exposure is determined according to the following formula:

[0170] (Formula 16) Exposure_max(f)=Int_time_max_act(f)*Ana_gain_max(f); wherein the value of the actual maximum exposure Exposure_max(f) is equal to the product of the actual maximum integration time Int_time_max_act(f) and the analog gain Int_time_pre(f).

[0171] The step S304, adjusting the current exposure of each frame of image according to the brightness variation coefficient to obtain the target exposure of each frame of image, includes:

[0172] When the product of the brightness variation coefficient and the current exposure is less than or equal to the actual maximum exposure, the product is determined as the target exposure of the current frame image.

[0173] Specifically, according to the following formula:

[0174] (Formula 17) Exposure(f) = Exposure_pre(f)*Multi(f); The product of the current exposure Exposure_pre(f) and the brightness change coefficient Multi(f) is determined as the target exposure Exposure(f) of the frame image. If the value of the exposure is less than or equal to the actual maximum exposure Exposure_max(f), the value of the exposure can be directly determined as the target exposure Exposure(f) of the current frame image.

[0175] When the product of the brightness variation coefficient and the current exposure is greater than the actual maximum exposure, the actual maximum exposure is determined as the target exposure of the current frame image.

[0176] In this embodiment, the actual maximum exposure represents the maximum exposure that can be achieved when actually shooting an image, so the calculated product cannot exceed the actual maximum exposure Exposure_max(f). If the value of the product is greater than the actual maximum exposure Exposure_max(f), the value of the actual maximum exposure Exposure_max(f) is determined as the target exposure Exposure(f) of the current frame image. Therefore, this embodiment further limits the adjustment range of the exposure by calculating the actual maximum exposure that can be achieved, thereby improving the practicality of the exposure control method.

[0177] Step S305 , shooting according to the target exposure to obtain new multi-frame images.

[0178] In this embodiment, after the target exposure required for each frame of the image is calculated, the image is re-photographed according to the corresponding target exposure, thereby obtaining new multiple frames of images. Exemplarily, for image 1, the target exposure Exposure 1 is calculated, and the image is re-photographed according to the target exposure Exposure 1 to obtain a new image 1; for image 2, the target exposure Exposure 2 is calculated, and the image is re-photographed according to the target exposure 2 to obtain a new image 2, ..., for image F, the target exposure Exposure F is calculated, and the image is re-photographed according to the target exposure Exposure F to obtain a new image F.

[0179] The exposure control method proposed in this embodiment is different from the previous exposure control method based on a single-frame image, and is suitable for automatic exposure control of multiple-frame images for HDR image fusion. The exposure control method of this embodiment simultaneously considers multiple-frame images with different exposure amounts, calculates the focus area of ​​the current exposure image according to the overexposure point threshold and the underexposure point threshold, respectively, calculates the brightness of the focus area according to the required data, and uses a quasi-mean square error to calculate the exposure variation coefficient. Finally, the target exposure amount is obtained according to restrictions such as the operating frequency of the light source, the maximum exposure time, and the analog gain, so that the shot is re-shot according to the target exposure amount, and a suitable exposure image can be quickly obtained in various scenarios. The final multi-frame image is used for HDR image fusion, so that the obtained fused image has better detail expression and a higher dynamic range.

[0180] In an optional implementation, the step S305 of shooting according to the target exposure to obtain new multi-frame images includes:

[0181] Step S3051, determining shooting parameter values ​​according to the target exposure.

[0182] In this embodiment, the parameter values ​​of various parameters required for shooting can be determined according to the target exposure. Specifically, the shooting parameter values ​​may include: integration time, analog gain, aperture size, pixel photosensitive area size, superposition of multiple frames of images with the same exposure, digital gain, and one or more parameter values ​​of HCG / DCG mode.

[0183] In an optional implementation, the operating frequency of the light source used to shoot the multiple frames of images is a fixed light source operating frequency, the shooting parameter value includes the integration time required for shooting, and the step S3051, determining the shooting parameter value according to the target exposure, includes:

[0184] Step S3051-a, determining the step length required for shooting according to the target exposure and the operating frequency of the light source.

[0185] The light source operating frequency in this embodiment may be the light source operating frequency Int_time_step(f) pre-acquired in the above embodiment. The step length required for shooting is determined according to the following formula:

[0186] (Formula 17) Step(f) = Exposure(f) / Int_time_step(f); wherein the quotient of the target exposure value Exposure(f) and the light source operating frequency Int_time_step(f) is determined as the value of the step length Step(f) required for shooting. In addition, it is also necessary to consider the limitation of the predetermined parameter exposure maximum step length Max_step(f). The exposure maximum step length Max_step(ff) represents the maximum step length that can be used for exposure. When the quotient of the target exposure value Exposure(f) and the light source operating frequency Int_time_step(f) exceeds the exposure maximum step length Max_step(f), the value of the exposure maximum step length Max_step(f) is determined as the value of the step length Step(f) required for shooting.

[0187] Step S3051-b, determining the integration time required for the shooting according to the step length required for the shooting and the operating frequency of the light source.

[0188] Specifically, after obtaining the step length required for shooting according to the target exposure mapping, the integration time required for shooting is determined according to the following formula based on the step length required for shooting and the operating frequency of the light source:

[0189] (Formula 18) Int_time(f) = Int_time_step(f)*Step(f); According to Formula 18, the value of the product of the light source operating frequency Int_time_step(f) and the step length Step(f) required for shooting is determined as the integration time required for shooting. In addition, it is also necessary to consider the special case that the value of the step length Step(f) required for shooting is equal to 0 or the value of the maximum exposure step length Max_step(f) is equal to 0.

[0190] In an optional implementation, determining the integration time required for the shooting according to the step length required for the shooting and the operating frequency of the light source includes:

[0191] When the maximum exposure step Max_step(f) used for capturing the multiple frames of images is equal to 0 and the target exposure is greater than the maximum exposure time of the image (Exposure(f)>Int_time_max(f)), the value of the integration time Int_time(f) required for the capturing is equal to the value of the maximum exposure time Int_time_max(f) of the image.

[0192] When the step length Step(f) required for shooting is equal to 0, the value of the integration time Int_time(f) required for shooting is equal to the value of the target exposure Exposure(f).

[0193] In addition, in this embodiment, among the three cases of the integration time required for shooting, the calculation method is selected according to the priority, and the priority of case 1 (the maximum exposure step Max_step(f) is equal to 0, and the target exposure is greater than the maximum exposure time of the image) is greater than the priority of case 2 (the step required for shooting Step(f) is equal to 0). That is, when both cases 1 and 2 exist, the calculation method corresponding to case 1 is preferentially selected (determined as the value of the maximum exposure time Int_time_max(f) of the image) to calculate the integration time required for shooting.

[0194] In an optional implementation, the shooting parameter value also includes an analog gain required for shooting, and the step S3051 of determining the shooting parameter value according to the target exposure also includes:

[0195] Step S3051 - c , determining the analog gain required for the shooting according to the target exposure and the integration time required for the shooting.

[0196] Specifically, the analog gain required for the shooting can be calculated according to the following formula:

[0197] (Formula 19) Ana_gain(f) = Exposure(f) / Int_time(f); wherein the value of the analog gain Ana_gain(f) required for shooting is equal to the product of the target exposure Exposure(f) and the integration time Int_time(f) required for shooting.

[0198] Step S3052: Control the camera to shoot according to the shooting parameter value to obtain the new multi-frame image.

[0199] In this embodiment, the parameter values ​​of various parameters required for shooting can be determined according to the target exposure. The camera is controlled to shoot according to the shooting parameter values ​​to obtain new multi-frame images. Specifically, the shooting parameter values ​​may include: one or more parameter values ​​of the integration time required for shooting, the analog gain required for shooting, the aperture size, the size of the pixel photosensitive area, the superposition of multiple frames of images with the same exposure, the digital gain, and the HCG / DCG mode.

[0200] In an optional embodiment, the method further includes:

[0201] The captured new multi-frame images are used as the multi-frame images of the current batch;

[0202] Determine the target exposure of each of the multiple frames of images in the current batch;

[0203] When a difference between a target exposure of the current batch and a target exposure of a previous batch of multi-frame images is within a preset range, outputting the multi-frame images of the current batch as final multi-frame images;

[0204] When the difference between the target exposure of the current batch and the target exposure of the previous batch of multiple-frame images is not within a preset range, reshooting is performed to obtain the next batch of multiple-frame images.

[0205] In this embodiment, after completing the exposure control of a multi-frame image and obtaining a new multi-frame image, the exposure of the image can be continuously adjusted through continuous loop iterations according to the same exposure control method, so that each frame of the image can eventually reach a suitable exposure. When the multi-frame image is finally used for HDR image fusion, better detail performance and a higher dynamic range can be obtained when the HDR image is synthesized later.

[0206] A second aspect of the present application provides an exposure control device, referring to Figure 6 , Figure 6 A schematic diagram of the structure of an exposure control device is shown. Figure 6 As shown, the device comprises:

[0207] An image acquisition module, used to acquire multiple frames of images and brightness information of the multiple frames of images;

[0208] A focus region determination module, used to determine a focus region of each frame of image by using brightness information of the multiple frames of image, an underexposure point threshold and an overexposure point threshold, wherein the focus region indicates a region in the image that needs to be fully exposed;

[0209] A brightness variation coefficient determination module, used to determine the brightness variation coefficient of each frame of the image according to the difference between the brightness of the focus area and the target brightness;

[0210] A target exposure determination module, used to adjust the current exposure of each frame of image according to the brightness variation coefficient to obtain a target exposure of each frame of image;

[0211] The shooting module is used to shoot according to the target exposure to obtain new multiple-frame images.

[0212] In an optional implementation, the region of interest determination module includes:

[0213] A first image point determination submodule is used to determine, for each frame of the multiple images, an image point in the frame of the image whose brightness value is less than the underexposure point threshold as a first image point according to brightness information of the frame of the image;

[0214] a second image point determination submodule, configured to determine, for each frame of the multiple images, an image point in the frame of the image whose brightness value is greater than or equal to the underexposure point threshold and less than m times the underexposure point threshold as a second image point according to brightness information of the frame of the image, where m is a constant greater than 1;

[0215] A third image point determination submodule is used to determine, for each frame of the multiple images, an image point in the frame of the image whose brightness value is greater than the overexposure point threshold as a third image point according to the brightness information of the frame of the image, and m times the underexposure point threshold is less than the overexposure point threshold;

[0216] The focus area determination submodule is used to determine the focus area of ​​each frame image according to the first image point, the second image point and the third image point of the multiple frames of images.

[0217] In an optional implementation, the exposure amounts of the multiple frames of images are sequentially reduced according to the image shooting order, and the region of interest determination submodule includes:

[0218] A first region of interest determination unit, wherein the plurality of frames of images include F frames of images, and the first region of interest determination unit is configured to determine the region of interest of each frame of images from the 2nd to F-1 frames of the F frames of images according to the image shooting sequence; the first region of interest determination unit includes:

[0219] A first region determination subunit is used to determine the first region of the current frame image according to the image shooting sequence and based on the location information of the third image point in the previous frame image of the current frame image;

[0220] A second region determining subunit, configured to determine a second region of the current frame image according to location information of all image points except the third image point in the current frame image;

[0221] A third region determining subunit, configured to determine a third region of the current frame image according to location information of a second image point in a frame image subsequent to the current frame image;

[0222] A fourth region determining subunit, configured to determine a fourth region of the current frame image according to the location information of the first image point in a frame image subsequent to the current frame image;

[0223] The first focus area determination subunit is used to determine the focus area of ​​the current frame image according to the first area, the second area, the third area and the fourth area.

[0224] In an optional implementation, the first region determination subunit includes:

[0225] The first area determination block is used to determine the vertical projection area of ​​the third image point in the previous frame image of the current frame image as the first area of ​​the current frame image, and the vertical projection area represents the area vertically projected in the current frame image through the parallel projection line perpendicular to the current frame image point when the image is parallel to the current frame image.

[0226] In an optional implementation, the second region determination subunit includes:

[0227] The second region determination block is used to determine the locations of all image points except the third image point in the current frame image as the second region of the current frame image.

[0228] In an optional implementation, the third region determination subunit includes:

[0229] The third area determination block is used to determine the vertical projection area in the current frame image where the second image point is located in the frame image after the current frame image as the third area of ​​the current frame image. The vertical projection area represents the area vertically projected in the current frame image through parallel projection lines perpendicular to the current frame image point when the image is parallel to the current frame image.

[0230] In an optional implementation, the fourth region determination subunit includes:

[0231] The fourth area determination block is used to determine the vertical projection area of ​​the position of the first image point in the frame image after the current frame image as the fourth area of ​​the current frame image. The vertical projection area represents the area vertically projected in the current frame image through parallel projection lines perpendicular to the current frame image points when the image is parallel to the current frame image.

[0232] In an optional implementation, the first region of interest determination subunit includes:

[0233] A first candidate region determination block, used for determining the intersection of the first region and the second region as a first candidate region;

[0234] A second candidate region determination block, used for determining the intersection of the first candidate region and the third region as the second candidate region;

[0235] The first focus region determination block is used to determine the union of the second candidate region and the fourth region as the focus region of the current frame image.

[0236] In an optional implementation, the region of interest determination submodule includes:

[0237] A second region of interest determining unit is used to determine the region of interest of the first frame image in the F frame images; the second region of interest determining unit includes:

[0238] a fifth region determining subunit, configured to determine a fifth region of the current frame image according to the image capturing sequence and based on the position information of all image points except the third image point in the current frame image;

[0239] A sixth region determining subunit, configured to determine a sixth region of the current frame image according to the location information of the second image point in a frame image subsequent to the current frame image;

[0240] A seventh region determination subunit, configured to determine a seventh region of the current frame image according to the location information of the first image point in a frame image subsequent to the current frame image;

[0241] The second region-of-interest determining subunit is configured to determine the region of interest of the current frame image according to the fifth region, the sixth region, and the seventh region.

[0242] In an optional implementation, the fifth region determination subunit includes:

[0243] The fifth region determination block is used to determine the positions of all image points except the third image point in the current frame image as the fifth region of the current frame image.

[0244] In an optional implementation, the sixth region determination subunit includes:

[0245] The sixth area determination block is used to determine the vertical projection area of ​​the second image point in the next frame image of the current frame image as the sixth area of ​​the current frame image, wherein the vertical projection area represents the area vertically projected in the current frame image through the parallel projection line perpendicular to the current frame image point when the image is parallel to the current frame image.

[0246] In an optional implementation, the seventh region determination subunit includes:

[0247] The seventh area determination block is used to determine the vertical projection area of ​​the first image point in the next frame image of the current frame image as the seventh area of ​​the current frame image, wherein the vertical projection area represents the area vertically projected in the current frame image through the parallel projection line perpendicular to the current frame image point when the image is parallel to the current frame image.

[0248] In an optional implementation, the second region of interest determination subunit includes:

[0249] A third candidate region determination block, used for determining the intersection of the fifth region and the sixth region as a third candidate region;

[0250] The second focus region determination block is used to determine the union of the third candidate region and the seventh region as the focus region of the current frame image.

[0251] In an optional implementation, the third region of interest determining unit is configured to determine the region of interest of the Fth frame image among the F frame images according to the following steps:

[0252] The focus area of ​​the frame image is determined according to the location information of the third image point in the frame image before the frame image.

[0253] In an optional implementation, the third region of interest determination unit includes:

[0254] The third focus area determination subunit is used to determine the vertical projection area of ​​the third image point in the previous frame image of the current frame image as the focus area of ​​the current frame image, and the vertical projection area represents the area vertically projected in the current frame image through the parallel projection line perpendicular to the current frame image point when the image is parallel to the current frame image.

[0255] In an optional implementation, the brightness information of the multiple frames of images includes brightness information of image points of each frame of images, and the brightness variation coefficient determination module includes:

[0256] A first brightness variation coefficient determination submodule, used to determine the brightness of the focus area of ​​the current frame image according to the image points contained in the focus area of ​​the current frame image and the brightness information of each image point;

[0257] A second brightness variation coefficient determination submodule, used to determine a loss function according to the target brightness and the brightness of the focus area of ​​the current frame image;

[0258] The third brightness change coefficient determination submodule is used to determine the brightness change coefficient of the current frame image according to the minimum value of the loss function.

[0259] In an optional implementation, the loss function is determined according to the following mean square error formula:

[0260] Wherein, Ytar get is the target brightness, Cal_Pixel (x, y, f) is the focus area of ​​the f-th frame image, and Yf ​​(x, y, f) is the brightness of the focus area of ​​the f-th frame image;

[0261] The third brightness variation coefficient determination submodule includes:

[0262] The derivation unit is used to derive the loss function according to the following formula to determine the brightness change coefficient of the current frame image:

[0263] Among them, a and b are parameters greater than 0.

[0264] In an optional implementation, the target exposure determination module includes:

[0265] A first determination submodule, configured to determine the product of the brightness variation coefficient and the current exposure as the target exposure of the current frame image when the product of the brightness variation coefficient and the current exposure is less than or equal to the actual maximum exposure;

[0266] The second determination submodule is used to determine the actual maximum exposure as the target exposure of the current frame image when the product of the brightness change coefficient and the current exposure is greater than the actual maximum exposure.

[0267] In an optional embodiment, the device further comprises:

[0268] A parameter value acquisition module, used to acquire parameter values ​​used for capturing the multiple frames of images;

[0269] The current exposure determination module is used to calculate the current exposure of each frame of the multiple frames of images according to the parameter value.

[0270] In an optional implementation, the parameter value includes an integration time and an analog gain for capturing the multiple frames of images, and the current exposure determination module includes:

[0271] The first current exposure determination submodule is used to take the product of the integration time of the current frame image and the analog gain as the current exposure of the current frame image.

[0272] In an optional implementation, the parameter value further includes a light source operating frequency and an image maximum exposure time, and the device further includes:

[0273] An exposure maximum step length determination module, used to determine the exposure maximum step length according to the image maximum exposure time and the light source operating frequency;

[0274] An actual maximum integration time determination module, used to determine the actual maximum integration time according to the maximum exposure step length, the maximum image exposure time and the light source operating frequency;

[0275] The actual maximum exposure determination module is used to determine the actual maximum exposure according to the actual maximum integration time and the analog gain.

[0276] In an optional implementation, the shooting module includes:

[0277] A shooting parameter value determination submodule, used to determine the shooting parameter value according to the target exposure;

[0278] The shooting submodule is used to control the camera to shoot according to the shooting parameter value to obtain the new multi-frame image.

[0279] In an optional implementation, the operating frequency of the light source used to shoot the multiple frames of images is a fixed light source operating frequency, the shooting parameter value includes an integration time required for shooting, and the shooting parameter value determination submodule includes:

[0280] A unit for determining a step length required for shooting, used to determine a step length required for shooting according to the target exposure and the operating frequency of the light source;

[0281] The unit for determining the integration time required for shooting is used to determine the integration time required for shooting according to the step length required for shooting and the working frequency of the light source.

[0282] In an optional implementation, the unit for determining the integration time required for shooting includes:

[0283] A first determining subunit is used for, when the maximum exposure step length used for shooting the multiple frames of images is equal to 0 and the target exposure is greater than the maximum exposure time of the image, the integration time required for shooting is equal to the maximum exposure time of the image;

[0284] The second determining subunit is used for, when the step length required for shooting is equal to 0, the integration time required for shooting is equal to the target exposure.

[0285] In an optional implementation manner, the shooting parameter value further includes an analog gain required for shooting, and the shooting parameter value determination submodule further includes:

[0286] The analog gain determination unit required for shooting is used to determine the analog gain required for shooting according to the target exposure and the integration time required for shooting.

[0287] In an optional embodiment, the device further comprises:

[0288] A current batch determination module, used for taking the captured new multi-frame images as the multi-frame images of the current batch;

[0289] An exposure determination module, used to determine a target exposure for each of the multiple frames of images in the current batch;

[0290] An output module, configured to output the multi-frame images of the current batch as final multi-frame images when the difference between the target exposure of the current batch and the target exposure of the multi-frame images of the previous batch is within a preset range;

[0291] The loop module is used for reshooting to obtain the next batch of multi-frame images when the difference between the target exposure of the current batch and the target exposure of the previous batch of multi-frame images is not within a preset range.

[0292] The third aspect of the embodiment of the present application further provides an HDR image fusion method, referring to Figure 7 , Figure 7 A flowchart of the steps of an HDR image fusion method is shown, such as Figure 7 As shown, the method includes:

[0293] Step S701, acquiring a plurality of frames of images, wherein the plurality of frames of images are obtained by the exposure control method according to any one of the first aspect of the embodiments of the present application;

[0294] Step S702: Use the multiple frames of images to perform HDR image fusion to obtain a fused image.

[0295] A fourth aspect of the embodiments of the present application further provides a computing and processing device, including:

[0296] a memory having computer readable code stored therein; and

[0297] One or more processors, when the computer readable code is executed by the one or more processors, the computing and processing device executes the exposure control method as described in any one of the first aspects of the embodiments of the present application, or the computing and processing device executes the HDR image fusion method as described in the third aspect of the embodiments of the present application.

[0298] The fifth aspect of the embodiments of the present application further provides a computer-readable medium on which a computer program / instruction is stored, and the computer program / instruction is executed by a processor as the exposure control method described in any one of the first aspect of the embodiments of the present application, or the computing processing device executes the HDR image fusion method described in the third aspect of the embodiments of the present application.

[0299] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0300] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0301] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0302] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0303] The above is a detailed introduction to an exposure control method, device, HDR image fusion method and product provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An exposure control method, characterized in that: The method comprises: Acquire multiple frames of images and brightness information of the multiple frames of images; Determine a focus area of ​​each frame of image by using brightness information of the multiple frames of image, an underexposure point threshold, and an overexposure point threshold, wherein the focus area represents an area in the image that needs to be fully exposed; Determining a brightness variation coefficient of each frame of the image according to a difference between the brightness of the focus area and the target brightness; According to the brightness variation coefficient, the current exposure of each frame of image is adjusted to obtain the target exposure of each frame of image; Shooting is performed according to the target exposure amount to obtain new multi-frame images.

2. The exposure control method according to claim 1, characterized in that: The method of determining the focus area of ​​each frame of image by using the brightness information of the multiple frames of image, the underexposure point threshold and the overexposure point threshold, comprises: For each frame of the multiple frames of images, according to brightness information of the frame of image, an image point in the frame of image whose brightness value is less than the underexposure point threshold is determined as a first image point; For each frame of the multiple frames of images, according to brightness information of the frame of image, an image point in the frame of image whose brightness value is greater than or equal to the underexposure point threshold and less than m times the underexposure point threshold is determined as a second image point, where m is a constant greater than 1; For each frame of the multiple frames of images, according to the brightness information of the frame of image, an image point in the frame of image whose brightness value is greater than the overexposure point threshold is determined as a third image point; wherein m times the underexposure point threshold is less than the overexposure point threshold; A focus area of ​​each frame of image is determined according to the first image point, the second image point and the third image point of the multiple frames of image.

3. The exposure control method according to claim 2, characterized in that: The exposure amounts of the multiple frames of images are sequentially reduced according to the image shooting order, and the determining the focus area of ​​each frame of image according to the first image point, the second image point, and the third image point of the multiple frames of images includes: The multiple frames of images include F frames of images. According to the image shooting sequence, for each frame of images from the 2nd to the F-1st frame of images in the F frames of images, the focus area of ​​the current frame of image is determined according to the following steps: Determine the first area of ​​the current frame image according to the image shooting sequence and based on the location information of the third image point in the previous frame image of the current frame image; Determine a second area of ​​the current frame image according to location information of all image points except the third image point in the current frame image; Determine a third area of ​​the current frame image according to the location information of the second image point in the frame image after the current frame image; Determine a fourth area of ​​the current frame image according to the location information of the first image point in the frame image subsequent to the current frame image; A focus area of ​​the current frame image is determined according to the first area, the second area, the third area, and the fourth area.

4. The exposure control method according to claim 3, characterized in that: The determining the first area of ​​the current frame image according to the location information of the third image point in the previous frame image of the current frame image includes: The vertical projection area of ​​the third image point in the previous frame image of the current frame image is determined as the first area of ​​the current frame image. The vertical projection area represents the area vertically projected in the current frame image through the parallel projection line perpendicular to the current frame image point when the image is parallel to the current frame image.

5. The exposure control method according to claim 3, characterized in that: The determining the second area of ​​the current frame image according to the location information of all image points except the third image point in the current frame image includes: The positions of all image points except the third image point in the current frame image are determined as the second area of ​​the current frame image.

6. The exposure control method according to claim 3, characterized in that: The determining the third area of ​​the current frame image according to the location information of the second image point in the frame image after the current frame image includes: The vertical projection area in the current frame image of the position of the second image point in the frame image after the current frame image is determined as the third area of ​​the current frame image. The vertical projection area represents the area vertically projected in the current frame image through the parallel projection line perpendicular to the current frame image point when the image is parallel to the current frame image.

7. The exposure control method according to claim 3, characterized in that: The determining the fourth area of ​​the current frame image according to the location information of the first image point in the frame image subsequent to the current frame image includes: The vertical projection area of ​​the position of the first image point in the frame image after the current frame image is determined as the fourth area of ​​the current frame image. The vertical projection area represents the area vertically projected in the current frame image through parallel projection lines perpendicular to the current frame image points when the image is parallel to the current frame image.

8. The exposure control method according to claim 3 or 4, characterized in that: The determining the focus area of ​​the current frame image according to the first area, the second area, the third area, and the fourth area includes: Determine the intersection of the first region and the second region as a first candidate region; Determine the intersection of the first candidate area and the third area as the second candidate area; The union of the second candidate area and the fourth area is determined as the focus area of ​​the current frame image.

9. The exposure control method according to claim 3, characterized in that: For the first frame of the F frame images, the focus area of ​​the current frame image is determined according to the following steps: Determine a fifth area of ​​the current frame image according to the image shooting sequence and based on the location information of all image points except the third image point in the current frame image; Determine a sixth area of ​​the current frame image according to the location information of the second image point in the frame image subsequent to the current frame image; Determine a seventh region of the current frame image according to the location information of the first image point in the frame image subsequent to the current frame image; The focus area of ​​the current frame image is determined according to the fifth area, the sixth area and the seventh area.

10. The exposure control method according to claim 9, characterized in that: The step of determining the fifth area of ​​the current frame image according to the location information of all image points except the third image point in the current frame image includes: The positions of all image points except the third image point in the current frame image are determined as the fifth area of ​​the current frame image.

11. The exposure control method according to claim 9, characterized in that: The determining the sixth area of ​​the current frame image according to the position information of the second image point in the frame image after the current frame image includes: The vertical projection area of ​​the second image point in the next frame image of the current frame image is determined as the sixth area of ​​the current frame image, and the vertical projection area represents the area vertically projected in the current frame image through the parallel projection line perpendicular to the current frame image point when the image is parallel to the current frame image.

12. The exposure control method according to claim 9, characterized in that: The determining the seventh area of ​​the current frame image according to the location information of the first image point in the frame image after the current frame image includes: The vertical projection area of ​​the first image point in the frame image after the current frame image point is determined as the seventh area of ​​the current frame image, and the vertical projection area represents the area vertically projected in the current frame image through the parallel projection line perpendicular to the current frame image point when the image is parallel to the current frame image.

13. The exposure control method according to claim 9, characterized in that: The determining the focus area of ​​the current frame image according to the fifth area, the sixth area, and the seventh area includes: Determine the intersection of the fifth region and the sixth region as a third candidate region; The union of the third candidate region and the seventh region is determined as the focus region of the current frame image.

14. The exposure control method according to claim 3, characterized in that: For the Fth frame image among the F frame images, the focus area of ​​the current frame image is determined according to the following steps: According to the image shooting sequence, the focus area of ​​the current frame image is determined based on the location information of the third image point in the frame image before the current frame image.

15. The exposure control method according to claim 14, characterized in that: The determining the focus area of ​​the current frame image according to the location information of the third image point in the previous frame image of the current frame image includes: The vertical projection area of ​​the third image point in the previous frame image of the current frame image is determined as the focus area of ​​the current frame image. The vertical projection area represents the area vertically projected in the current frame image through a parallel projection line perpendicular to the current frame image point when the image is parallel to the current frame image.

16. The exposure control method according to claim 1, characterized in that: The brightness information of the multiple frames of images includes brightness information of image points of each frame of images, and determining the brightness variation coefficient of each frame of images according to the difference between the brightness of the focus area and the target brightness includes: Determine the brightness of the focus area of ​​the current frame image according to the image points contained in the focus area of ​​the current frame image and the brightness information of each image point; Determine a loss function according to the target brightness and the brightness of the focus area of ​​the current frame image; The brightness variation coefficient of the current frame image is determined according to the minimum value of the loss function.

17. The exposure control method according to claim 16, characterized in that: The loss function is determined according to the following mean square error formula: Wherein, Ytar get is the target brightness, Cal_Pixel (x, y, f) is the focus area of ​​the f-th frame image, and Yf ​​(x, y, f) is the brightness of the focus area of ​​the f-th frame image; Determining the brightness change coefficient of the current frame image according to the minimum value of the loss function includes: The loss function is derived according to the following formula to determine the brightness change coefficient of the current frame image: Among them, a and b are parameters greater than 0.

18. The exposure control method according to claim 1, characterized in that: The step of adjusting the current exposure of each frame of image according to the brightness variation coefficient to obtain the target exposure of each frame of image includes: When the product of the brightness variation coefficient and the current exposure is less than or equal to the actual maximum exposure, the product is determined as the target exposure of the current frame image; When the product of the brightness variation coefficient and the current exposure is greater than the actual maximum exposure, the actual maximum exposure is determined as the target exposure of the current frame image.

19. The exposure control method according to claim 1, characterized in that: The method further comprises: Acquire parameter values ​​used for capturing the multiple frames of images; The current exposure value of each frame of the multiple frames of images is calculated according to the parameter value.

20. The exposure control method according to claim 19, characterized in that: The parameter values ​​include an integration time and an analog gain for capturing the multiple frames of images, and the current exposure of each frame of the multiple frames of images is calculated based on the parameter values, including: The product of the integration time of the current frame image and the analog gain is used as the current exposure amount of the current frame image.

21. The exposure control method according to claim 20, characterized in that: The parameter value also includes the light source operating frequency and the image maximum exposure time. The method also includes: Determine a maximum exposure step length according to the maximum exposure time of the image and the operating frequency of the light source; Determine the actual maximum integration time according to the maximum exposure step length, the maximum image exposure time and the operating frequency of the light source; The actual maximum exposure is determined according to the actual maximum integration time and the analog gain.

22. The exposure control method according to claim 1, characterized in that: The step of shooting according to the target exposure to obtain new multiple-frame images includes: Determining shooting parameter values ​​according to the target exposure; According to the shooting parameter value, the camera is controlled to shoot to obtain the new multi-frame images.

23. The exposure control method according to claim 22, characterized in that: The operating frequency of the light source used for shooting the multiple frames of images is a fixed operating frequency of the light source, the shooting parameter value includes the integration time required for shooting, and the determining of the shooting parameter value according to the target exposure includes: Determining a step length required for shooting according to the target exposure and the operating frequency of the light source; The integration time required for the shooting is determined according to the step length required for the shooting and the operating frequency of the light source.

24. The exposure control method according to claim 23, characterized in that: The step of determining the integration time required for the shooting according to the step length required for the shooting and the operating frequency of the light source comprises: When the maximum exposure step length used for shooting the multiple frames of images is equal to 0 and the target exposure is greater than the maximum exposure time of the image, the integration time required for shooting is equal to the maximum exposure time of the image; When the step length required for shooting is equal to 0, the integration time required for shooting is equal to the target exposure.

25. The exposure control method according to claim 22, characterized in that: The shooting parameter value also includes an analog gain required for shooting, and the determining of the shooting parameter value according to the target exposure also includes: The analog gain required for the shooting is determined according to the target exposure and the integration time required for the shooting.

26. The exposure control method according to claim 1, characterized in that: The method further comprises: The captured new multi-frame images are used as the multi-frame images of the current batch; Determine the target exposure of each of the multiple frames of images in the current batch; When a difference between a target exposure of the current batch and a target exposure of a previous batch of multi-frame images is within a preset range, outputting the multi-frame images of the current batch as final multi-frame images; When the difference between the target exposure of the current batch and the target exposure of the previous batch of multiple-frame images is not within a preset range, reshooting is performed to obtain the next batch of multiple-frame images.

27. An exposure control device, characterized in that: The device comprises: An image acquisition module, used to acquire multiple frames of images and brightness information of the multiple frames of images; A focus region determination module, used to determine a focus region of each frame of image by using brightness information of the multiple frames of image, an underexposure point threshold and an overexposure point threshold, wherein the focus region indicates a region in the image that needs to be fully exposed; A brightness variation coefficient determination module, used to determine the brightness variation coefficient of each frame of the image according to the difference between the brightness of the focus area and the target brightness; A target exposure determination module, used to adjust the current exposure of each frame of image according to the brightness variation coefficient to obtain a target exposure of each frame of image; The shooting module is used to shoot according to the target exposure to obtain new multiple-frame images.

28. A HDR image fusion method, characterized in that: The method comprises: Acquire a plurality of frames of images, wherein the plurality of frames of images are obtained by the exposure control method according to any one of claims 1 to 26; The multiple frames of images are used to perform HDR image fusion to obtain a fused image.

29. A computing device, characterized in that: include: a memory having computer readable code stored therein; as well as One or more processors, when the computer readable code is executed by the one or more processors, the computing and processing device executes the exposure control method as described in any one of claims 1-26, or the computing and processing device executes the HDR image fusion method as described in claim 28.

30. A computer-readable storage medium, characterized in that: A computer program / instruction is stored thereon, and the computer program / instruction is executed by the processor to implement the exposure control method as described in any one of claims 1 to 26, or the computing processing device implements the HDR image fusion method as described in claim 28.

Images