A method for acquiring a photographed picture, an electronic device, and a storage medium

Abstract

Embodiments of the present application disclose a method for obtaining a photographed picture in a multi-frame exposure shooting mode, an electronic device and a storage medium. The method comprises: obtaining overexposure information of a photographed picture, and determining multi-frame exposure parameters according to the overexposure information; performing multi-frame exposure according to the multi-frame exposure parameters, and obtaining corresponding multi-frame images; determining a first image according to a long-frame image in the multi-frame images, and determining a second image according to a short-frame image in the multi-frame images; and superimposing the first image and the second image to obtain the photographed picture. The overexposure information comprises brightness information of a pixel point whose brightness exceeds an overexposure threshold. The multi-frame exposure parameters comprise a total number of frames of multi-frame exposure, a number of short frames and a number of long frames. The scheme of the embodiments of the present application utilizes the characteristics of long-frame and short-frame images in the multi-frame exposure shooting mode, so as to guarantee that different types of targets in the fused photographed picture all have high image quality.

Description

Technical Field

[0001] This invention relates to, but is not limited to, the field of camera device control, and particularly to a method, electronic device, and storage medium for acquiring captured images in a multi-frame exposure shooting mode. Background Technology

[0002] In nighttime surveillance applications using cameras, the lighting along the road is often uneven due to the influence of road lights, trees, and billboards. Pedestrians, people riding electric bikes, or cyclists, who are usually located on the side of the road, are easily blocked from the light by trees, billboards, or road signs, resulting in poor image quality of captured faces or bodies. Therefore, it is crucial to assess the image quality of captured license plates, faces, or bodies and to adjust it to ensure that the captured faces, bodies, and license plates are of good quality in mixed traffic mode. Summary of the Invention

[0003] This disclosure provides a method, electronic device, and storage medium for acquiring captured images in a multi-frame exposure shooting mode. The total number of frames, the number of long frames, and the number of short frames during multi-frame exposure are dynamically determined based on the overexposure of the captured image. The corresponding number of long frame images and short frame images are acquired, and then the long and short frame images are fused to obtain the captured image. This method can utilize the characteristics of long and short frame images to ensure that different types of target objects in relevant business scenarios have the best image quality in the fused captured image.

[0004] On one hand, embodiments of this disclosure provide a method for acquiring captured images, applied in a multi-frame exposure shooting mode, including:

[0005] Obtain overexposure information of the captured image, and determine multi-frame exposure parameters based on the overexposure information;

[0006] Perform multi-frame exposure based on the multi-frame exposure parameters to obtain the corresponding multi-frame images;

[0007] The first image is determined based on the long frame image in the multi-frame image, and the second image is determined based on the short frame image in the multi-frame image;

[0008] The captured image is obtained by superimposing the first image and the second image;

[0009] The overexposure information includes: brightness information of pixels whose brightness exceeds the overexposure threshold; the multi-frame exposure parameters include: the total number of frames, the number of short frames, and the number of long frames in the multi-frame exposure.

[0010] On the other hand, embodiments of this disclosure also provide an electronic device, including:

[0011] One or more processors;

[0012] Storage device for storing one or more programs.

[0013] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for acquiring captured images as described in any embodiment of this disclosure.

[0014] On the other hand, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for acquiring captured images as described in any embodiment of this disclosure.

[0015] The solution provided by this invention can fully utilize the characteristics of long and short frame images in a multi-frame exposure scheme to generate a fused image, effectively improving the image quality of captured images. In some exemplary embodiments, the image of a region containing a specific target is quality-assessed and enhanced, and the captured image is generated by fusing and overlaying the enhanced region image, ensuring further improvement in image quality.

[0016] After reading and understanding the accompanying diagrams and detailed descriptions, the other aspects can be understood. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a method for acquiring captured images provided by an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of screen partitioning in an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of a segmented histogram in an embodiment of the present invention;

[0021] Figure 4 This is a flowchart of another method for acquiring captured images provided by an embodiment of the present invention.

[0022] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0024] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0025] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0026] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0027] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0028] In nighttime surveillance applications using cameras, the lighting along the road is uneven due to the influence of streetlights, trees, and billboards. Pedestrians, those riding electric bikes, or cyclists, typically positioned on the roadside, are easily obstructed by trees, billboards, or road signs, resulting in poor facial or body capture. Vehicles, on the other hand, generally travel in the middle of the road where lighting is relatively better. For this mixed-traffic scenario, current technical solutions employ a two-device approach, each with adaptive shooting parameters, to capture two separate images: one for faces / bodies and another for license plates / vehicles. However, using two separate images makes subsequent searching and use inconvenient and also creates significant storage pressure.

[0029] To address the aforementioned mixed-mode scenarios, this disclosure provides a method for acquiring captured images, applicable to multi-frame exposure shooting mode. This method enables the aggregation of multiple high-quality target images onto a single captured image, improving the convenience of post-processing of captured images and significantly reducing the storage pressure on captured images.

[0030] Before introducing the relevant embodiments, several aspects involved in the embodiments of this disclosure will be described first:

[0031] Wide Dynamic Range (WDR) technology refers to capturing multiple frames of the same scene with varying exposure levels and then combining them into a single frame. Frames with longer exposures are called long-exposure frames because the longer exposure time effectively preserves information in dark areas. Frames with shorter exposures are called short-exposure frames because the shorter exposure time effectively preserves information in bright areas. By combining the bright area information from short frames and the dark area information from long frames into a single frame, WDR technology can simultaneously capture both bright and dark area information.

[0032] Multi-frame exposure, also known as multiple exposure, can obtain corresponding multi-frame images using a sensor that supports multi-frame exposure. For example, for n-frame exposures, the n frames obtained by sorting them according to their exposure duration are denoted as F1, F2, F3, ..., Fn, where n>=2. Among them, the frames with an exposure duration greater than a set exposure duration threshold are called long frames, and those less than or equal to the set exposure duration threshold are called short frames; or, the frames with an exposure amount greater than a set threshold F are called long frames, and those less than or equal to the threshold F are called short frames. In the n-frame images, the frames with larger exposure amounts correspond to longer exposure times, and the frames with smaller exposure amounts correspond to shorter exposure times. The frame with the largest exposure amount (longest exposure time) in the n frames is called the longest frame or the longest frame image, and the frame with the second largest exposure amount (exposure duration) is called the second longest frame or the second longest frame image; the frame with the smallest exposure amount (shortest exposure time) in the n frames is called the shortest frame or the shortest frame image, and the frame with the second smallest exposure amount (exposure duration) is called the second shortest frame or the second shortest frame image; the definitions of other frames follow the same pattern and will not be elaborated further.

[0033] Exposure ratio, or R, is the ratio of the exposure (amount) of different frames on an image sensor. In multi-frame exposure schemes, the exposure ratio between different long frames, the exposure ratio between long and short frames, and the exposure ratio between short frames can be set to the same value, set separately, or set to different values ​​according to set rules or functions.

[0034] Brightness, also known as intensity or lightness, refers to the human eye's perception of changes in the lightness or darkness of an object's surface; in some embodiments, grayscale values ​​can be used to indicate brightness.

[0035] Overexposure refers to excessive exposure, meaning that the brightness of the image (pixels) exceeds the set overexposure threshold; overexposed area refers to the area in the image composed of pixels whose brightness (grayscale) is greater than the overexposure threshold.

[0036] This disclosure provides a method for acquiring captured images, applicable to multi-frame exposure shooting mode, such as... Figure 1 As shown, including,

[0037] Step 101: Obtain overexposure information of the captured image, and determine multi-frame exposure parameters based on the overexposure information; wherein, the overexposure information includes: brightness information of pixels whose brightness exceeds the overexposure threshold, and the multi-frame exposure parameters include: the total number of frames, the number of short frames, and the number of long frames in the multi-frame exposure.

[0038] Step 102: Perform multi-frame exposure according to the multi-frame exposure parameters to obtain the corresponding multi-frame images;

[0039] Step 103: Determine the first image based on the long frame image in the multi-frame image, and determine the second image based on the short frame image in the multi-frame image;

[0040] Step 104: Superimpose the first image and the second image to obtain the captured image.

[0041] In some exemplary embodiments, step 102 includes:

[0042] Based on the first type of target, exposure is performed according to the number of long frames N to obtain the corresponding number of long frame images;

[0043] Based on the number of short frames k, exposure is performed on the second type of target to obtain the corresponding number of short frame images.

[0044] In some exemplary embodiments, the first type of target is a target in a nighttime environment with relatively poor lighting and / or relatively low movement speed, such as a face or a human body; the second type of target is a target in a nighttime environment with relatively good lighting and / or relatively high movement speed, such as a license plate or a vehicle (body). The license plate can be a car license plate or a motorcycle license plate. Based on the above examples, the first type of target and the second type of target can be determined according to the characteristics of the application scenario, and are not limited to the aspects of the embodiments of this disclosure.

[0045] In some exemplary embodiments, the first type of target includes one or more; the second type of target includes one or more. For example, the first type of target includes: a face and / or a human body; the second type of target includes: a license plate and / or a vehicle.

[0046] It should be noted that the total number of frames M (M = N + k) in the multi-frame exposure is less than or equal to the maximum number of exposure frames supported by the sensor. The camera can perform exposures simultaneously to obtain the aforementioned M frames. This can be achieved by executing a relevant multi-frame exposure scheme, but the specific scheme is not within the scope of protection and limitation of this disclosure. Those skilled in the art will understand that this scheme can obtain multiple frames of images for the same scene using the same camera through multi-frame exposure.

[0047] In some exemplary embodiments, when the first type of target includes a face and / or a human body, and the second type of target includes a license plate and / or a vehicle, in step 102, multiple frames are exposed based on the face and / or human body according to the number of long frames N, to obtain N long frame images with good image quality for the face and / or human body portion; and multiple frames are exposed based on the license plate and / or vehicle according to the number of short frames k, to obtain k short frame images with good image quality for the license plate and / or vehicle portion. It can be understood that high-quality images of different types of targets can be obtained from these two types of frame images (long frame images and short frame images). The fused image obtained by merging and overlaying these images can take into account the image quality of different types of targets in the same scene, improving the convenience of subsequent recognition processing of different types of targets based on the fused image and accelerating the efficiency of related subsequent recognition processing. This ensures that in scenarios with mixed pedestrian and vehicle traffic, the image quality of the face, human body, license plate, and vehicle in the final captured image is relatively good, meeting the needs of subsequent business processing.

[0048] It should be noted that in step 102, when acquiring multiple frames in the multi-frame exposure shooting mode, in some embodiments, the exposure ratio between long and short frames, between long frames, and between short frames is R; or, the exposure ratio between long and short frames, between long frames, and between short frames may be different.

[0049] In some exemplary embodiments, step 103 involves identifying the first type of target based on a long frame image;

[0050] If the recognition result includes a first type of target, the long frame image in the multi-frame image is exposed based on the region where the first type of target is located to obtain at least one long frame image with exposure adjustment; based on the at least one long frame image with exposure adjustment, the recognition result of the first type of target is obtained respectively; based on the recognition result of the first type of target recognition, the recognition result quality score is calculated, and the long frame image with the highest score is determined as the first image;

[0051] or,

[0052] If the recognition result includes multiple first-type targets, the long frame image in the multi-frame image is exposed and adjusted based on the average brightness and / or weighted brightness of the area where the multiple first-type targets are located, to obtain at least one exposure-adjusted long frame image; based on the at least one exposure-adjusted long frame image, the recognition result of recognizing multiple first-type targets is obtained respectively; based on the recognition result quality score of the recognition result of multiple first-type targets, the long frame image with the highest score is determined as the first image;

[0053] The long frame image is either the longest or the second longest frame image among all long frame images.

[0054] It should be noted that the long frame image mentioned can also be any other image among N long frame images, as long frame images that meet the image quality requirements for further identification of the first type of target are acceptable. Exposure adjustment and recognition result quality scoring of the long frame image can be performed by adjusting all N long frame images and then selecting the first image; alternatively, a subset of images can be selected from the N long frame images for adjustment and then the first image can be selected; the method can be flexibly determined according to the relevant needs of the actual application. Step 103 employs a relevant AI recognition scheme to identify one or more first type of targets and to evaluate the quality of the recognition results to obtain a corresponding recognition result quality score. Specific recognition and evaluation schemes are not within the scope of protection and limitation of this disclosure.

[0055] In some exemplary embodiments, step 103, determining the second image based on the short frame image among the multiple frame images, includes:

[0056] Identification of the first type of target based on a short frame image;

[0057] If the recognition result includes a second type of target, the exposure of the short frame image in the multi-frame image is adjusted based on the region where the second type of target is located to obtain at least one exposure-adjusted short frame image; based on the at least one exposure-adjusted short frame image, the recognition result of a second type of target is obtained respectively; based on the recognition result of the second type of target recognition, the recognition result quality score is calculated, and the short frame image with the highest score is determined as the second image;

[0058] or,

[0059] If the recognition result includes multiple second-type targets, the exposure of short-frame images in the multi-frame images is adjusted based on the average brightness and / or weighted brightness of the regions where the multiple second-type targets are located, to obtain at least one exposure-adjusted short-frame image; based on the at least one exposure-adjusted short-frame image, the recognition results for recognizing multiple second-type targets are obtained respectively; based on the recognition results for recognizing multiple second-type targets, a recognition result quality score is calculated, and the short-frame image with the highest score is determined as the second image;

[0060] The short frame image is either the shortest or the second shortest frame image among all short frame images.

[0061] It should be noted that the short frame image mentioned can also be any other image among the k short frame images, as long as it meets the image quality requirements for further identification of the second type of target. Exposure adjustment and recognition result quality scoring of the short frame image can be performed by adjusting all k short frame images and then selecting the second image; alternatively, a subset of images can be selected from the k short frame images for adjustment, and then the second image can be selected; the method can be flexibly determined according to the relevant needs of the actual application. Step 103 employs a relevant AI recognition scheme to identify one or more second type targets and to evaluate the quality of the recognition results to obtain a corresponding recognition result quality score. Specific recognition and evaluation schemes are not within the scope of protection and limitation of this disclosure.

[0062] In some exemplary embodiments, step 104, which involves superimposing the first image and the second image to obtain the captured image, includes:

[0063] At least one region image containing a second type of target is obtained from the second image, and the at least one region image is superimposed on the first image to obtain the captured image.

[0064] It should be noted that the first image determined in step 103 is a high-quality long-frame image determined after relevant exposure adjustments and / or quality evaluation of the long-frame image from the multi-frame exposure images; the second image is a high-quality short-frame image determined after relevant exposure adjustments and / or quality evaluation of the short-frame image from the multi-frame exposure images. Step 104 is then performed to overlay the images to obtain a fused image that takes into account both the first and second type of targets. Those skilled in the art can implement the overlay in step 104 according to relevant overlay or fusion schemes, and are not limited to any specific scheme.

[0065] In this process, at least one region image containing a second type of target is obtained from the second image, also known as image matting. This can be implemented by those skilled in the art according to relevant image matting schemes, and is not limited to a specific scheme.

[0066] In some exemplary embodiments, step 104, which involves superimposing the first image and the second image to obtain the captured image, includes:

[0067] For each region image, perform the following steps sequentially:

[0068] Determine whether the image of the region meets the preset first image quality requirement. If it meets the first image quality requirement, it is determined as the image to be superimposed. If it does not meet the first image quality requirement, optimize the image of the region and determine the optimized image of the region to be superimposed.

[0069] The entire area to be superimposed is superimposed onto the first image to obtain the captured image.

[0070] As can be seen, after acquiring at least one region image containing the second type of target from the second image, the image quality of each region image is judged. For region images that do not meet the quality requirements of the first image, further optimization is performed before they are superimposed onto the first image. This ensures that the region images acquired from the second image reach high quality before fusion, guaranteeing the overall high quality of the final captured image.

[0071] In some exemplary embodiments, optimizing each region image includes performing at least one of the following processes on the region image: sharpening, noise reduction.

[0072] In some exemplary embodiments, optimizing the image of the region when the first image quality requirement is not met includes:

[0073] If the first image quality requirement is not met, the image of the region is optimized until the optimized image of the region meets the second image quality requirement.

[0074] Those skilled in the art will understand that the optimization of the regional image continues until the second image quality requirement is met, i.e., the first or multiple optimizations are performed until the second image quality requirement is met before the optimization ends.

[0075] In some exemplary embodiments, whether a region image meets a preset first image quality requirement is determined according to the following method:

[0076] Based on the preset sharpness evaluation function, determine the sharpness score of the image in this area;

[0077] Based on the preset noise evaluation function, determine the noise score of the image in this region;

[0078] If the sharpness score is less than a first sharpness threshold or the noise score is greater than a first noise threshold, the image of that region is determined not to meet the preset first image quality requirement; otherwise, the image of that region is determined to meet the preset first image quality requirement.

[0079] In some exemplary embodiments, whether the optimized region image meets a preset second image quality requirement is determined according to the following method:

[0080] Based on the preset sharpness evaluation function, the sharpness score of the optimized region image is determined;

[0081] Based on the preset noise evaluation function, the noise score of the optimized region image is determined;

[0082] If the sharpness score is less than the second sharpness threshold or the noise score is greater than the second noise threshold, the optimized region image is determined to not meet the preset second image quality requirement; otherwise, the optimized region image is determined to meet the preset second image quality requirement.

[0083] The first image quality requirement and the second image quality requirement are set independently, and they can be the same or different.

[0084] In some exemplary embodiments, the sharpness score of the region image to be scored (the region image or the optimized region image) is determined according to the following manner:

[0085] A blurred image of the region to be scored is obtained using a low-pass filter. The difference between this blurred image and the original image (the region to be scored) is then calculated to obtain the edge image of the region to be scored, which is used to characterize the sharpness of the region to be scored. Let the original image be I, and the low-pass filter be F.

[0086]

[0087] Therefore, the average intensity of image sharpness in the region to be scored can be obtained as follows:

[0088]

[0089] The sharpness evaluation function is:

[0090]

[0091] The sharpness score Qs is obtained based on this sharpness evaluation function.

[0092] Where p and q are the subscripts of pixels in the image to be scored, P and Q represent the width and height of the image to be scored, respectively, t1, t2, t3, t4, and t5 are the boundary thresholds, and the number of boundaries and each boundary threshold can be adjusted according to actual needs. I1(p, q) is the information of the edge corresponding to pixel (p, q).

[0093] In some exemplary embodiments, the noise score is determined in the image of the region to be scored (the region image or the optimized region image) according to the following manner:

[0094] The image of the region to be scored is obtained by using a smoothing filter and then subtracting it from the original image (the image of the region to be scored) to obtain the noisy image. Let the original image be I and the smoothing filter be G, then the noisy image is:

[0095]

[0096] Therefore, the average intensity of the noise can be obtained as follows:

[0097]

[0098] The noise evaluation function is:

[0099]

[0100] The noise score Nn is obtained based on the noise evaluation function.

[0101] Where p and q are the subscripts of pixels in the image to be scored, P and Q represent the width and height of the image to be scored, respectively, t1, t2, and t3 are the boundary thresholds, and the number of boundaries and each boundary threshold can be adjusted according to actual needs, and I2(p, q) is the noise information corresponding to pixel (p, q).

[0102] In some exemplary embodiments, other schemes may be used for sharpness evaluation to determine a sharpness score; other schemes may also be used for noise evaluation to determine a noise score. Accordingly, when different evaluation schemes are used, the corresponding first image quality requirement and second image requirement are also adjusted accordingly. In some exemplary embodiments, in addition to sharpness and / or noise, other image quality evaluation factors may be selected, and scoring schemes corresponding to each evaluation factor may be selected to determine the corresponding image quality requirements, not limited to the aspects exemplified in the embodiments of this disclosure.

[0103] In some exemplary embodiments, step 101, determining multi-frame exposure parameters based on the overexposure information, includes:

[0104] The total number of frames M for multi-frame exposures is calculated using the following method:

[0105]

[0106] The number of short frames k is determined according to the following method:

[0107] k = M * b (Area(Tn) / H*L)-1 ;

[0108] Where b is any value greater than 2.

[0109] Based on the total number of frames M and the number of short frames k, the number of long frames N = Mk is determined;

[0110] Where Area(Tn) is the area of ​​the overexposed region, thr is the overexposed brightness threshold, f(Tn) is the number of pixels with brightness Tn in the captured image, a is any value greater than 1, H is the maximum vertical number of pixels in the camera's viewfinder, L is the maximum horizontal number of pixels in the camera's viewfinder, and Max is the maximum number of multi-frame exposure frames supported by the camera sensor.

[0111] The overexposed area Area(Tn) is calculated as follows:

[0112]

[0113] It is understandable that the overexposed area represents the total number of pixels in the captured image whose brightness exceeds the set overexposure threshold (thr).

[0114] In some exemplary embodiments, obtaining overexposure information of the captured image in step 101 includes:

[0115] Based on the initial total number of frames K, obtain the longest frame image F1 in the captured image, and divide the longest frame image into m x n regions, such as... Figure 2 As shown, m and n are integers greater than 1; obtain the segmented histogram statistics of the longest frame image F1, as shown below. Figure 3 As shown.

[0116] in, Figure 3 The meaning of the histogram shown:

[0117] 1. The horizontal axis represents the pixel grayscale value, and the vertical axis represents the number of pixels.

[0118] 2. Gray values ​​can be segmented to count the number of pixels whose gray values ​​fall within the segment range.

[0119] 3. It can be divided into any number of grayscale segments.

[0120] When 255≥Tn≥thr, the overexposed area Area(Tn) is calculated as follows:

[0121]

[0122] wherein, thr is defined as an overexposure threshold, which may be fixed to a certain value or a certain range, and may also be calculated according to statistical information, 0<thr≤255, and f(Tn) represents the number of pixels with brightness Tn.

[0123] Correspondingly, further determining multi-frame exposure parameters includes:

[0124] The total number of frames M of multi-frame exposure is calculated as follows:

[0125]

[0126] wherein, Area(Tn) is the area of the overexposed region, thr is the overexposure threshold, f(Tn) is the number of pixels with brightness Tn in a captured picture, a is any value greater than 1, H is the maximum vertical pixel count of the camera's framing frame, L is the maximum horizontal pixel count of the camera's framing frame, and Max is the maximum number of multi-frame exposure frames supported by the camera sensor;

[0127] The number of short frames k is calculated as follows:

[0128] k=M*b (Area(Tn) / H*L)-1 ;

[0129] According to the total number of frames M and the number of short frames k, the number of long frames N is determined as N=M-k.

[0130] In some exemplary embodiments, acquiring overexposure information of a captured picture and determining multi-frame exposure parameters in step 101 may also be acquired (calculated) according to one captured picture in a non-multi-frame exposure mode. Based on the above examples, those skilled in the art can know the corresponding implementation manners, which will not be repeated one by one herein.

[0131] In some exemplary embodiments, sharpening a regional image includes:

[0132] a. Assuming that the original image (regional image) is P, calculate a Gaussian blurred image G with a radius r

[0133] b. E=P(i,j)-G(i,j)

[0134] c. The sharpened image is S.

[0135]

[0136] wherein, a is a constant greater than 1, k is a constant, and thr is an overexposure threshold.

[0137] In some exemplary embodiments, denoising a region image includes:

[0138] A bilateral filter can be used to filter the noise, which can remove noise while preserving the object's edges.

[0139]

[0140]

[0141] Where δ is a constant, pixel (p,q) represents the neighboring pixels of pixel (i,j), f is the region image, f(p,q) represents the output value of pixel (p,q), f(i,j) represents the output value of pixel (i,j), and N(i,j) represents the output value of pixel (i,j) after the above noise reduction process.

[0142] In some exemplary embodiments, other methods may be employed to optimize the regional image to improve its image quality, not limited to sharpening and / or noise reduction. Furthermore, the specific sharpening and noise reduction methods are not limited to the specific aspects of the above examples. Those skilled in the art can select different optimization, sharpening, or noise reduction methods based on the specific device and business scenario. This aspect is not within the scope of protection or limitation sought in this disclosure.

[0143] As can be seen, the image acquisition scheme provided in this disclosure can perform multi-frame exposure with different exposure durations for different types of targets in relevant business scenarios, thereby obtaining high-quality long-frame and short-frame images corresponding to different types of targets. Based on this, image overlay can be performed to obtain high image quality for multiple types of targets in the same captured image. In some embodiments, by extracting the region image containing the set type of target, optimizing the region image, and then overlaying it to generate the final captured image, the image quality of the final captured image can be further improved.

[0144] This disclosure also provides a method for acquiring captured images, applied in a multi-frame exposure shooting mode. This embodiment is applied to a road monitoring scenario with mixed pedestrian and vehicle traffic at night, wherein the first type of target includes: faces and bodies, and the second type of target includes: license plates and vehicle bodies; a first sharpness threshold Q1, a first noise threshold N1, a second sharpness threshold Q2, and a second noise threshold N2. Figure 4 As shown, the method includes:

[0145] Step 401: Obtain the longest frame image F1 in the captured image, divide it into m*n regions and obtain the segmented histogram;

[0146] Step 402: acquiring overexposure information, and determining the total frame number M, the number of short frames k and the number of long frames N of multi-frame exposure;

[0147] Step 403: performing exposure based on the face / human body to obtain N long-frame images; performing exposure based on the license plate to obtain k short-frame images;

[0148] Step 404: determining a first image containing the face / human body according to the longest frame or the second longest frame in the N long-frame images; determining a second image containing the license plate according to the shortest frame or the second shortest frame in the k short-frame images;

[0149] Step 405: calculating a sharpness score Qs and a noise score Nn of L regional images of the license plate included in the second image;

[0150] Step 406: for each regional image, determining whether Qs < Q1 or Nn > N1, if yes, executing step 407; if no, executing step 410;

[0151] Step 407: optimizing the regional image;

[0152] Step 408: calculating Qs and Nn according to the optimized regional image;

[0153] Step 409: determining whether Qs < Q2 or Nn > N2, if yes, executing step 407; if no, executing step 410;

[0154] Step 410: determining the regional image or the optimized regional image as a to-be-superimposed regional image;

[0155] Step 411: superimposing all to-be-superimposed regional images onto the first image to obtain a fused captured image.

[0156] It can be seen that in step 411, the finally obtained captured image has high quality of the face, human body, license plate and vehicle body, which can meet the requirements of subsequent further identification or other business processing. Meanwhile, only outputting one fused image can effectively reduce storage pressure.

[0157] Wherein, in step 401, according to the initial total frame number K, the captured image is divided into m×n regions, and the piecewise histogram statistical information output by the longest frame F1 is acquired.

[0158] In step 402, the total frame number M, the number of short frames k and the number of long frames N of multi-frame exposure can be determined according to the detailed steps recorded in the foregoing embodiments.

[0159] In step 403, a longer exposure time is performed based on the face / body to obtain N long frame images, which can ensure that the face / body in the image is clear and the shooting effect is good; a shorter exposure time is performed based on the license plate / body to obtain k short frame images, which can ensure that the license plate / body in the image is clear and the shooting effect is good.

[0160] Step 404 includes: sending the longest or relatively long frame FL to the backend AI for face / body recognition, and then adjusting the exposure based on face / body region metering; if there is only one face / body in the image, the exposure is adjusted based on the face / body region metering, the AI ​​captures the face / body and performs a comparison and scoring, selecting the long frame image P1 with the highest score as the first image; if there are multiple faces / body in the image, the exposure is adjusted based on the average brightness and / or weighted brightness of the multiple faces / body, the AI ​​captures the face / body and performs a comparison and scoring, selecting the long frame image P1 with the highest score as the first image. In other words, the exposure of N long frame images is adjusted based on face / body region metering, and the long frame image P1 with the highest AI face / body recognition score is selected as the first image.

[0161] Step 404 further includes: sending the shortest frame or a shorter frame FS to the backend AI for license plate / vehicle recognition, and then adjusting the exposure based on the metering of the license plate / vehicle; if there is only one license plate / vehicle in the image, the exposure is adjusted based on the metering of that license plate / vehicle area, the AI ​​captures the license plate / vehicle and compares and scores it, selecting the shortest frame image P2 with the highest score as the second image; if there are multiple license plates / vehicles in the image, the exposure is adjusted based on the average brightness and / or weighted brightness of the multiple license plates / vehicles, the AI ​​captures the license plates / vehicles and compares and scores it, selecting the shortest frame image P2 with the highest score as the second image. That is, the exposure of k short frames is adjusted based on the metering of the license plate / vehicle area, and the shortest frame image P2 with the highest score for AI license plate / vehicle recognition is selected as the first image.

[0162] Although the exposures are performed simultaneously, there is still a time difference between long and short frames. Therefore, for the vehicles and license plates appearing in P2, their corresponding positions need to be found in P1 to facilitate subsequent steps. The algorithm for finding the same license plate / vehicle body in different images can be any algorithm from the relevant technical solutions and is not within the scope of protection and limitation of this disclosure.

[0163] In steps 405-406, before extracting the region containing the license plate / vehicle from the second image (short frame image) and overlaying it onto the first image (long frame image), it is determined whether each region image meets the image quality requirements. If not, steps 407-409 are executed to optimize and improve the quality of the region image before overlaying.

[0164] It's understandable that license plates contain Chinese characters, some of which are quite complex. Even with proper brightness, the text may not appear at its best. Therefore, to ensure the text on the captured license plate is clearly visible and produces better results, the image quality of the area containing the license plate is evaluated. If the quality is poor, optimization is performed. This ensures that the final superimposed image also has a high-quality portion of the license plate area.

[0165] In particular, the calculation of Qs and Nn in steps 405 and 408 can be carried out according to the specific steps described above.

[0166] As can be seen in step 411, after the image of the area including the license plate and the vehicle body is evaluated for quality, and after it is determined that the image quality meets the standards (meets the first image quality requirement or the second image quality requirement), the image of the area including the license plate and the vehicle body is overlaid back into the original position of the license plate and the vehicle body in the first image (long frame image), and finally an image is output. The images of the face / body and the license plate / vehicle body in this image are of high quality and can meet the needs of further recognition or other business processing based on the captured image.

[0167] This disclosure also provides an electronic device, including:

[0168] One or more processors;

[0169] Storage device for storing one or more programs.

[0170] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for acquiring captured images as described in any of the above embodiments.

[0171] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, the program being implemented by a processor as described in any of the above embodiments for acquiring captured images.

[0172] As can be seen, the image acquisition scheme provided in this disclosure, applied to a multi-frame exposure shooting mode, proposes a method to adjust the number of exposures across multiple frames based on the overexposure level of the captured image, and dynamically allocate the number of long and short frames. This allows the long and short frames to be exposed based on different metering regions, thereby ensuring that images containing the first type of target and images containing the second type of target, both with good quality, can be obtained separately. These images are then superimposed to obtain a single image with a good overall effect. In some embodiments, areas that do not meet quality requirements are optimized before being superimposed, which can further improve the image quality of the captured image.

[0173] When applied to nighttime mixed-traffic road monitoring scenarios, this disclosed solution can dynamically determine the number of long frames and short frames captured in multi-frame exposure. Long frames are based on face / body exposure, while short frames are based on license plate / vehicle exposure, resulting in high-quality face / body and license plate / vehicle images. These images are then superimposed to generate a single high-quality image where the face / body, license plate, and vehicle are all clearly visible. This improves the convenience and efficiency of subsequent recognition or other business processing, while simultaneously reducing the storage pressure on captured images.

[0174] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0175] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made under the concept of the present invention using the contents of the present invention specification and drawings, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A method for acquiring captured images, applied in a multi-frame exposure shooting mode, characterized in that, include: Obtain overexposure information of the captured image, and determine multi-frame exposure parameters based on the overexposure information; Perform multi-frame exposure based on the multi-frame exposure parameters to obtain the corresponding multi-frame images; The first image is determined based on the long frame image in the multi-frame image, and the second image is determined based on the short frame image in the multi-frame image; The captured image is obtained by superimposing the first image and the second image; The overexposure information includes: brightness information of pixels whose brightness exceeds the overexposure threshold; the multi-frame exposure parameters include: the total number of frames, the number of short frames, and the number of long frames in the multi-frame exposure. The step of performing multi-frame exposure based on the multi-frame exposure parameters to obtain the corresponding multi-frame images includes: Based on the number of long frames, exposure is performed according to the first type of target to obtain a corresponding number of long frame images; Based on the number of short frames, exposure is performed on the second type of target to obtain the corresponding number of short frame images.

2. The method as described in claim 1, characterized in that, Determining the first image based on the long frame image among the multiple frames includes: Identification of the first type of target based on a long frame image; If the recognition result includes a first type of target, the long frame image in the multi-frame image is exposed based on the region where the first type of target is located to obtain at least one long frame image with exposure adjustment; based on the at least one long frame image with exposure adjustment, the recognition result of the first type of target is obtained respectively; based on the recognition result of the first type of target recognition, the recognition result quality score is calculated, and the long frame image with the highest score is determined as the first image; or, If the recognition result includes multiple first-type targets, the long frame image in the multi-frame image is exposed and adjusted based on the average brightness and / or weighted brightness of the area where the multiple first-type targets are located, to obtain at least one exposure-adjusted long frame image; based on the at least one exposure-adjusted long frame image, the recognition result of recognizing multiple first-type targets is obtained respectively; based on the recognition result quality score of the recognition result of multiple first-type targets, the long frame image with the highest score is determined as the first image; The long frame image is either the longest or the second longest frame image among all long frame images.

3. The method as described in claim 1, characterized in that, Determining the second image based on the short frame image from the multi-frame image includes: Identification of the first type of target based on a short frame image; If the recognition result includes a second type of target, the exposure of the short frame image in the multi-frame image is adjusted based on the region where the second type of target is located to obtain at least one exposure-adjusted short frame image; based on the at least one exposure-adjusted short frame image, the recognition result of a second type of target is obtained respectively; based on the recognition result of the second type of target recognition, the recognition result quality score is calculated, and the short frame image with the highest score is determined as the second image; or, If the recognition result includes multiple second-type targets, the exposure of short-frame images in the multi-frame images is adjusted based on the average brightness and / or weighted brightness of the regions where the multiple second-type targets are located, to obtain at least one exposure-adjusted short-frame image; based on the at least one exposure-adjusted short-frame image, the recognition results for recognizing multiple second-type targets are obtained respectively; based on the recognition results for recognizing multiple second-type targets, a recognition result quality score is calculated, and the short-frame image with the highest score is determined as the second image; The short frame image is either the shortest or the second shortest frame image among all short frame images.

4. The method as described in claim 1, characterized in that, The process of superimposing the first image and the second image to obtain the captured image includes: At least one region image containing a second type of target is obtained from the second image, and the at least one region image is superimposed on the first image to obtain the captured image.

5. The method as described in claim 1, characterized in that, The process of superimposing the first image and the second image to obtain the captured image includes: Obtain at least one region image containing a second type of target from the second image; For each region image, the following steps are performed sequentially: determine whether the region image meets the preset first image quality requirement, and if it meets the first image quality requirement, determine it as the region image to be superimposed; if it does not meet the first image quality requirement, optimize the region image, and determine the optimized region image as the region image to be superimposed. The entire area to be superimposed is superimposed onto the first image to obtain the captured image.

6. The method as described in claim 5, characterized in that, The following method is used to determine whether the region image meets the preset first image quality requirement: Based on the preset sharpness evaluation function, determine the sharpness score of the image in this area; Based on the preset noise evaluation function, determine the noise score of the image in this region; If the sharpness score is less than a first sharpness threshold or the noise score is greater than a first noise threshold, the image of that region is determined not to meet the preset first image quality requirement; otherwise, the image of that region is determined to meet the preset first image quality requirement.

7. The method as described in claim 1, characterized in that, The step of determining the multi-frame exposure parameters based on the overexposure information includes: The total number of frames M for the multi-frame exposure is calculated using the following method: M = [( ) / 255 ] Max; The number of short frames k is determined according to the following method: k = ； Based on the total number of frames M and the number of short frames k, the number of long frames N = Mk is determined; in, Area (Tn) tr is the overexposure area, thr is the overexposure threshold, f(Tn) is the number of pixels with a brightness of Tn in the captured image, a is any value greater than 1, H is the maximum vertical number of pixels in the camera's viewfinder, L is the maximum horizontal number of pixels in the camera's viewfinder, Max is the maximum number of multi-frame exposures supported by the camera sensor, and b is any value greater than 2. The area of ​​the overexposed region Area (Tn) Calculated as follows: 。 8. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method for acquiring captured images as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the method for acquiring captured images as described in any one of claims 1-7.

Images