Shooting method, apparatus, controller, device and computer-readable storage medium

By extracting brightness feature information from the preview image from the shooting device and using a neural network to generate an exposure strategy, the problem of insufficient scene recognition capability in existing technologies is solved, and higher quality shooting results are achieved.

CN115696027BActive Publication Date: 2026-11-17ZTE CORP
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202110828028.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-07-22
Publication Date
2026-11-17
Estimated Expiration
2041-07-22

AI Technical Summary

Technical Problem

Existing HDR processing technologies have limited scene recognition capabilities and cannot accurately guide exposure parameters, thus affecting shooting quality.

Method used

By acquiring a preview image of the current scene, brightness features are extracted, and an exposure strategy is generated using a strategy generation model trained by a neural network to guide the shooting process.

Benefits of technology

It improved the accuracy of exposure strategies and enhanced shooting quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115696027B_ABST
    Figure CN115696027B_ABST
Patent Text Reader

Abstract

This invention provides a shooting method, apparatus, controller, device, and computer-readable storage medium, including: acquiring a preview image of a current scene; extracting features from the preview image to obtain brightness feature information; inputting the brightness feature information into a strategy generation model to obtain an exposure strategy, wherein the strategy generation model is trained by a neural network based on the brightness feature information corresponding to sample images; and shooting the current scene based on the exposure strategy. During previewing, the brightness feature information of the preview image of the current scene is extracted and input into the trained strategy generation model. Since the strategy generation model is trained by a neural network based on the brightness feature information corresponding to sample images, it will respond by outputting the exposure strategy corresponding to the preview image of the current scene. Therefore, the exposure strategy output by the strategy generation model can be used for shooting during the actual shooting process, thereby improving the shooting quality.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to, but are not limited to, the field of photography technology, and particularly to a photography method, a photography device, a controller, a photography equipment, and a computer-readable storage medium. Background Technology

[0002] Currently, due to the physical characteristics of sensors, such as dynamic range and noise, when shooting outdoor backlit scenes, some areas need to be enhanced while others need to be weakened. Therefore, HDR (High-Dynamic Range) processing can be used to obtain images with an ideal wide dynamic range. Existing processing techniques use multiple exposure combinations to suppress the brightness of bright scenes and enhance the brightness of dark scenes. Therefore, existing processing techniques require accurate exposure parameters. However, existing HDR triggering mechanisms mostly rely on statistical characteristics of overall brightness, decision numbers, or preset thresholds for judgment. These mechanisms have limited scene recognition capabilities and cannot accurately guide exposure parameters after scene recognition, thus affecting image quality. Summary of the Invention

[0003] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0004] The present invention provides a shooting method, shooting device, controller, shooting equipment, and computer-readable storage medium, which can obtain a more accurate exposure strategy, thereby improving the shooting quality.

[0005] In a first aspect, embodiments of the present invention provide a shooting method, comprising: acquiring a preview image of a current scene; extracting features from the preview image to obtain brightness feature information of the preview image; inputting the brightness feature information into a strategy generation model to obtain an exposure strategy, wherein the strategy generation model is trained by a neural network based on the brightness feature information corresponding to a sample image; and shooting the current scene based on the exposure strategy.

[0006] Secondly, embodiments of the present invention also provide a shooting device, including a processor, a memory, a shooting component, and a display screen. The processor is connected to the memory, the shooting component, and the display screen. The processor includes a feature extraction unit and a strategy generation unit. The shooting component includes an optical camera. The shooting component is used to acquire a preview image corresponding to the current scene through the optical camera. The display screen is used to display the preview image corresponding to the current scene. The feature extraction unit is used to acquire the preview image displayed on the display screen and perform feature extraction on the preview image to obtain brightness feature information of the preview image. The strategy generation unit is used to input the brightness feature information into a pre-trained strategy generation model in the processor to obtain an exposure strategy. The strategy generation model is trained by a neural network based on the brightness feature information corresponding to a sample image. The shooting component is also used to acquire the exposure strategy and, based on the exposure strategy, shoot the current scene through the optical camera.

[0007] Thirdly, embodiments of the present invention also provide a controller, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the shooting method as described in the first aspect above.

[0008] Fourthly, embodiments of the present invention also provide a shooting device, including the shooting device as described in the second aspect above or the controller as described in the third aspect above.

[0009] Fifthly, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions for performing the shooting method as described in the first aspect above.

[0010] This invention includes the following steps: first, acquiring a preview image of the current scene; then, extracting features from the preview image to obtain brightness feature information; next, inputting the brightness feature information into a strategy generation model to obtain an exposure strategy; wherein the strategy generation model is trained by a neural network based on the brightness feature information corresponding to sample images; and finally, taking a picture of the current scene based on the aforementioned exposure strategy. According to the technical solution of this invention, during the preview period of shooting, this invention extracts the brightness feature information of the preview image of the current scene and inputs it into the trained strategy generation model. Since the strategy generation model of this invention is trained by a neural network based on the brightness feature information corresponding to sample images, the strategy generation model will respond by outputting the exposure strategy corresponding to the preview image of the current scene. Therefore, when the shooting button is pressed, the exposure strategy output by the strategy generation model can be used for shooting. Because the exposure strategy output by the strategy generation model has higher accuracy, this invention can improve the shooting quality.

[0011] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the description, claims, and drawings. Attached Figure Description

[0012] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0013] Figure 1 This is a schematic diagram of a system architecture platform for performing a shooting method according to an embodiment of the present invention;

[0014] Figure 2 This is a flowchart of a shooting method provided in one embodiment of the present invention;

[0015] Figure 3 This is a flowchart of the method for extracting brightness feature information in an embodiment of the present invention;

[0016] Figure 4 This is a flowchart illustrating the acquisition of brightness position weights in an embodiment of the shooting method provided by the present invention;

[0017] Figure 5 This is a flowchart illustrating the process of obtaining brightness proportion weights in a shooting method provided by an embodiment of the present invention;

[0018] Figure 6This is a flowchart illustrating how a shooting method according to an embodiment of the present invention obtains brightness feature information based on brightness position weight, brightness proportion weight, and brightness interval ratio.

[0019] Figure 7 This is a schematic diagram of an exposure strategy combination provided in an embodiment of the present invention;

[0020] Figure 8 This is a schematic diagram illustrating the construction of a neural network model provided in one embodiment of the present invention;

[0021] Figure 9 This is a schematic diagram of the imaging device provided in one embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0023] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, or the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0024] In related technologies, due to the physical characteristics of sensors, such as dynamic range and noise, when shooting outdoor backlit scenes, some areas need to be enhanced while others need to be weakened. Therefore, HDR processing can be used to obtain images with an ideal wide dynamic range. Existing processing techniques use multiple exposure combinations to suppress the brightness of bright scenes and enhance the brightness of low-light scenes. Therefore, existing processing techniques require accurate exposure parameters. However, existing HDR triggering mechanisms mostly rely on statistical characteristics of overall brightness, decision numbers, or preset thresholds for judgment. These mechanisms have limited scene recognition capabilities and cannot accurately guide exposure parameters after scene recognition, thus affecting image quality.

[0025] It should be noted that existing multi-frame dynamic calculation, single-frame scene decision tree recognition, and whole-frame feature brightness recognition for HDR scenes are not accurate enough and cannot precisely guide the selection of multi-exposure parameters in HDR.

[0026] Based on the above, embodiments of the present invention provide a shooting method, shooting device, controller, shooting equipment, and computer-readable storage medium. The shooting method includes, but is not limited to, the following steps: acquiring a preview image of the current scene; extracting features from the preview image to obtain brightness feature information; inputting the brightness feature information into a strategy generation model to obtain an exposure strategy, wherein the strategy generation model is trained by a neural network based on the brightness feature information corresponding to sample images; and shooting the current scene based on the exposure strategy. According to the technical solution of the embodiments of the present invention, during the preview period of shooting, the embodiments of the present invention extract the brightness feature information of the preview image of the current scene and input it into the trained strategy generation model. Since the strategy generation model of the embodiments of the present invention is trained by a neural network based on the brightness feature information corresponding to sample images, the strategy generation model will respond by outputting the exposure strategy corresponding to the preview image of the current scene. Therefore, when the shooting button is pressed, the exposure strategy output by the strategy generation model can be used for shooting processing. Because the exposure strategy output by the strategy generation model has higher accuracy, the embodiments of the present invention can improve the shooting quality.

[0027] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0028] like Figure 1 As shown, Figure 1 This is a schematic diagram of a system architecture platform for performing a shooting method provided in an embodiment of the present invention.

[0029] exist Figure 1 In this example, the system architecture platform 100 includes a processor 110 and a memory 120, which can be connected via a bus or other means. Figure 1 Taking the example of a connection between China and Israel via a bus.

[0030] Memory 120, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory 120 may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 120 may optionally include memory 120 remotely located relative to processor 110, and these remote memories can be connected to the system architecture platform via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0031] Those skilled in the art will understand that this system architecture platform can be applied to 3G communication network systems, LTE communication network systems, 5G communication network systems and subsequent evolved mobile communication network systems, etc., and this embodiment does not specifically limit it in this regard.

[0032] It will be understood by those skilled in the art that Figure 1 The system architecture platform shown does not constitute a limitation on the embodiments of the present invention, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0033] exist Figure 1 In the system architecture platform shown, the processor 110 can call the shooting program stored in the memory 120 to execute the shooting method.

[0034] Based on the above system architecture platform, various embodiments of the shooting method of the present invention are proposed below.

[0035] like Figure 2 As shown, Figure 2 This is a flowchart of a shooting method provided in an embodiment of the present invention, which includes, but is not limited to, steps S100, S200, S300 and S400.

[0036] Step S100: Obtain a preview image of the current scene;

[0037] Step S200: Extract features from the preview image to obtain the brightness feature information of the preview image;

[0038] Step S300: Input the brightness feature information into the strategy generation model to obtain the exposure strategy. The strategy generation model is trained by a neural network based on the brightness feature information corresponding to the sample image.

[0039] Step S400: Take a picture of the current scene based on the exposure strategy.

[0040] Specifically, during the preview period of the shooting, the brightness feature information of the preview image of the current scene is extracted and input into the trained strategy generation model. Since the strategy generation model of the present invention is trained by a neural network based on the brightness feature information corresponding to the sample image, the strategy generation model will respond by outputting the exposure strategy corresponding to the preview image of the current scene. Thus, when the shooting button is pressed, the exposure strategy output by the strategy generation model can be used for shooting. Since the exposure strategy output by the strategy generation model is more accurate, the present invention can improve the shooting quality.

[0041] It is worth noting that, regarding the acquisition of a preview image of the current scene in step S100 above, this preview image refers to the image displayed on the screen of the shooting device before the shutter button is pressed. For example, the preview image refers to the image displayed on the screen of the shooting device before the user holds the shooting device towards the scenery and presses the shutter button.

[0042] Additionally, it is worth noting that the aforementioned strategy generation model is a pre-trained, pre-defined model. During training, this embodiment of the invention extracts brightness feature information from sample images and uses it as input to the neural network, causing the neural network to output a corresponding exposure strategy. Once the strategy generation model is trained, if the brightness feature information of a preview image of the current scene is input into the strategy generation model, the model will respond by outputting a more accurate exposure strategy.

[0043] It is understandable that the number of sample images mentioned above can be multiple.

[0044] Additionally, it is understandable that the aforementioned exposure strategy can have multiple exposure parameters.

[0045] Furthermore, it is understandable that the aforementioned neural network is an algorithmic mathematical model that mimics the behavioral characteristics of animal neural networks to perform distributed parallel information processing. This type of network relies on the complexity of the system, adjusting the relationships between a large number of interconnected nodes to achieve the purpose of information processing.

[0046] In addition, such as Figure 3 As shown, Figure 3 This is a flowchart of a shooting method for extracting brightness feature information according to an embodiment of the present invention. The above step S200 includes, but is not limited to, steps S410, S420, S430 and S440.

[0047] Step S410: Divide the preview image into multiple sub-regions, and determine multiple target sub-regions from the multiple sub-regions;

[0048] Step S420: For each target sub-region, calculate the cumulative brightness distribution value based on the brightness value of the target sub-region, and obtain the brightness interval ratio of the target sub-region based on the cumulative brightness distribution value and the preset brightness threshold.

[0049] Step S430: Obtain the brightness position weight of each target sub-region;

[0050] Step S440: Calculate the brightness feature information of the preview image based on the brightness position weights and brightness interval ratios of all target sub-regions.

[0051] Specifically, in the extraction process of this embodiment of the invention, the preview image is divided into multiple sub-regions, and a certain number of target sub-regions are selected from these sub-regions. Then, the cumulative brightness distribution value of each target sub-region is calculated, i.e., the CDF (cumulative distribution function) value between high and low brightness. Since this embodiment of the invention sets a preset brightness threshold, which represents a certain brightness range, the brightness range ratio of the target sub-region is obtained based on the cumulative brightness distribution value and the preset brightness threshold, i.e., the proportion in each brightness range. Next, since different target sub-regions have different positions and different distances from the exposure center, this embodiment of the invention also obtains the brightness position weight of the target sub-regions. Finally, based on the brightness position weights and brightness range ratios of all target sub-regions, the brightness feature information of the preview image is calculated, thereby realizing the extraction of the brightness feature information of the preview image.

[0052] It should be noted that, in this embodiment of the invention, the brightness feature value is extracted as the input of the neural network by calculating the brightness distribution of different regions of the preview image corresponding to the current scene, and the calibrated scene classification is used as the output of the neural network. After pre-training, the network structure can effectively solve the problem of selecting exposure parameters in scenes that require exposure fusion.

[0053] It is worth noting that the aforementioned preset brightness threshold may include, but is not limited to, a preset high brightness threshold and a preset low brightness threshold. Correspondingly, since there are preset brightness thresholds in different ranges, for each target sub-region, the brightness interval ratio obtained in this embodiment of the invention also includes, but is not limited to, a high brightness interval ratio and a low brightness interval ratio. Correspondingly, since there are high brightness interval ratios and low brightness interval ratios, the brightness feature information obtained subsequently also includes high brightness interval feature information and low brightness interval feature information.

[0054] Specifically, when the preset brightness threshold includes a preset high brightness threshold and a preset low brightness threshold, the brightness interval ratio of the target sub-region obtained based on the cumulative brightness distribution value and the preset brightness threshold in the above step S420 includes: obtaining the high brightness interval ratio of the target sub-region based on the cumulative brightness distribution value and the preset high brightness threshold, and obtaining the low brightness interval ratio of the target sub-region based on the cumulative brightness distribution value and the preset low brightness threshold.

[0055] In addition, correspondingly, the calculation of the brightness feature information of the preview image based on the brightness position weight and brightness interval ratio of all target sub-regions in step S440 above includes: calculating the high brightness interval feature information of the preview image based on the brightness position weight and high brightness interval ratio of all target sub-regions, and calculating the low brightness interval feature information of the preview image based on the brightness position weight and low brightness interval ratio of all target sub-regions.

[0056] Additionally, it is worth noting that the number of target sub-regions is less than or equal to the number of sub-regions. Specifically, the number of sub-regions and the number of target sub-regions can be the same, or the number of sub-regions and the number of target sub-regions can be different.

[0057] Specifically, when the number of sub-regions and the number of target sub-regions are the same, all the divided sub-regions are target sub-regions. Therefore, the brightness position weight mentioned above can refer to the global brightness position weight of the preview image.

[0058] Alternatively, when the number of sub-regions and the number of target sub-regions are inconsistent, that is, when the number of target sub-regions is less than the number of sub-regions, that is, only some of the sub-regions in the divided sub-regions are target sub-regions, the brightness position weight mentioned above may refer to the local brightness position weight of the preview image.

[0059] In addition, when the number of target sub-regions is less than the number of sub-regions, the target sub-regions are located in the center of the preview image or around the perimeter of the preview image. Alternatively, embodiments of the present invention can select the location of interest according to the user's needs.

[0060] In addition, such as Figure 4 As shown, Figure 4 This is a flowchart of obtaining brightness position weight in a shooting method provided by an embodiment of the present invention. The above step S430 includes, but is not limited to, steps S510 and S520.

[0061] Step S510: Obtain the distance information between the target sub-region and the preset position in the preview image;

[0062] Step S520: Calculate the brightness position weight of each target sub-region based on the distance information.

[0063] Specifically, since different target sub-regions have different positions and different distances from the exposure center, this embodiment of the invention also obtains the distance information between the target sub-region and the preset position in the preview image, and calculates the brightness position weight of each target sub-region based on the distance information.

[0064] It is understood that the aforementioned preset position may, but is not limited to, refer to the position of the exposure center or the center position of the preview image.

[0065] In addition, such as Figure 5 As shown, Figure 5 This is a flowchart of obtaining the brightness ratio weight in a shooting method provided by an embodiment of the present invention. The shooting method of the present invention also includes, but is not limited to, step S600.

[0066] Step S600: Calculate the brightness proportion weight based on the brightness interval ratio of the target sub-region.

[0067] Specifically, in this embodiment of the invention, the weight value is also adjusted according to the brightness ratio value, i.e. the brightness range ratio.

[0068] It should be noted that, compared to the brightness position weight mentioned above, the brightness proportion weight mentioned here is used for fine-tuning the exposure strategy, while the brightness position weight mentioned above is used for coarse-tuning the exposure strategy.

[0069] based on Figure 5 ,like Figure 6 As shown, Figure 6 This is a flowchart of a shooting method provided in an embodiment of the present invention, in which brightness feature information is obtained based on brightness position weight, brightness proportion weight and brightness interval ratio. Regarding the above step S440, it includes, but is not limited to, step S700.

[0070] Step S700: Calculate the brightness feature information of the preview image based on the brightness position weight, brightness proportion weight, and brightness interval ratio of all target sub-regions.

[0071] Specifically, in this embodiment of the invention, when inputting to the strategy generation model, the brightness proportion weight can also be input to the strategy generation model. Since the brightness proportion weight can play a role in fine-tuning the exposure strategy, the brightness feature information output by the strategy generation model in step S700 will be more accurate.

[0072] based on Figures 2 to 6 The method steps in the text, when the number of sub-regions is M*N, the number of target sub-regions is i*j, where i≤M, j≤N; the cumulative brightness distribution value is cdf. i,j ,i∈[1,M],j∈[1,N]; when the preset highlight threshold is [lum3,lum4] and the preset low brightness threshold is [lum1,lum2], where lum4>lum3≥lum2>lum1.

[0073] Specifically, the highlight area ratio is obtained by the following formula: L i,j =cdf i,j (lum4)-cdfi,j (lum3), where L i,j The ratio of the highlighted area;

[0074] The proportion of the low-brightness region is obtained by the following formula: D i,j =cdf i,j (lum2)-cdf i,j (lum1), where D i,j This represents the proportion of the low-brightness region.

[0075] In addition, the brightness position weight can be obtained by the following formula:

[0076]

[0077] Among them, W d (i,j) represents the brightness position weight, and δ1 is an adjustable parameter.

[0078] In addition, the brightness weighting can be obtained by the following formula:

[0079]

[0080] Among them, W a (lum) represents the brightness position weight, and δ2 and u are adjustable parameters.

[0081] In addition, the feature information of the highlighted area can be obtained by the following formula:

[0082]

[0083] Among them, vl k Highlight the feature information of the highlighted area;

[0084] In addition, the feature information of the low-brightness region can be obtained by the following formula:

[0085]

[0086] Among them, vd k This represents the feature information of the low-brightness region.

[0087] based on Figures 2 to 6 Based on the method steps and the derivation formulas described above, this embodiment of the invention provides an overall implementation scheme, as follows:

[0088] Step 1: Obtain the brightness values ​​of an image of size (w, h), Lum i,j Indicates the brightness at position (i,j):

[0089] Lum i,j =Max(R) i,j G i,j B i,j), i≤h, j≤w

[0090] Step 2: Divide the obtained brightness image into M*N sub-regions, and calculate the cumulative brightness distribution value for each sub-region: cdf m,n ,m∈[1,M],n∈[1,N]. Set the preset low brightness threshold: [lum1,lum2], set the preset high brightness threshold: [lum3,lum4], where lum4>lum3≥lum2>lum1.

[0091] Obtain the highlight ratio of each sub-region, i.e., the highlight interval ratio: L m,n =cdf m,n (lum4)-cdf m,n (lum3)

[0092] Obtain the low-brightness ratio of each sub-region, i.e., the low-brightness interval ratio: D m,n =cdf m,n (lum2)-cdf m,n (lum1)

[0093] Step 3: Obtain the weights of the region to be analyzed.

[0094] Global brightness weight W d That is, the global brightness location weight W d The weights are adjusted based on the distance from the center. Examples of weights are shown below, but are not limited to the following forms:

[0095]

[0096] Where δ1 is an adjustable parameter, m≤M,n≤N.

[0097] Weighting W a That is, the brightness proportion weight W a The weight values ​​are adjusted based on the brightness percentage, and examples of the weights are shown below, but are not limited to the following forms:

[0098]

[0099] Among them, W a (lum) is the brightness position weight, δ2 and u are adjustable parameters, m≤M,n≤N.

[0100] It should be noted that weights can also be stored in a table, and the calculation is implemented using a LUT (Look-Up Table) to simplify the calculation and increase efficiency.

[0101] Local brightness weight W d: Assign weight values ​​to different sub-regions to obtain the ROI (Region of Interest): W d (i,j).

[0102] For example, the central area W d (i,j) are distributed in the form shown in Table 1 below:

[0103] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

[0104] Table 1

[0105] Or, for example, the surrounding area W d (i,j), in a similar form to Table 2 below:

[0106] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

[0107] Table 2

[0108] Alternatively, a similar method can be used for other local areas such as corners and diagonals to obtain different local feature weight tables.

[0109] Step four: Obtain the brightness feature distribution value of the image by weighted averaging, i.e. brightness feature information.

[0110] 2k feature values ​​(k>1) of the entire image are obtained through local or global weights, where the weights W for bright and dark areas are... d W a They can be different; the weights of bright and dark areas can be calculated to be the same as follows:

[0111] Bright area feature values, i.e., the feature information of the highlighted area:

[0112] Dark area feature values, i.e., feature information of low-brightness areas:

[0113] It should be noted that the weights can also be implemented using LUTs during the calculation, resulting in a 2k-dimensional vector composed of a set of brightness feature information:

[0114] vec in = vector(vl1, vd1, ..., vl k ,vd k )

[0115] Step 5: Input the calculated feature vector into the trained neural network to obtain an accurate exposure strategy, thereby launching the HDR algorithm and synthesizing multiple images.

[0116] Exposure strategies are categorized into corresponding output values. Through actual photographic captures, the required exposure strategy values ​​for each image in the HDR algorithm are determined. For example, the exposure strategy output might be an L-dimensional vector: vec out = vector(exp1,...,exp L )

[0117] Regarding the method for selecting exposure parameters, for example, HDR synthesis requires three photos with different exposure parameters. In addition to the normally exposed photo, other high and low exposure parameters are combined as the output of the neural network, such as the low exposure EV. - Divided into three levels High Exposure EV + Divided into three levels: Among them, exposure strategy combinations can be as follows: Figure 7 As shown, Figure 7 This is a schematic diagram of an exposure strategy combination provided in an embodiment of the present invention.

[0118] Specifically, in this embodiment of the invention, the required exposure strategy for the corresponding sample photo can be selected as the output of the network, and then the network is trained to obtain the optimal structure.

[0119] When HDR fusion is not limited to three exposure levels (normal, underexposure, and overexposure), each additional photo to be fused requires an additional exposure setting. For example, when fusion of five photos, the underexposure setting can be refined: in addition to the original three fusion settings, two more exposure settings are added: EV -- =[ev -- 1,ev -- 2,ev -- 3,...],EV ++ =[ev ++ 1,ev ++ 2,ev ++ 3,...].

[0120] It is understandable that the neural network model construction used above can be referenced. Figure 8 The form shown illustrates this. That is, one or more hidden layers are included between the input and output layers to better perform linear partitioning or improve regression performance, etc. In other preferred embodiments, any other suitable neural network model can also be used.

[0121] When there are many input feature vectors and output values, the network results can be adjusted as needed, such as adding or removing network nodes or increasing network layers, to meet the output requirements. During shooting, the trained network runs on the terminal via hardware support or software embedding. During preview, the brightness feature values ​​of the current scene are calculated and input into the network to calculate the exposure parameters required to activate HDR. After activation, the captured photo completes the HDR photo fusion.

[0122] Regarding the shooting method of this invention, the brightness distribution of the entire image is statistically analyzed by sub-regions, and the CDF value of the sub-regions is calculated to obtain the ratio of high brightness to low brightness. Several weighting coefficients are designed to obtain the brightness distribution characteristics of the current scene: distance weights are obtained by varying the distance from the center to obtain global brightness and low brightness distribution characteristics; different brightness weights are adjusted by varying the brightness ratios to obtain brightness and low brightness distribution characteristics; the weight distribution is adjusted so that the center is 1 and the surrounding area is 0 to obtain local high and low brightness distribution characteristic values; the weight distribution is adjusted so that the center is 0 and the surrounding area is 1 to obtain local high and low brightness distribution characteristic values.

[0123] Based on the calibrated image, the network is trained using the scene's backlight intensity or HDR exposure combination as output values ​​to obtain the network structure. During shooting, global or local brightness feature values ​​obtained through several weights are input into the pre-trained network to obtain the required exposure parameters for the backlight intensity or HDR trigger.

[0124] Additionally, it should be noted that the training process for the policy generation model and the shooting process for the current scene are similar in terms of methods and steps.

[0125] Specifically, the training process of the strategy generation model is as follows: Multiple sample images are acquired and divided into multiple sub-regions, and multiple target sub-regions are determined from these sub-regions. Then, for each target sub-region in the sample images, a cumulative brightness distribution value is calculated based on the brightness value of the target sub-region in the sample images. The proportion of the high-brightness interval of the target sub-region is obtained based on the cumulative brightness distribution value and a preset high-brightness threshold, and the proportion of the low-brightness interval of the target sub-region is obtained based on the cumulative brightness distribution value and a preset low-brightness threshold. Next, the distance information between the target sub-region and a preset position in the sample image is acquired, and the brightness position weight of each target sub-region is calculated based on the distance information. Then, the brightness proportion weight is calculated based on the brightness interval proportion of the target sub-region. Next, the brightness feature information of the sample image is calculated based on the brightness position weight, brightness proportion weight, and brightness interval proportion of all target sub-regions. Finally, the calculated brightness feature information is used to train the neural network, and the category vector corresponding to the exposure parameter combination is used as the network output, thereby obtaining the exposure parameters required for the degree of backlighting or HDR triggering.

[0126] Based on the shooting method described in the above embodiments, various embodiments of the shooting device of the present invention are presented below.

[0127] like Figure 9 As shown, Figure 9This is a schematic diagram of the structure of a shooting device provided in an embodiment of the present invention. The shooting device 200 includes, but is not limited to, a processor 110, a memory 120, a shooting component 130, and a display screen 140. The processor 110 is connected to the memory 120, the shooting component 130, and the display screen 140. The processor 110 includes, but is not limited to, a feature extraction unit 111 and a strategy generation unit 112. The processor 110 can call the shooting program stored in the memory 120 to execute the shooting method. The shooting component 130 includes, but is not limited to, an optical camera 131.

[0128] Specifically, the capturing component 130 aims the optical camera 131 at the current scene to be captured and displays the preview image corresponding to the current scene on the display screen 140; then, the feature extraction unit 111 in the processor 110 can acquire the preview image displayed on the display screen 140 and perform feature extraction on the preview image to obtain the brightness feature information of the preview image; next, the strategy generation unit 112 in the processor 110 will input the brightness feature information into the pre-trained strategy generation model in the processor 110 to obtain the exposure strategy, wherein the strategy generation model is trained by a neural network based on the brightness feature information corresponding to the sample image; finally, the capturing component 130 will acquire the obtained exposure strategy and capture the current scene using the optical camera 131 based on the exposure strategy.

[0129] It is worth noting that the specific implementation methods and technical effects of the shooting device in the embodiments of the present invention can be referred to the specific implementation methods and technical effects of the shooting method in the above embodiments.

[0130] Based on the shooting method described in the above embodiments, various embodiments of the controller, shooting device, and computer-readable storage medium of the present invention are presented below.

[0131] In addition, one embodiment of the present invention provides a controller, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0132] The processor and memory can be connected via a bus or other means.

[0133] It should be noted that the controller in this embodiment may include, for example, the controller with the following components: Figure 1 The memory and processor in the illustrated embodiment can constitute Figure 1 The system architecture platform shown in the embodiment is part of the same inventive concept, and therefore has the same implementation principle and beneficial effects, which will not be described in detail here.

[0134] The non-transitory software program and instructions required to implement the shooting method of the above embodiments are stored in memory. When executed by a processor, the shooting method of the above embodiments is executed, for example, the method described above is executed. Figure 2 Method steps S100 to S400 in the text Figure 3 Method steps S410 to S440 in the text Figure 4 Method steps S510 to S520 in the text Figure 5 Method steps S600, Figure 6 Method step S700.

[0135] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0136] In addition, one embodiment of the present invention provides a shooting device, including the shooting device of the above embodiment or the controller of the above embodiment.

[0137] Specifically, the shooting device in this embodiment of the invention can be a terminal device, such as a mobile phone, tablet computer, wearable shooting device, etc. The shooting device itself can be equipped with a camera for capturing images.

[0138] It should also be understood that in some other embodiments of the present invention, the shooting device in the above embodiments may not be a terminal shooting device, but a desktop computer or server capable of shooting.

[0139] Furthermore, one embodiment of the present invention also provides a computer-readable storage medium storing computer-executable instructions, which, when used to perform the above-described shooting method, for example, perform the actions described above... Figure 2 Method steps S100 to S400 in the text Figure 3 Method steps S410 to S440 in the text Figure 4 Method steps S510 to S520 in the text Figure 5 Method steps S600, Figure 6 Method step S700.

[0140] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0141] The above provides a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A shooting method, comprising: Get a preview image of the current scene; Feature extraction is performed on the preview image to obtain the brightness feature information of the preview image; The brightness feature information is input into the strategy generation model to obtain the exposure strategy, wherein the strategy generation model is trained by a neural network based on the brightness feature information corresponding to the sample image; The current scene is captured based on the exposure strategy described above; The step of extracting features from the preview image to obtain the brightness feature information of the preview image includes: The preview image is divided into multiple sub-regions, and multiple target sub-regions are determined from the multiple sub-regions; For each target sub-region, a cumulative brightness distribution value is calculated based on the brightness value of the target sub-region, and the brightness interval ratio of the target sub-region is obtained based on the cumulative brightness distribution value and a preset brightness threshold. Obtain the brightness position weight of each target sub-region; The brightness feature information of the preview image is calculated based on the brightness position weights and brightness interval ratios of all the target sub-regions. The preset brightness threshold includes a preset high brightness threshold and a preset low brightness threshold; Correspondingly, obtaining the brightness interval ratio of the target sub-region based on the cumulative brightness distribution value and the preset brightness threshold includes: obtaining the high brightness interval ratio of the target sub-region based on the cumulative brightness distribution value and the preset high brightness threshold, and obtaining the low brightness interval ratio of the target sub-region based on the cumulative brightness distribution value and the preset low brightness threshold. Correspondingly, the step of calculating the brightness feature information of the preview image based on the brightness position weights and brightness interval ratios of all the target sub-regions includes: calculating the high brightness interval feature information of the preview image based on the brightness position weights and high brightness interval ratios of all the target sub-regions, and calculating the low brightness interval feature information of the preview image based on the brightness position weights and low brightness interval ratios of all the target sub-regions. The shooting method further includes: calculating the brightness proportion weight based on the brightness interval ratio of the target sub-region; Correspondingly, the step of calculating the brightness feature information of the preview image based on the brightness position weight and brightness interval ratio of all the target sub-regions includes: calculating the brightness feature information of the preview image based on the brightness position weight, brightness ratio weight and brightness interval ratio of all the target sub-regions.

2. The shooting method according to claim 1, characterized in that, The brightness position weight of each target sub-region is obtained as follows: Obtain the distance information between the target sub-region and the preset position in the preview image; The brightness position weight of each target sub-region is calculated based on the distance information.

3. The shooting method according to claim 1, characterized in that, The number of the sub-regions is In this case, the number of target sub-regions is ,in, ≤ , ≤ ; The cumulative brightness distribution value is: When the preset highlight threshold is The preset low brightness threshold is ,in, ; The proportion of the highlighted area is obtained by the following formula: , wherein The ratio of the highlighted area; The proportion of the low-brightness region is obtained by the following formula: , wherein This represents the proportion of the low-brightness region.

4. The shooting method according to claim 3, characterized in that, The brightness position weight is obtained by the following formula: Among them, the The brightness position weight, the This is an adjustable parameter.

5. The shooting method according to claim 4, characterized in that, The brightness proportion weight is obtained by the following formula: ; ; Among them, the The brightness proportion weight corresponding to the proportion of the bright area, the The brightness proportion weight corresponding to the proportion of the low brightness range, the and stated This is an adjustable parameter.

6. The shooting method according to claim 5, characterized in that, The highlighted region feature information is obtained by the following formula: Among them, the The highlighted region feature information; The low-brightness region feature information is obtained by the following formula: Among them, the This refers to the feature information of the low-brightness region.

7. The shooting method according to any one of claims 1 to 6, characterized in that, When the number of target sub-regions is less than the number of sub-regions, the target sub-regions are located at the center of the preview image or around the perimeter of the preview image.

8. A shooting device, characterized in that, It includes a processor, a memory, a camera, and a display screen. The processor is connected to the memory, the camera, and the display screen. The processor includes a feature extraction unit and a strategy generation unit. The camera includes an optical camera. The shooting component is used to acquire a preview image of the current scene through the optical camera; The display screen is used to display the preview image corresponding to the current scene; The feature extraction unit is used to acquire the preview image displayed on the display screen, and to extract features from the preview image to obtain the brightness feature information of the preview image; The strategy generation unit is used to input the brightness feature information into the pre-trained strategy generation model in the processor to obtain the exposure strategy, wherein the strategy generation model is trained by a neural network based on the brightness feature information corresponding to the sample image; The imaging component is also used to acquire the exposure strategy and, based on the exposure strategy, capture the current scene using the optical camera; The feature extraction unit is further configured to: divide the preview image into multiple sub-regions, and determine multiple target sub-regions from the multiple sub-regions; for each target sub-region, calculate a cumulative brightness distribution value based on the brightness value of the target sub-region, and obtain the brightness interval ratio of the target sub-region based on the cumulative brightness distribution value and a preset brightness threshold; obtain the brightness position weight of each target sub-region; and calculate the brightness feature information of the preview image based on the brightness position weights and brightness interval ratios of all target sub-regions. The preset brightness threshold includes a preset high brightness threshold and a preset low brightness threshold; correspondingly, the step of obtaining the brightness interval ratio of the target sub-region based on the cumulative brightness distribution value and the preset brightness threshold includes: obtaining the high brightness interval ratio of the target sub-region based on the cumulative brightness distribution value and the preset high brightness threshold, and obtaining the low brightness interval ratio of the target sub-region based on the cumulative brightness distribution value and the preset low brightness threshold. Correspondingly, the step of calculating the brightness feature information of the preview image based on the brightness position weights and brightness interval ratios of all the target sub-regions includes: calculating the high brightness interval feature information of the preview image based on the brightness position weights and high brightness interval ratios of all the target sub-regions, and calculating the low brightness interval feature information of the preview image based on the brightness position weights and low brightness interval ratios of all the target sub-regions. The feature extraction unit is further configured to: calculate the brightness proportion weight based on the brightness interval ratio of the target sub-region; Correspondingly, the feature extraction unit is further configured to: calculate the brightness feature information of the preview image based on the brightness position weight, brightness proportion weight and brightness interval ratio of all the target sub-regions.

9. A controller, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the shooting method as described in any one of claims 1 to 7.

10. A shooting device, characterized in that, It includes the shooting device as described in claim 8 or the controller as described in claim 9.

11. A computer-readable storage medium, characterized in that, The device stores computer-executable instructions for performing the photographing method as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Image shooting brightness control method and device

    CN104301624A

  • Image processing method and device, and electronic equipment

    CN108174118A

  • Image brightness processing method and device, image processing method and device

    CN112788251A