Exposure compensation amount prediction model training method, image acquisition method and system

Abstract

The invention provides a training method of an exposure compensation amount prediction model, and an image acquisition method and system. The training method comprises the following steps: acquiring a plurality of groups of sample image sequence data and corresponding sample exposure compensation amounts; wherein each group of sample image sequence data comprises a plurality of sample images arranged according to a time sequence; and taking each group of sample image sequence data as input and the corresponding sample exposure compensation amount as output, and training to obtain an exposure compensation amount prediction model. According to the invention, the plurality of groups of sample image sequence data are obtained, each group of sample image sequence data comprises the plurality of sample images arranged according to the time sequence, and the exposure compensation amount prediction model is obtained through training according to the plurality of groups of sample image sequence data, so that the accuracy and reliability of the exposure compensation amount prediction model are improved; therefore, the accuracy and reliability of the target exposure compensation amount output by the exposure compensation amount prediction model are improved, and the quality of the target image is ensured.

Description

Technical Field

[0001] The present disclosure relates to the field of exposure control technology, and in particular to a training method for an exposure compensation amount prediction model, an image acquisition method and a system. Background Art

[0002] With the development of smart phones and camera devices, the camera function in users' mobile phones is becoming more and more important. Among them, automatic exposure makes the captured image have appropriate brightness, which is the basis of high-quality and high-definition imaging. The light intensity of different scenes in nature varies greatly. The human visual system has the ability to adapt quickly, but the image sensor does not have the ability to adapt.

[0003] In order to obtain an image with appropriate brightness, the general steps are to measure the light intensity, perform scene analysis, adjust the target brightness, and converge to obtain the appropriate exposure. The light intensity measurement comes from the ambient light information and the image brightness. The scene analysis determines whether there is backlighting or strong front light. The more accurate the information analysis is, the more conducive it is to improving the usability of the image sensor. The commonly used scene analysis technology for automatic exposure is fuzzy logic or the introduction of detection or segmentation results. After the scene analysis is completed, the target brightness is set based on the debugging experience, and the exposure compensation amount obtained based on experience is less accurate. After the target brightness is set, it is necessary to converge and adjust the exposure so that the brightness of the image meets the set target brightness. Convergence is prone to overshoot or oscillation, and a certain number of frames are required to complete the convergence process.

[0004] During the preview and video recording process, the movement of the target object, the change in the angle of shooting the target object, and the change in the environment can easily lead to changes in the target brightness. The change in target brightness triggers convergence, resulting in poor stability in the image frame timing and low image quality. Summary of the invention

[0005] The technical problem to be solved by the present disclosure is to overcome the defects of low accuracy of exposure compensation and low image quality in the prior art of obtaining exposure compensation amount based on experience for exposure control, and to provide a training method for an exposure compensation amount prediction model, an image acquisition method and a system.

[0006] The present invention solves the above technical problems through the following technical solutions:

[0007] The present disclosure provides a training method for an exposure compensation amount prediction model, the training method comprising:

[0008] Obtaining several groups of sample image sequence data and corresponding sample exposure compensation amounts;

[0009] Wherein, each group of the sample image sequence data includes a number of sample images arranged in time sequence;

[0010] Each group of sample image sequence data is taken as input, and the corresponding sample exposure compensation amount is taken as output, and the exposure compensation amount prediction model is obtained through training.

[0011] Optionally, the step of taking each group of sample image sequence data as input and the corresponding sample exposure compensation amount as output, and training to obtain the exposure compensation amount prediction model comprises:

[0012] Inputting each group of the sample image sequence data into the exposure compensation amount prediction model to output a predicted exposure compensation amount;

[0013] Obtaining a first objective function based on the predicted exposure compensation amount and the corresponding sample exposure compensation amount;

[0014] Based on the first objective function, the exposure compensation amount prediction model is obtained through iterative training.

[0015] Optionally, the step of taking each group of sample image sequence data as input and the corresponding sample exposure compensation amount as output, and training to obtain the exposure compensation amount prediction model comprises:

[0016] Inputting each group of the sample image sequence data into the exposure compensation amount prediction model to output a predicted exposure compensation amount;

[0017] Based on the predicted exposure compensation amount, obtaining a predicted value of brightness information of a corresponding image;

[0018] Obtaining a sample value of brightness information of an image corresponding to the sample exposure compensation amount;

[0019] Based on the predicted value and the corresponding sample value, a second objective function is obtained;

[0020] Based on the second objective function, the exposure compensation amount prediction model is obtained by iterative training.

[0021] Optionally, the step of obtaining the sample value of the brightness information of the image corresponding to the sample exposure compensation amount includes:

[0022] Obtaining a region of interest of the image corresponding to the sample exposure compensation amount;

[0023] Acquire the sample value of the brightness information corresponding to the region of interest;

[0024] and / or,

[0025] The brightness information includes at least one of weighted average brightness, average brightness, brightness histogram information, dynamic range distribution information, color temperature information and color distribution information.

[0026] Optionally, before the step of iteratively training based on the second objective function to obtain the exposure compensation amount prediction model, the step further includes:

[0027] Obtaining a first objective function based on the predicted exposure compensation amount and the corresponding sample exposure compensation amount;

[0028] The step of iteratively training the exposure compensation amount prediction model based on the second objective function includes:

[0029] Based on the first objective function and the second objective function, the exposure compensation amount prediction model is obtained through iterative training.

[0030] Optionally, the step of iteratively training to obtain the exposure compensation amount prediction model based on the first objective function and the second objective function includes:

[0031] The sum of the first objective function and the second objective function is used as a third objective function;

[0032] Based on the third objective function, the exposure compensation amount prediction model is obtained through iterative training.

[0033] Optionally, the brightness of some of the sample images is within a preset brightness range;

[0034] and / or,

[0035] The difference between the brightness of the sample images is within a preset brightness difference range;

[0036] and / or,

[0037] The timing of the image corresponding to the sample exposure compensation amount is after the timing of the last sample image;

[0038] and / or,

[0039] Each group of sample image sequence data corresponds to a sample video.

[0040] The present disclosure also provides an image acquisition method, the image acquisition method comprising:

[0041] Inputting the target image sequence data into the exposure compensation amount prediction model to output the target exposure compensation amount;

[0042] Wherein, the exposure compensation amount prediction model is obtained based on the training method of the exposure compensation amount prediction model described above;

[0043] Based on the target exposure compensation amount, a target image is obtained.

[0044] The present disclosure also provides a training system for an exposure compensation amount prediction model, the training system comprising:

[0045] A data acquisition module, used to acquire several groups of sample image sequence data and corresponding sample exposure compensation amounts;

[0046] Wherein, each group of the sample image sequence data includes a number of sample images arranged in time sequence;

[0047] The model training module is used to take each group of sample image sequence data as input and the corresponding sample exposure compensation amount as output, and train to obtain the exposure compensation amount prediction model.

[0048] Optionally, the model training module includes:

[0049] A first output unit, configured to input each group of the sample image sequence data into the exposure compensation amount prediction model to output a predicted exposure compensation amount;

[0050] A first function acquisition unit, configured to obtain a first objective function based on the predicted exposure compensation amount and the corresponding sample exposure compensation amount;

[0051] The first training unit is used to iteratively train the exposure compensation amount prediction model based on the first objective function.

[0052] Optionally, the model training module includes:

[0053] a second output unit, configured to input each group of the sample image sequence data into the exposure compensation amount prediction model to output a predicted exposure compensation amount;

[0054] A prediction value acquisition unit, configured to obtain a prediction value of brightness information of a corresponding image based on the predicted exposure compensation amount;

[0055] A sample value acquisition unit, used to acquire a sample value of brightness information of an image corresponding to the sample exposure compensation amount;

[0056] A second function acquisition unit, used for obtaining a second objective function based on the predicted value and the corresponding sample value;

[0057] The second training unit is used to iteratively train the exposure compensation amount prediction model based on the second objective function.

[0058] Optionally, the sample value acquiring unit includes:

[0059] A region acquisition subunit, used for acquiring a region of interest of the image corresponding to the sample exposure compensation amount;

[0060] A sample value acquisition subunit, used to acquire the sample value of the brightness information corresponding to the region of interest;

[0061] and / or,

[0062] The brightness information includes at least one of weighted average brightness, average brightness, brightness histogram information, dynamic range distribution information, color temperature information and color distribution information.

[0063] Optionally, the model training module also includes:

[0064] A third function acquisition unit, configured to obtain a first objective function based on the predicted exposure compensation amount and the corresponding sample exposure compensation amount;

[0065] The second training unit is further used to iteratively train the exposure compensation amount prediction model based on the first objective function and the second objective function.

[0066] Optionally, the second training unit includes:

[0067] a function acquisition subunit, configured to take the sum of the first objective function and the second objective function as a third objective function;

[0068] A training subunit is used to iteratively train the exposure compensation amount prediction model based on the third objective function.

[0069] Optionally, the brightness of some of the sample images is within a preset brightness range;

[0070] and / or,

[0071] The difference between the brightness of the sample images is within a preset brightness difference range;

[0072] and / or,

[0073] The timing of the image corresponding to the sample exposure compensation amount is after the timing of the last sample image;

[0074] and / or,

[0075] Each group of sample image sequence data corresponds to a sample video.

[0076] The present disclosure also provides an image acquisition system, the image acquisition system comprising:

[0077] A compensation amount output module, used for inputting target image sequence data into an exposure compensation amount prediction model to output a target exposure compensation amount;

[0078] Wherein, the exposure compensation amount prediction model is obtained based on the training system of the exposure compensation amount prediction model described above;

[0079] The target image acquisition module is used to obtain a target image based on the target exposure compensation amount.

[0080] The present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the above-mentioned training method of the exposure compensation amount prediction model, or implements the above-mentioned image acquisition method.

[0081] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements the above-mentioned method for training the exposure compensation amount prediction model, or implements the above-mentioned image acquisition method.

[0082] The present disclosure also provides a computer program product, including a computer program, which, when executed by a processor, implements the training method of the exposure compensation amount prediction model as described above, or implements the image acquisition method as described above.

[0083] The present disclosure also provides a chip, which includes at least one processor, and the processor is used to execute program instructions to execute the training method of the exposure compensation amount prediction model as described above, or to execute the image acquisition method as described above.

[0084] The present disclosure also provides a chip module, which is applied to an electronic device, including a transceiver component and a chip, wherein the chip is used to execute the training method of the exposure compensation amount prediction model as described above, or to execute the image acquisition method as described above.

[0085] On the basis of being in accordance with the common sense in the art, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present disclosure.

[0086] The positive and progressive effects of this disclosure are:

[0087] The present disclosure obtains several groups of sample image sequence data, each group of sample image sequence data includes several sample images arranged in time sequence, and trains an exposure compensation amount prediction model based on the several groups of sample image sequence data, thereby improving the accuracy and reliability of the exposure compensation amount prediction model, thereby improving the accuracy and reliability of the target exposure compensation amount output by the exposure compensation amount prediction model, and ensuring the quality of the target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 This is a flowchart of a method for training an exposure compensation amount prediction model according to Embodiment 1 of the present disclosure;

[0089] Figure 2 This is a first flow chart of step S12 in the method for training the exposure compensation amount prediction model of embodiment 2 of the present disclosure;

[0090] Figure 3A second flow chart of step S12 in the method for training the exposure compensation amount prediction model of embodiment 2 of the present disclosure;

[0091] Figure 4 This is a flowchart of step S126 in the method for training the exposure compensation amount prediction model of embodiment 2 of the present disclosure;

[0092] Figure 5 The third flow chart of step S12 in the method for training the exposure compensation amount prediction model of embodiment 2 of the present disclosure;

[0093] Figure 6 This is a flowchart of step S1282 in the method for training the exposure compensation amount prediction model of embodiment 2 of the present disclosure;

[0094] Figure 7 This is a specific example diagram of extracting sample image sequence data from a sample video in Embodiment 2 of the present disclosure;

[0095] Figure 8 This is a specific example diagram of the training method of the exposure compensation amount prediction model of Embodiment 2 of the present disclosure;

[0096] Fig. 9 This is a flowchart of the image acquisition method of Embodiment 3 of the present disclosure;

[0097] Fig.10 A schematic diagram of a module of a training system for an exposure compensation amount prediction model according to Embodiment 4 of the present disclosure;

[0098] Fig.11 A schematic diagram of a module of a training system for an exposure compensation amount prediction model according to Embodiment 5 of the present disclosure;

[0099] Fig.12 This is a module schematic diagram of an image acquisition system according to Embodiment 6 of the present disclosure;

[0100] Fig.13 This is a schematic diagram of the structure of an electronic device according to Embodiment 7 of the present disclosure. DETAILED DESCRIPTION

[0101] The present disclosure is further described below by way of examples, but the present disclosure is not limited to the scope of the examples.

[0102] Prefixes such as "first" and "second" are used in the embodiments of the present disclosure only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers to distinguish description objects in the embodiments of the present disclosure does not constitute a limitation on the described objects. For the statement of the described objects, please refer to the description in the context of the embodiments, and no unnecessary limitation should be constituted due to the use of such prefixes. In addition, in the description of the present embodiment, unless otherwise specified, the meaning of "plurality" is two or more.

[0103] In the embodiments of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0104] Example 1

[0105] This embodiment provides a training method for an exposure compensation amount prediction model. Figure 1 As shown, the training method includes:

[0106] S11, obtaining several groups of sample image sequence data and corresponding sample exposure compensation amounts;

[0107] Wherein, each set of sample image sequence data includes a number of sample images arranged in time sequence;

[0108] S12. Taking each group of sample image sequence data as input and the corresponding sample exposure compensation amount as output, training is performed to obtain an exposure compensation amount prediction model.

[0109] Specifically, K groups of sample data are collected, each group of sample data includes L frames of image data, each frame of image data is recorded as I, its width is W, its height is H, and the number of bits of a single channel is n bits. From the L frames of image data, m frames of sample images are selected continuously in time sequence or m frames of sample images are selected in time sequence intervals, and each m frame of sample image is used as a group of sample image sequence data. K, L, W, H, n, m can be set or adjusted according to actual conditions. Among them, K, L, m are all positive integers, and L>m. In addition, the brightness information and exposure compensation amount of each group of sample image sequence data can also be obtained. The sample image sequence data and its brightness information and exposure compensation amount are the input data of the preset network. The preset network performs deep network learning on the brightness information and exposure compensation amount in time sequence to obtain an exposure compensation amount prediction model, and the exposure compensation amount prediction model is used to predict the predicted exposure compensation amount of the image corresponding to the sample exposure compensation amount.

[0110] In this embodiment, by acquiring several groups of sample image sequence data, each group of sample image sequence data includes several sample images arranged in time sequence, and training an exposure compensation amount prediction model based on the several groups of sample image sequence data, the accuracy and reliability of the exposure compensation amount prediction model are improved, thereby improving the accuracy and reliability of the target exposure compensation amount output by the exposure compensation amount prediction model, thereby ensuring the quality of the target image.

[0111] The execution subject of the training method of the exposure compensation amount prediction model provided in this embodiment may be a separate chip, a chip module or an electronic device, or a chip or a chip module integrated in an electronic device.

[0112] Example 2

[0113] This embodiment provides a training method for an exposure compensation amount prediction model, which is a further improvement on Embodiment 1.

[0114] In one feasible solution, Figure 2 As shown, step S12 includes:

[0115] S121, inputting each group of sample image sequence data into an exposure compensation amount prediction model to output a predicted exposure compensation amount;

[0116] S122, obtaining a first objective function based on the predicted exposure compensation amount and the corresponding sample exposure compensation amount;

[0117] S123. Based on the first objective function, iteratively train to obtain an exposure compensation amount prediction model.

[0118] Specifically, the sum of any one of the difference, ratio, ratio of the difference to the predicted exposure compensation amount, and ratio of the difference to the sample exposure compensation amount of each group of predicted exposure compensation amounts and sample exposure compensation amounts is used as the first objective function. Iterative training is performed to minimize the first objective function to obtain an exposure compensation amount prediction model. The calculation formula corresponding to the first objective function is as follows:

[0119]

[0120] or,

[0121]

[0122] or,

[0123]

[0124] or,

[0125]

[0126] Where z represents the first objective function, i represents the i-th group of sample image sequence data, m represents the number of groups of sample image sequence data, and x i represents the predicted exposure compensation corresponding to the i-th group of sample image sequence data Compensation, y i Represents the sequence of the i-th group of sample images The sample exposure compensation amount corresponding to the column data, x i -y i Indicates The difference between the predicted exposure compensation amount corresponding to the i group of sample image sequence data and the sample exposure compensation amount.

[0127] In this solution, a first objective function is obtained according to the predicted exposure compensation amount and the corresponding sample exposure compensation amount, and then an exposure compensation amount prediction model is obtained by iterative training according to the first objective function, thereby ensuring the accuracy and reliability of the exposure compensation amount prediction model.

[0128] In another feasible solution, Figure 3 As shown, step S12 includes:

[0129] S124, inputting each group of sample image sequence data into an exposure compensation amount prediction model to output a predicted exposure compensation amount;

[0130] S125, obtaining a predicted value of brightness information of a corresponding image based on the predicted exposure compensation amount;

[0131] S126, obtaining a sample value of brightness information of an image corresponding to the sample exposure compensation amount;

[0132] S127, obtaining a second objective function based on the predicted value and the corresponding sample value;

[0133] S128. Based on the second objective function, iteratively train to obtain an exposure compensation amount prediction model.

[0134] Specifically, a corresponding image can be generated according to the predicted exposure compensation amount, and the brightness information of the corresponding image can be obtained as a predicted value. The brightness information, that is, the brightness feature, can be a feature vector extracted from the corresponding image using an algorithm, such as extracting an edge through a convolutional network to obtain a feature vector. The feature vector can be gradient information or edge information. The sum of any one of the difference, ratio, ratio of the difference to the predicted value, and ratio of the difference to the sample value of each group of brightness information is used as the second objective function. Iterative training is performed to minimize the second objective function to obtain an exposure compensation amount prediction model. The calculation formula corresponding to the second objective function is as follows:

[0135]

[0136] or,

[0137]

[0138] or,

[0139]

[0140] or,

[0141]

[0142] Among them, e represents the second objective function, i represents the i-th group of sample image sequence data, and m represents the sample The number of image sequences The number of groups of data, f i represents the predicted value corresponding to the i-th group of sample image sequence data, g i surface represents the sample value corresponding to the i-th group of sample image sequence data, f i -g i Represents the difference between the predicted value and the sample value corresponding to the i-th group of sample image sequence data.

[0143] In this solution, a second objective function is obtained according to the predicted value of the brightness information and the corresponding sample value, and then an exposure compensation amount prediction model is obtained by iterative training according to the second objective function, thereby ensuring the accuracy and reliability of the exposure compensation amount prediction model.

[0144] In one feasible solution, Figure 4 As shown, step S126 includes:

[0145] S1261, obtaining a region of interest of the image corresponding to the sample exposure compensation amount;

[0146] S1262: Obtain sample values ​​of brightness information corresponding to the region of interest.

[0147] Specifically, the region of interest is the statistical region. If the sample exposure compensation value corresponds to an image with no face, the entire image is used as the region of interest; if the sample exposure compensation value corresponds to an image with a face, the face area is used as the region of interest. The area selected by manual touch can also be used as the region of interest. The sample value is the ground truth.

[0148] In this solution, by obtaining the area of ​​interest of the image corresponding to the sample exposure compensation amount, the sample value of the brightness information corresponding to the area of ​​interest is obtained, thereby ensuring the reliability and validity of the sample value.

[0149] In an implementable solution, the brightness information includes at least one of weighted average brightness, average brightness, brightness histogram information, dynamic range distribution information, color temperature information, and color distribution information.

[0150] In this solution, by setting various forms of brightness information, the diversity of the predicted values ​​and sample values ​​of the brightness information is guaranteed, and the practicality of the exposure compensation amount prediction model is improved.

[0151] In one feasible solution, Figure 5 As shown, before step S128, it also includes:

[0152] S1281, obtaining a first objective function based on the predicted exposure compensation amount and the corresponding sample exposure compensation amount;

[0153] Step S128 includes:

[0154] S1282. Based on the first objective function and the second objective function, iteratively train to obtain an exposure compensation amount prediction model.

[0155] Specifically, the exposure compensation amount prediction model performs deep network learning on the brightness features and the exposure compensation amount in time series, and iteratively trains to minimize the first objective function and the second objective function to obtain the final exposure compensation amount prediction model.

[0156] In this solution, an exposure compensation amount prediction model is obtained by iterative training according to the first objective function and the second objective function, thereby ensuring the accuracy and reliability of the exposure compensation amount prediction model.

[0157] In one feasible solution, Figure 6 As shown, step S1282 includes:

[0158] S12821. Taking the sum of the first objective function and the second objective function as the third objective function;

[0159] S12822. Based on the third objective function, iteratively train to obtain an exposure compensation amount prediction model.

[0160] Specifically, the calculation formula corresponding to the third objective function is as follows:

[0161] h=z+e;

[0162] Among them, h represents the third objective function, z represents the first objective function, and e represents the second objective function.

[0163] In this solution, the sum of the first objective function and the second objective function is used as the third objective function, and an exposure compensation amount prediction model is obtained through iterative training, thereby ensuring the accuracy and reliability of the exposure compensation amount prediction model.

[0164] In an implementable solution, the brightness of the plurality of sample images is within a preset brightness range; and / or the difference between the brightness of the plurality of sample images is within a preset brightness difference range.

[0165] Specifically, when the current environment is a relatively fixed scene, the exposure time and exposure gain are fixed to collect sample image sequence data, and the brightness of the sample image sequence data is stable. When the current environment is not a relatively fixed scene, the parameters are adjusted to make the brightness of the collected sample image sequence data stable.

[0166] In this solution, by acquiring a number of sample images whose brightness is within a preset brightness range, and / or by acquiring a number of sample images whose brightness differences are within a preset brightness difference range, that is, the brightness of each group of sample image sequence data is stable, thereby ensuring the reliability of the sample image sequence data.

[0167] In an implementable solution, the timing of the image corresponding to the sample exposure compensation amount is located after the timing of the last sample image.

[0168] Specifically, m frames of sample images are selected from L frames of image data as a group of sample image sequence data. The m frames of sample images may be arranged in time sequence as the 1st frame, the 2nd frame, the 3rd frame ... the mth frame in the image data, and the image corresponding to the sample exposure compensation amount is the m+1th frame in the image data. The L frames of image data may obtain LM groups of sample image sequence data. The m frames of sample images may also be arranged in time sequence as the 1st frame, the 3rd frame, the 5th frame ... the mth frame in the image data, and the image corresponding to the sample exposure compensation amount is the m+2th frame in the image data. That is, the image corresponding to the sample exposure compensation amount and the sample image may be selected continuously or at intervals, and the number of interval frames of the interval selection may be set or adjusted according to actual conditions. The sample exposure compensation amount is a reference true value.

[0169] In this scheme, by selecting an image whose timing is after the timing of the last sample image as the image corresponding to the sample exposure compensation amount, the accuracy and reliability of the sample exposure compensation amount are guaranteed, thereby ensuring the accuracy and reliability of the exposure compensation amount prediction model.

[0170] In an implementable solution, each set of sample image sequence data corresponds to a sample video.

[0171] Specifically, each set of sample image sequence data, that is, each L frame image data corresponds to a sample video. A total of K sample videos are collected, and each sample video includes L frames of image data. Figure 7 As shown in FIG. 1 , it is a specific example diagram of extracting sample image sequence data from a sample video. The original video sequence is the sample video, with a total of L frames of image data. Sequence 1, sequence 2, ... sequence X all include m consecutive frames of images, i.e., sample images, and the image to be predicted GT is the m+1th frame image. The first frame image of sequence 2 is the second frame image of sequence 1, and so on, that is, X sequences are obtained. That is, X groups of sample image sequence data can be extracted from 1 sample video, and X is a positive integer.

[0172] In this solution, by selecting sample image sequence data from sample videos, the accuracy and reliability of the sample image sequence data are guaranteed.

[0173] The working principle of the training method of the exposure compensation amount prediction model of this embodiment is explained below with reference to specific examples. Figure 8As shown, the steps include:

[0174] S21, collecting a number of groups of sample image sequence data with stable brightness in time sequence;

[0175] S22, determining the number of sample images in each group of sample image sequence data, for example, m frames;

[0176] S23, obtaining brightness information and exposure compensation amount of sample image sequence data;

[0177] S24, obtaining the sample value of the sample exposure compensation amount and brightness information corresponding to the image of the m+1th frame;

[0178] S25, training the exposure compensation amount prediction model to extract the temporal features of the sample image sequence data, and predicting the exposure compensation amount of the next frame, i.e., the m+1th frame;

[0179] S26, iterative training reduces the difference between the predicted value and the sample value of the brightness information of the m+1 frame, and between the predicted exposure compensation amount and the sample exposure compensation amount of the m+1 frame, that is, the first objective function and the second objective function are reduced;

[0180] S27, the first objective function and the second objective function converge, completing the parameter training of the exposure compensation amount prediction model.

[0181] In this embodiment, by acquiring several groups of sample image sequence data, each group of sample image sequence data includes several sample images arranged in time sequence, and training an exposure compensation amount prediction model based on the several groups of sample image sequence data, the accuracy and reliability of the exposure compensation amount prediction model are improved, thereby improving the accuracy and reliability of the target exposure compensation amount output by the exposure compensation amount prediction model, thereby ensuring the quality of the target image.

[0182] Example 3

[0183] This embodiment provides an image acquisition method, such as Fig. 9 As shown, the image acquisition method includes:

[0184] S31, inputting the target image sequence data into the exposure compensation amount prediction model to output the target exposure compensation amount;

[0185] The exposure compensation amount prediction model is obtained based on the above-mentioned exposure compensation amount prediction model training method;

[0186] S32: Obtain a target image based on the target exposure compensation amount.

[0187] Specifically, in the application process of the exposure compensation amount prediction model, target image sequence data is obtained in a specific scene, and the target exposure compensation amount can be inferred through the target image sequence data. The target exposure compensation amount is controlled to take effect to obtain the target image, and the brightness of the target image is the optimal brightness or the appropriate brightness in the specific scene. For example, the target image sequence data includes m frames of images, and the exposure compensation amount of the next frame in the specific scene, that is, the m+1th frame of the image, can be obtained through the exposure compensation amount prediction model.

[0188] In this scheme, the target image sequence data is input into the exposure compensation amount prediction model to obtain the target exposure compensation amount, and then the target image is obtained, which improves the accuracy and reliability of the target exposure compensation amount and ensures the quality of the target image.

[0189] The execution subject of the image acquisition method provided in this embodiment may be a separate chip, a chip module or an electronic device, or may be a chip or a chip module integrated in an electronic device.

[0190] Example 4

[0191] This embodiment provides a training system for an exposure compensation amount prediction model. Fig.10 As shown, the training system includes:

[0192] Data acquisition module 1, used to acquire several groups of sample image sequence data and corresponding sample exposure compensation values;

[0193] Wherein, each set of sample image sequence data includes a number of sample images arranged in time sequence;

[0194] The model training module 2 is used to take each group of sample image sequence data as input and the corresponding sample exposure compensation amount as output, and train to obtain an exposure compensation amount prediction model.

[0195] In this embodiment, by acquiring several groups of sample image sequence data, each group of sample image sequence data includes several sample images arranged in time sequence, and training an exposure compensation amount prediction model based on the several groups of sample image sequence data, the accuracy and reliability of the exposure compensation amount prediction model are improved, thereby improving the accuracy and reliability of the target exposure compensation amount output by the exposure compensation amount prediction model, thereby ensuring the quality of the target image.

[0196] The training system of the exposure compensation amount prediction model described in the embodiment may specifically be a separate chip, a chip module or an electronic device, or a chip or a chip module integrated in an electronic device. The various modules / units included in the exposure control may be software modules / units, hardware modules / units, or partly software modules / units and partly hardware modules / units. For example, for various devices and products applied to or integrated in a chip, the various modules / units included therein may all be implemented in the form of hardware such as circuits, or at least some of the modules / units may be implemented in the form of software programs, which run on a processor integrated inside the chip, and the remaining modules / units may be implemented in the form of hardware such as circuits; for various devices and products applied to or integrated in a chip module, the various modules / units included therein may all be implemented in the form of hardware such as circuits, and different modules / units may be located in the same component (such as a chip, circuit module, etc.) or different components of the chip module, or at least some of the modules / units may be implemented in the form of hardware such as circuits. The element can be implemented in the form of a software program, which runs on a processor integrated inside the chip module, and the remaining modules / units can be implemented in the form of hardware such as circuits; for various devices and products applied to or integrated in electronic devices, the various modules / units contained therein can be implemented in the form of hardware such as circuits, and different modules / units can be located in the same component (for example, chip, circuit module, etc.) or in different components in the terminal, or, at least some modules / units can be implemented in the form of a software program, which runs on a processor integrated inside the electronic device, and the remaining modules / units can be implemented in the form of hardware such as circuits.

[0197] Example 5

[0198] This embodiment provides a training system for an exposure compensation amount prediction model, which is a further improvement on Embodiment 4.

[0199] In one feasible solution, Fig.11 As shown, the model training module 2 includes:

[0200] The first output unit 21 is used to input each group of sample image sequence data into the exposure compensation amount prediction model to output the predicted exposure compensation amount;

[0201] A first function acquisition unit 22, configured to obtain a first objective function based on the predicted exposure compensation amount and the corresponding sample exposure compensation amount;

[0202] The first training unit 23 is used for iteratively training to obtain an exposure compensation amount prediction model based on the first objective function.

[0203] In one feasible solution, the model training module 2 includes:

[0204] The second output unit 24 is used to input each set of sample image sequence data into the exposure compensation amount prediction model to output the predicted exposure compensation amount;

[0205] A prediction value acquisition unit 25, configured to obtain a prediction value of brightness information of a corresponding image based on the predicted exposure compensation amount;

[0206] A sample value acquisition unit 26, used to acquire a sample value of brightness information of an image corresponding to a sample exposure compensation amount;

[0207] A second function acquisition unit 27, configured to obtain a second objective function based on the predicted value and the corresponding sample value;

[0208] The second training unit 28 is used for iteratively training to obtain an exposure compensation amount prediction model based on the second objective function.

[0209] In an implementable solution, the sample value acquisition unit 26 includes:

[0210] The region acquisition subunit 261 is used to acquire the region of interest of the image corresponding to the sample exposure compensation amount;

[0211] The sample value acquisition subunit 262 is used to acquire the sample value of the brightness information corresponding to the region of interest.

[0212] In an implementable solution, the brightness information includes at least one of weighted average brightness, average brightness, brightness histogram information, dynamic range distribution information, color temperature information, and color distribution information.

[0213] In an implementable solution, the model training module 2 further includes:

[0214] A third function acquisition unit 29, configured to obtain a first objective function based on the predicted exposure compensation amount and the corresponding sample exposure compensation amount;

[0215] The second training unit 28 is further used for iteratively training to obtain an exposure compensation amount prediction model based on the first objective function and the second objective function.

[0216] In one feasible solution, the second training unit 28 includes:

[0217] A function acquisition subunit 281, configured to use the sum of the first objective function and the second objective function as a third objective function;

[0218] The training subunit 282 is used to iteratively train the exposure compensation amount prediction model based on the third objective function.

[0219] In an implementable solution, the brightness of a plurality of sample images is within a preset brightness range.

[0220] In an implementable solution, the brightness difference between the plurality of sample images is within a preset brightness difference range.

[0221] In an implementable solution, the timing of the image corresponding to the sample exposure compensation amount is located after the timing of the last sample image.

[0222] In an implementable solution, each set of sample image sequence data corresponds to a sample video.

[0223] In this embodiment, by acquiring several groups of sample image sequence data, each group of sample image sequence data includes several sample images arranged in time sequence, and training an exposure compensation amount prediction model based on the several groups of sample image sequence data, the accuracy and reliability of the exposure compensation amount prediction model are improved, thereby improving the accuracy and reliability of the target exposure compensation amount output by the exposure compensation amount prediction model, thereby ensuring the quality of the target image.

[0224] Example 6

[0225] This embodiment provides an image acquisition system, such as Fig.12 As shown, the image acquisition system includes:

[0226] The compensation amount output module 3 is used to input the target image sequence data into the exposure compensation amount prediction model to output the target exposure compensation amount;

[0227] The exposure compensation amount prediction model is obtained based on the above-mentioned exposure compensation amount prediction model training system;

[0228] The target image acquisition module 4 is used to obtain the target image based on the target exposure compensation amount.

[0229] In this embodiment, the target image sequence data is input into the exposure compensation amount prediction model to obtain the target exposure compensation amount, and then the target image is obtained, which improves the accuracy and reliability of the target exposure compensation amount and ensures the quality of the target image.

[0230] As for the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The system embodiment described above is only illustrative, wherein the units described as separate components may or may not be physically separated, and the components as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution.

[0231] The image acquisition system described in the embodiment may specifically be a separate chip, a chip module or an electronic device, or a chip or a chip module integrated into an electronic device. The various modules / units included in the exposure control may be software modules / units, hardware modules / units, or partly software modules / units and partly hardware modules / units. For example, for various devices and products applied to or integrated in a chip, the various modules / units included therein may all be implemented in the form of hardware such as circuits, or at least some of the modules / units may be implemented in the form of software programs, which run on a processor integrated inside the chip, and the remaining modules / units may be implemented in the form of hardware such as circuits; for various devices and products applied to or integrated in a chip module, the various modules / units included therein may all be implemented in the form of hardware such as circuits, and different modules / units may be located in the same component (such as a chip, circuit module, etc.) or different components of the chip module, or at least some of the modules / units may be implemented in the form of hardware such as circuits. The element can be implemented in the form of a software program, which runs on a processor integrated inside the chip module, and the remaining modules / units can be implemented in the form of hardware such as circuits; for various devices and products applied to or integrated in electronic devices, the various modules / units contained therein can be implemented in the form of hardware such as circuits, and different modules / units can be located in the same component (for example, chip, circuit module, etc.) or in different components in the terminal, or, at least some modules / units can be implemented in the form of a software program, which runs on a processor integrated inside the electronic device, and the remaining modules / units can be implemented in the form of hardware such as circuits.

[0232] Example 7

[0233] like Fig.13 As shown, it is a schematic diagram of the structure of an electronic device provided in Embodiment 7 of the present disclosure, the electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor, and when the processor executes the computer program, the training method or image acquisition method of the exposure compensation amount prediction model of the above-mentioned embodiment is implemented. Fig.13 The electronic device 90 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0234] The electronic device 90 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 90 may include, but are not limited to: at least one processor 91, at least one memory 92, and a bus 93 connecting different system components (including the memory 92 and the processor 91).

[0235] The bus 93 includes a data bus, an address bus, and a control bus.

[0236] The memory 92 may include a volatile memory, such as a random access memory (RAM) 921 and / or a cache memory 922 , and may further include a read-only memory (ROM) 923 .

[0237] The memory 92 may also include a program tool 925 (or utility) having a set (at least one) of program modules 924, such program modules 924 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0238] The processor 91 executes various functional applications and data processing by running the computer programs stored in the memory 92, such as the training method or image acquisition method of the exposure compensation amount prediction model provided in the above embodiments.

[0239] The electronic device 90 may also communicate with one or more external devices 94 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 95. Furthermore, the electronic device 90 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 96. As shown, the network adapter 96 communicates with other modules of the electronic device 90 via a bus 93. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 90, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0240] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0241] Example 8

[0242] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the exposure compensation amount prediction model training method or image acquisition method provided in any of the above embodiments.

[0243] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0244] Example 9

[0245] The embodiment of the present disclosure also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for training an exposure compensation amount prediction model or image acquisition methods.

[0246] Among them, the program code for executing the computer program product of the present disclosure can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.

[0247] Although the specific embodiments of the present disclosure are described above, those skilled in the art should understand that this is only an example, and the protection scope of the present disclosure is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present disclosure, but these changes and modifications all fall within the protection scope of the present disclosure.

Claims

1. A method for training an exposure compensation amount prediction model, characterized in that: The training method comprises: Obtaining several groups of sample image sequence data and corresponding sample exposure compensation amounts; Wherein, each group of the sample image sequence data includes a number of sample images arranged in time sequence; Each group of sample image sequence data is taken as input, and the corresponding sample exposure compensation amount is taken as output, and the exposure compensation amount prediction model is obtained through training.

2. The method for training the exposure compensation amount prediction model according to claim 1, wherein: The step of taking each group of sample image sequence data as input and the corresponding sample exposure compensation amount as output, and training to obtain the exposure compensation amount prediction model comprises: Inputting each group of the sample image sequence data into the exposure compensation amount prediction model to output a predicted exposure compensation amount; Obtaining a first objective function based on the predicted exposure compensation amount and the corresponding sample exposure compensation amount; Based on the first objective function, the exposure compensation amount prediction model is obtained through iterative training.

3. The method for training the exposure compensation amount prediction model according to claim 1, wherein: The step of taking each group of sample image sequence data as input and the corresponding sample exposure compensation amount as output, and training to obtain the exposure compensation amount prediction model comprises: Inputting each group of the sample image sequence data into the exposure compensation amount prediction model to output a predicted exposure compensation amount; Based on the predicted exposure compensation amount, obtaining a predicted value of brightness information of a corresponding image; Obtaining a sample value of brightness information of an image corresponding to the sample exposure compensation amount; Based on the predicted value and the corresponding sample value, a second objective function is obtained; Based on the second objective function, the exposure compensation amount prediction model is obtained by iterative training.

4. The method for training the exposure compensation amount prediction model according to claim 3, wherein: The step of obtaining the sample value of the brightness information of the image corresponding to the sample exposure compensation amount comprises: Obtaining a region of interest of the image corresponding to the sample exposure compensation amount; Acquire the sample value of the brightness information corresponding to the region of interest; and / or, The brightness information includes at least one of weighted average brightness, average brightness, brightness histogram information, dynamic range distribution information, color temperature information and color distribution information.

5. The method for training the exposure compensation amount prediction model according to claim 3 or 4, characterized in that: Before the step of iteratively training to obtain the exposure compensation amount prediction model based on the second objective function, the method further includes: Obtaining a first objective function based on the predicted exposure compensation amount and the corresponding sample exposure compensation amount; The step of iteratively training the exposure compensation amount prediction model based on the second objective function includes: Based on the first objective function and the second objective function, the exposure compensation amount prediction model is obtained through iterative training.

6. The method for training the exposure compensation amount prediction model according to claim 5, wherein: The step of iteratively training to obtain the exposure compensation amount prediction model based on the first objective function and the second objective function includes: The sum of the first objective function and the second objective function is used as a third objective function; Based on the third objective function, the exposure compensation amount prediction model is obtained through iterative training.

7. The method for training an exposure compensation amount prediction model according to any one of claims 1 to 4, characterized in that: The brightness of some of the sample images is within a preset brightness range; and / or, The difference between the brightness of the sample images is within a preset brightness difference range; and / or, The timing of the image corresponding to the sample exposure compensation amount is after the timing of the last sample image; and / or, Each group of sample image sequence data corresponds to a sample video.

8. An image acquisition method, characterized in that: The image acquisition method comprises: Inputting the target image sequence data into the exposure compensation amount prediction model to output the target exposure compensation amount; Wherein, the exposure compensation amount prediction model is obtained based on the training method of the exposure compensation amount prediction model described in any one of claims 1 to 7; Based on the target exposure compensation amount, a target image is obtained.

9. A training system for an exposure compensation amount prediction model, characterized in that: The training system comprises: A data acquisition module, used to acquire several groups of sample image sequence data and corresponding sample exposure compensation amounts; Wherein, each group of the sample image sequence data includes a number of sample images arranged in time sequence; The model training module is used to take each group of sample image sequence data as input and the corresponding sample exposure compensation amount as output, and train to obtain the exposure compensation amount prediction model.

10. An image acquisition system, characterized in that: The image acquisition system comprises: A compensation amount output module, used for inputting target image sequence data into an exposure compensation amount prediction model to output a target exposure compensation amount; Wherein, the exposure compensation amount prediction model is obtained based on the training system of the exposure compensation amount prediction model according to claim 9; The target image acquisition module is used to obtain a target image based on the target exposure compensation amount.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, characterized in that: When the processor executes the computer program, the processor implements the training method of the exposure compensation amount prediction model described in any one of claims 1 to 7, or implements the image acquisition method described in claim 8.

Images