Image exposure quantification method and electronic equipment

Abstract

The invention discloses an image exposure quantification method and electronic equipment, and relates to the technical field of image processing, and the method comprises the steps: obtaining an input image; wherein the input image comprises one or more images to be subjected to exposure quantization. And inputting the input image into the image-text comparison model, and obtaining a semantic similarity value between the input image and a preset cue word. Wherein the preset prompt word comprises one or more description fields representing the exposure state. And determining an exposure quantization tag of the input image based on the semantic similarity value of the input image corresponding to the preset cue word. Wherein the exposure quantization label is used for representing that the input image is an underexposure image, a normal exposure image or an overexposure image. In the scheme, the image-text comparison model can learn the semantic relationship between the preset cue word (text) and the image, so that the semantic similarity value of the image and the preset cue word is obtained. And the exposure state label of the image obtained based on the semantic similarity value is objective and accurate.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to an image exposure quantization method and electronic device. Background Art

[0002] Image exposure refers to the process in which a photosensitive element (such as a camera sensor or film) receives light and converts it into an image signal during the image acquisition process. Image exposure involves multiple factors such as light intensity, exposure time, and sensitivity of the photosensitive element, which have a decisive influence on the brightness, contrast, and detail of the image. Overexposure of the image will cause the image to be brighter overall and details in the highlights to be lost; underexposure of the image will cause the image to be darker overall and details to be lost in the shadows.

[0003] In image processing scenarios or some shooting scenarios, it is necessary to judge the exposure of the image as underexposure, normal exposure or overexposure. The exposure judgment method provided by traditional technology, such as manual exposure judgment, has the problem of low judgment accuracy. Summary of the invention

[0004] The embodiment of the present application provides an image exposure quantification method and electronic device, which can obtain the similarity value between the input image and the prompt word through a trained image-text comparison model. The prompt word includes a field for characterizing different exposure states. Based on the similarity value, the exposure state label of the input image can be determined, avoiding the inaccurate exposure determination caused by the subjective ambiguity of manual exposure determination, and the complex process of exposure determination by finding a reference object and the inaccuracy of exposure determination, and the exposure quantification result of the input image obtained is more accurate.

[0005] In order to achieve the above-mentioned purpose, the embodiments of the present application adopt the following technical solutions.

[0006] In a first aspect, a method for quantifying image exposure is provided, comprising:

[0007] An input image is obtained, wherein the input image includes one or more images to be subjected to exposure quantization.

[0008] The input image is input into the image-text comparison model to obtain the semantic similarity value between the input image and the preset prompt word, wherein the preset prompt word includes one or more description fields representing the exposure state.

[0009] The model predicts the exposure quantization label (training label) of the sample image based on the sample image and the training prompt word, and is iteratively trained based on the loss of the training label and the reference label (actual exposure status) of the sample image. The model can output the semantic similarity between the image features and the text (prompt word) features, thereby achieving the purpose of understanding the exposure status of the image.

[0010] Based on the semantic similarity value of the input image corresponding to the preset prompt word, an exposure quantization label of the input image is determined, wherein the exposure quantization label is used to characterize whether the input image is an underexposed image, a normally exposed image, or an overexposed image.

[0011] Among them, the electronic device can serve as the execution subject of the image exposure quantification method.

[0012] In the present application, the semantic similarity value between the input image and the preset prompt word is obtained through a trained image-text comparison model, wherein the preset prompt word includes fields describing different exposure states. The exposure state label of the input image can be determined based on the semantic similarity value. Among them, the image-text comparison model can learn the semantic relationship between the preset prompt word (text) and the image, so as to obtain the semantic similarity value between the image and the preset prompt word. The exposure state label of the image obtained based on the semantic similarity value is objective and accurate. The subjective ambiguity of manual evaluation is avoided, and the entire process does not require the search for a reference benchmark image, nor does it require the formulation of complex multiple quantization strategies. While ensuring the accuracy of exposure quantification, it also reduces computing power.

[0013] In a possible implementation manner of the first aspect, the input image includes an image to be subjected to exposure quantization, and the preset prompt words include a plurality of prompt words describing different exposure states.

[0014] The electronic device inputs the input image into the image-text comparison model to obtain the semantic similarity value between the input image and the preset prompt word, including:

[0015] The electronic device inputs the input image into the image-text comparison model, obtains a first similarity value of the input image corresponding to each preset prompt word, and obtains a plurality of first similarity values.

[0016] Then, the electronic device determines the exposure quantization label of the input image based on the semantic similarity value, including:

[0017] The electronic device determines an exposure quantization label of the input image based on the exposure state corresponding to the preset prompt word with the largest first similarity value.

[0018] In the present application, the semantic similarity value between the input image and the preset prompt word is obtained through a trained image-text comparison model, wherein the preset prompt word includes a field describing different exposure states. The exposure state label of the input image can be determined based on the semantic similarity value. Among them, the image-text comparison model can learn the semantic relationship between the preset prompt word (text) and the image, thereby obtaining the semantic similarity value between the image and the preset prompt word. The larger the first similarity value, the higher the semantic similarity between the image and the preset prompt word. The exposure state corresponding to the preset prompt word with the largest first similarity value is used to determine the exposure quantization label of the input image objectively and accurately. The subjective ambiguity of manual evaluation is avoided, and the entire process does not require the search for a reference benchmark image, nor does it require the formulation of complex multiple quantization strategies. While ensuring the accuracy of exposure quantization, it also reduces computing power.

[0019] In another possible implementation manner of the first aspect, the input image includes an image to be subjected to exposure quantization, and the preset prompt words include a plurality of prompt words describing different exposure states.

[0020] The method further includes:

[0021] The electronic device performs saliency model inference on the input image to obtain a subject image corresponding to the input image.

[0022] The electronic device inputs the main image into the image-text comparison model to obtain a second similarity value of the main image corresponding to each preset prompt word.

[0023] Then, the electronic device inputs the input image into the image-text comparison model to obtain the semantic similarity value between the input image and the preset prompt word, including:

[0024] The electronic device inputs the input image into the image-text comparison model, obtains a first similarity value of the input image corresponding to each preset prompt word, and obtains a plurality of first similarity values.

[0025] Then, the electronic device determines the exposure quantization label of the input image based on the semantic similarity value, including:

[0026] Based on the first similarity value and the second similarity value, a third similarity value of the input image is determined, and the electronic device determines an exposure quantization label of the input image based on the third similarity value.

[0027] In the present application, the trained image-text comparison model can realize the semantic understanding of the correlation between the global image of the input image and the preset prompt words, as well as the semantic understanding of the correlation between the subject image corresponding to the input image and the preset prompt words. For input images containing some special scenes, the semantic understanding of the correlation between the subject image and the preset prompt words can better determine the actual exposure state of the input image, thereby making the final exposure quantization label of the input image more accurate.

[0028] In another possible implementation manner of the first aspect, the first similarity value includes semantic similarity values ​​of the input image corresponding to multiple preset prompt words, and the second similarity value includes semantic similarity values ​​of the subject image corresponding to multiple preset prompt words.

[0029] Determining a third similarity value of the input image based on the first similarity value and the second similarity value includes:

[0030] For each preset prompt word, a weighted sum calculation is performed on the first similarity value and the second similarity value to obtain a third similarity value corresponding to each preset prompt word, thereby obtaining a plurality of third similarity values.

[0031] Then, the electronic device determines the exposure quantization label of the input image based on the third similarity value, including:

[0032] The electronic device determines the exposure quantization label of the input image based on the exposure state corresponding to the preset prompt word with the largest third similarity value.

[0033] The weight corresponding to the first similarity value and the weight corresponding to the second similarity value in the weighted calculation can be determined according to actual conditions.

[0034] In the present application, a weighted sum calculation can be performed on the first similarity value and the second similarity value to obtain a third similarity value corresponding to each preset prompt word, and multiple third similarity values ​​can be obtained. Based on the exposure state corresponding to the preset prompt word with the largest third similarity value, the exposure quantization label of the input image is determined. The third similarity value combines the semantic understanding result (similarity value) of the input image (global image) and the semantic understanding result (similarity value) of the subject image. The third similarity value can more accurately characterize the exposure state of the input image, and the exposure quantization label of the input image obtained based on the third similarity value is more accurate.

[0035] In another possible implementation manner of the first aspect, the first similarity value includes semantic similarity values ​​of the input image corresponding to multiple preset prompt words, and the second similarity value includes semantic similarity values ​​of the subject image corresponding to multiple preset prompt words.

[0036] Determining a third similarity value of the input image based on the first similarity value and the second similarity value includes:

[0037] The electronic device obtains the first similarity value with the largest value and the second similarity value with the largest value.

[0038] When the preset prompt word corresponding to the largest first similarity value is consistent with the preset prompt word corresponding to the largest second similarity value, the largest first similarity value and the largest second similarity value are weightedly summed to obtain a third similarity value.

[0039] Then, the electronic device determines the exposure quantization label of the input image based on the third similarity value, including:

[0040] The electronic device determines an exposure quantization label of the input image based on the exposure state corresponding to the preset prompt word corresponding to the third similarity value.

[0041] In general, the similarity trend of the global image to the preset prompt word is consistent with the similarity trend of the local image to the preset prompt word.

[0042] In the present application, when the preset prompt word corresponding to the largest first similarity value is consistent with the preset prompt word corresponding to the largest second similarity value, the first similarity value with the largest value and the second similarity value with the largest value are weighted and summed to obtain a third similarity value. The third similarity value combines the semantic understanding result (similarity value) of the input image (global image) and the semantic understanding result (similarity value) of the subject image, and can more accurately characterize the exposure state of the subject in the input image, and the exposure quantization label of the input image obtained based on the third similarity value is more accurate.

[0043] In another possible implementation manner of the first aspect, the method further includes:

[0044] If the preset prompt word corresponding to the largest first similarity value is inconsistent with the preset prompt word corresponding to the largest second similarity value, the electronic device uses the largest second similarity value as the third similarity value.

[0045] In some other cases, there may be a situation where the similarity trend of the global image to the preset prompt word is inconsistent with the similarity trend of the local image to the preset prompt word.

[0046] In the present application, when the preset prompt word corresponding to the largest first similarity value is inconsistent with the preset prompt word corresponding to the largest second similarity value, the electronic device uses the largest second similarity value as the third similarity value. The third similarity value is based on the semantic understanding result (similarity value) of the main image, and the exposure quantization is more focused on the main body of the image. The third similarity value can more accurately characterize the exposure state of the input image, and the exposure quantization label of the input image obtained based on the third similarity value is more accurate.

[0047] In another possible implementation of the first aspect, the preset prompt words include a first prompt word, a second prompt word, and a third prompt word, the first prompt word is a prompt word representing underexposure, the second prompt word is a prompt word representing normal exposure, and the third prompt word is a prompt word representing overexposure.

[0048] In the present application, the preset prompt words may include prompt words that represent different image exposure states, and the image-text comparison model may learn text features for prompt words of different image exposure states, thereby achieving a more comprehensive understanding of the exposure state. When the image-text comparison model is used to perform semantic understanding of the image, the exposure state of the image can be matched more comprehensively, so that the similarity between the output image and the preset prompt words is more accurate, and the exposure quantization label of the image is more in line with the actual exposure state of the image and more accurate.

[0049] In another possible implementation manner of the first aspect, the input image includes a plurality of images to be subjected to exposure quantization, and the preset prompt word includes a target prompt word of a target exposure state.

[0050] The electronic device inputs the input image into the image-text comparison model to obtain the semantic similarity value between the input image and the preset prompt word, including:

[0051] The electronic device inputs each input image into the image-text comparison model to obtain a fourth similarity value of each input image corresponding to the target prompt word.

[0052] Then, the electronic device determines the exposure quantization label of the input image based on the semantic similarity value, including:

[0053] The electronic device determines the exposure quantization label of each input image based on the fourth similarity value of each input image and a preset threshold strategy.

[0054] In the present application, the semantic understanding of the correlation between the image and the target prompt word of the trained image-text comparison model can also be used to realize the query of the target image based on the target prompt word. For example, based on the similarity value between each input image and the target prompt word, and the preset threshold strategy, the exposure quantization label of each input image can be determined, and the image closer to the target prompt word can be determined from multiple input images as the target image. The exposure quantification of the input image is obtained through the image-text comparison model, and the quantification result is objective and accurate, which can meet the image exposure quantification requirements in various scenarios.

[0055] In another possible implementation manner of the first aspect, the preset threshold strategy is related to the target prompt word.

[0056] The preset threshold policies include:

[0057] When the target prompt word is a prompt word representing normal exposure, the exposure quantization label of the input image whose fourth similarity value is greater than or equal to the first threshold is a normal exposure image. The exposure quantization label of the input image whose fourth similarity value is less than the first threshold is an abnormal exposure image.

[0058] When the target prompt word is a prompt word representing underexposure, the exposure quantization label of the input image whose fourth similarity value is greater than or equal to the second threshold is an underexposed image. The fourth similarity value is less than the second threshold, and the exposure quantization label of the input image whose fourth similarity value is greater than or equal to the third threshold is a normally exposed image, and the exposure quantization label of the input image whose fourth similarity value is less than the third threshold is an overexposed image.

[0059] When the target prompt word is a prompt word representing overexposure, the exposure quantization label of the input image whose fourth similarity value is greater than or equal to the fourth threshold is an overexposed image. The fourth similarity value is less than the fourth threshold, and the exposure quantization label of the input image whose fourth similarity value is greater than or equal to the fifth threshold is a normally exposed image, and the exposure quantization label of the input image whose fourth similarity value is less than the fifth threshold is an underexposed image.

[0060] In the present application, the semantic understanding of the correlation between the image and the target prompt word of the trained image-text comparison model can also be used to realize the query of the target image based on the target prompt word. Through the method provided in the above embodiment, based on the similarity value between each input image and the target prompt word and the threshold strategy corresponding to different target prompt words, the image closer to the target prompt word can be determined from multiple input images as the target image. The exposure quantification of the input image is obtained through the image-text comparison model, and the quantification result is objective and accurate, which can meet the image exposure quantification requirements in various scenarios.

[0061] In another possible implementation manner of the first aspect, the input image includes a plurality of images to be subjected to exposure quantization, and the preset prompt word includes a target prompt word of a target exposure state.

[0062] The method further includes:

[0063] The electronic device performs saliency model inference on each input image to obtain a subject image corresponding to each input image.

[0064] The electronic device inputs each subject image into the image-text comparison model to obtain a fifth similarity value of each subject image corresponding to the target prompt word.

[0065] Then, the electronic device inputs the input image into the image-text comparison model to obtain the semantic similarity value between the input image and the preset prompt word, including:

[0066] The electronic device inputs each input image into the image-text comparison model to obtain a fourth similarity value of each input image corresponding to the target prompt word.

[0067] Then, the electronic device determines the exposure quantization label of the input image based on the semantic similarity value, including:

[0068] The electronic device determines a sixth similarity value of the input images based on the fourth similarity value corresponding to each input image and the fifth similarity value of the subject image corresponding to the input image.

[0069] The electronic device determines an exposure quantization label of each input image based on the sixth similarity value of each input image and a preset exposure threshold.

[0070] In the present application, the semantic understanding of the correlation between the image and the target prompt word of the trained image-text comparison model can also be used to realize the query of the target image based on the target prompt word. For example, when the target image is a normal exposure image, the sixth similarity value can be determined based on the similarity value between each input image and the target prompt word and the similarity value between each subject image and the target prompt word. Based on the sixth similarity value and the preset threshold strategy, the exposure quantization label of each input image is determined, and the image closer to the target prompt word is determined from multiple input images as the target image. The exposure quantification of the input image is obtained through the image-text comparison model. The quantification result is objective and accurate, and can meet the image exposure quantification requirements in various scenarios.

[0071] In another possible implementation manner of the first aspect, the electronic device determines a sixth similarity value of the input images based on the fourth similarity value corresponding to each input image and a fifth similarity value of a subject image corresponding to the input image, including:

[0072] The electronic device uses an average value of the fourth similarity value corresponding to each input image and the fifth similarity value of the subject image corresponding to the input image as a sixth similarity value of the input image.

[0073] or,

[0074] The electronic device uses a weighted sum of the fourth similarity value corresponding to each input image and the fifth similarity value of the subject image corresponding to the input image as a sixth similarity value of the input image.

[0075] In the present application, the sixth similarity value combines the semantic understanding result (similarity value) of the input image (global image) and the semantic understanding result (similarity value) of the subject image, which can more accurately characterize the exposure state of the subject in the input image. The exposure quantization label of the input image obtained based on the sixth similarity value is more accurate, and the target image obtained with the target prompt word is also more accurate.

[0076] In another possible implementation manner of the first aspect, the preset threshold strategy is related to the target prompt word.

[0077] The preset threshold policies include:

[0078] When the target prompt word is a prompt word representing normal exposure, the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the first threshold is a normal exposure image. The exposure quantization label of the input image whose sixth similarity value is less than the first threshold is an abnormal exposure image.

[0079] When the target prompt word is a prompt word representing underexposure, the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the second threshold is an underexposed image. The sixth similarity value is less than the second threshold, and the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the third threshold is a normally exposed image, and the exposure quantization label of the input image whose sixth similarity value is less than the third threshold is an overexposed image.

[0080] When the target prompt word is a prompt word representing overexposure, the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the fourth threshold is an overexposed image. The sixth similarity value is less than the fourth threshold, and the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the fifth threshold is a normally exposed image, and the exposure quantization label of the input image whose sixth similarity value is less than the fifth threshold is an underexposed image.

[0081] In the present application, the semantic understanding of the correlation between the image and the target prompt word of the trained image-text comparison model can also be used to realize the query of the target image based on the target prompt word. Through the method provided in the above embodiment, based on the similarity value between each input image and the target prompt word, the similarity value between the main image and the target prompt word, and the threshold strategy corresponding to different target prompt words, the image closer to the target prompt word can be determined from multiple input images as the target image. The exposure quantification of the input image is obtained through the image-text comparison model, and the quantification result is objective and accurate, which can meet the image exposure quantification requirements in various scenarios.

[0082] In a second aspect, the present application provides a model training method, which is applied to an electronic device, and the method further includes:

[0083] The electronic device acquires a plurality of sample images.

[0084] The electronic device inputs a plurality of sample images into an initial image-text comparison model to obtain a first training similarity value between each sample image and a plurality of first training prompt words, wherein the plurality of first training prompt words include prompt words representing different exposure states.

[0085] The electronic device determines a training label of the sample image based on the exposure state corresponding to the first training prompt word having the largest first training similarity value.

[0086] The electronic device calculates the loss based on the training label of the sample image and the reference label of the sample image; the reference label is the actual exposure state of the sample image, and the reference label is used to characterize whether the sample image is an underexposed image, a normally exposed image, or an overexposed image.

[0087] The electronic device iteratively trains the initial image-text comparison model based on the loss until the initial image-text comparison model meets the training conditions, thereby obtaining the image-text comparison model.

[0088] In the present application, the electronic device can calculate the loss between the training label of the sample image and the reference label corresponding to the sample image (for example, it can be a correlation loss), iteratively train the image-text comparison model based on the loss, adjust the model parameters and training prompt words until the training conditions are met, thereby obtaining a trained image-text comparison model. Among them, the training condition can be that the loss gradually approaches 0, or the number of iterations meets the number threshold. Model training can make the image-text comparison model have a more accurate understanding of the exposure of the image, and make the similarity between the output sample image and the training prompt words more accurate, so that the exposure status label of the image obtained based on the trained image-text comparison model is more accurate.

[0089] Compared with the conventional image exposure quantification method based on the quality assessment model, the image exposure quantification method provided in this embodiment does not need to formulate a complex quantization strategy, nor does it need to fine-tune the model based on the reference image. However, the trained image-text comparison model can still guarantee the accuracy of understanding the exposure status of the image. That is, the image exposure quantification method provided in this embodiment, while ensuring the accuracy of the output results of the image-text comparison model, only needs to adjust the model parameters (such as model weights) and training prompt words during the model training process, and does not need to fine-tune the entire model, so the model training is simpler and the model training efficiency is higher.

[0090] In a possible implementation manner of the second aspect, the method further includes:

[0091] The electronic device performs saliency model inference on a plurality of sample images to obtain a sample subject image corresponding to each sample image.

[0092] The electronic device inputs the sample subject image into the initial image-text comparison model to obtain a second training similarity value between each sample subject image and a plurality of second training prompt words, wherein the plurality of second training prompt words include prompt words representing different exposure states.

[0093] Then, based on the exposure state corresponding to the first training prompt word with the largest first training similarity value, determining the training label of the sample image includes:

[0094] For each sample image, the electronic device obtains a first training similarity value with a maximum value and a second training similarity value with a maximum value of a sample subject image corresponding to the sample image.

[0095] When the first training prompt word corresponding to the first training similarity value with the largest value of the sample image and the second training prompt word corresponding to the second training similarity value with the largest value represent the same exposure state, the electronic device determines the third training similarity value based on the largest first training similarity value and the largest second training similarity value.

[0096] The electronic device determines a training label of the sample image based on the first training prompt word or the second training prompt word corresponding to the third training similarity value.

[0097] In the present application, compared with the conventional image exposure quantization method based on the quality assessment model, there is no need to formulate a complex quantization strategy, and there is no need to fine-tune the model based on the reference image, but the trained image-text comparison model can still guarantee the accuracy of understanding the exposure status of the image. That is, the image exposure quantization method provided in this embodiment only needs to adjust the model parameters (such as model weights) and training prompt words, and there is no need to obtain image data of different exposure states for model fine-tuning. While ensuring the accuracy of the output results of the image-text comparison model, the model training process is simpler and the model training efficiency is higher. Moreover, in this embodiment, the correlation learning between the local image and the training prompt words is introduced, so that the image-text comparison model can obtain a more accurate exposure quantification result for the entire image by quantifying the exposure of the subject for some special scenes, such as images with a large area of ​​light background or a large area of ​​dark background, so that the image exposure quantification is more accurate.

[0098] In another possible implementation manner of the second aspect, determining the training label of the sample image based on the exposure state corresponding to the first training prompt word with the largest first training similarity value further includes:

[0099] If the preset prompt word corresponding to the largest first similarity value is inconsistent with the preset prompt word corresponding to the largest second similarity value, the largest second similarity value is used as the third training similarity value.

[0100] A training label of the sample image is determined based on the first training prompt word or the second training prompt word corresponding to the third training similarity value.

[0101] In this application, only the model parameters (such as model weights) and training prompt words need to be adjusted, and there is no need to obtain image data of different exposure states for fine-tuning the model. While ensuring the accuracy of the output results of the image-text comparison model, the model training process is simpler and the model training efficiency is higher. Moreover, in this embodiment, the correlation learning between local images and training prompt words is introduced, so that the image-text comparison model can obtain more accurate exposure quantification results for the entire image by quantifying the exposure of the subject for some special scenes, such as images with large areas of light background or large areas of dark background, so that the image exposure quantification is more accurate.

[0102] In a third aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any one of the methods described in the first aspect.

[0103] In a fourth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer program / instructions are executed by a processor, the steps of the method described in any one of the first aspects are implemented.

[0104] In a fifth aspect, a computer program product comprising instructions is provided, including a computer program / instructions, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect above.

[0105] In a sixth aspect, an embodiment of the present application provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute a method as described in any one of the methods in the first aspect.

[0106] It can be understood that the beneficial effects that can be achieved by the electronic device described in the third aspect, the computer-readable storage medium described in the fourth aspect, the computer program product described in the fifth aspect, and the chip described in the sixth aspect provided above can refer to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0107] Figure 1 A schematic diagram showing a comparison between a normally exposed image and an underexposed image;

[0108] Figure 2 A schematic diagram showing the comparison between a normal exposure image and an overexposed image;

[0109] Figure 3 A schematic diagram of the appropriate exposure image selected by different observers under a subjective evaluation method;

[0110] Figure 4 This is a before and after comparison of the corrected exposure of a snow scene;

[0111] Figure 5 Schematic diagram of the training pipeline for a quality assessment model for a common image exposure quantization;

[0112] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0113] Figure 7 A schematic diagram of a group of images in different exposure states provided by an embodiment of the present application;

[0114] Figure 8 A schematic diagram of a line diagram of similarity values ​​between different prompt words and images provided in an embodiment of the present application;

[0115] Fig. 9 A schematic diagram of a training path of a picture-text comparison model provided in an embodiment of the present application;

[0116] Fig.10 A schematic diagram of a flow chart of an image exposure quantification method provided in an embodiment of the present application;

[0117] Fig.11 A schematic diagram of a training path of another image-text comparison model provided in an embodiment of the present application;

[0118] Fig.12 A schematic diagram of a flow chart of another image exposure quantization method provided in an embodiment of the present application;

[0119] Fig.13 A schematic diagram of a flow chart of another image exposure quantization method provided in an embodiment of the present application;

[0120] Fig.14 A comparative schematic diagram of exposure quantitative evaluation provided in an embodiment of the present application;

[0121] Fig.15 A schematic diagram of the structure of another electronic device provided in an embodiment of the present application;

[0122] Fig.16 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0123] In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0124] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0125] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0126] Image exposure refers to the process in which a photosensitive element (such as a camera sensor or film) receives light and converts it into an image signal during the image acquisition process. Image exposure involves multiple factors such as light intensity, exposure time, and sensitivity of the photosensitive element, which have a decisive influence on the brightness, contrast, and detail of the image. Underexposure and overexposure are two common situations of improper image exposure, both of which will have a significant impact on the quality and visual effects of the image.

[0127] Specifically, the overall brightness of the image is moderate, neither too bright nor too dark. Details in both bright and dark areas can be well presented, such as the facial expressions and skin texture of the people in the photo, the wrinkles of clothes, the gloss of accessories and other details can also be accurately displayed; in landscape photos, details such as the layers of clouds in the sky, the outlines of distant mountains, and the veins of nearby flowers and plants can all be preserved, making the image look rich and complete.

[0128] Underexposure refers to the phenomenon that the image is not exposed enough, resulting in a dark image as a whole and loss of details in the shadows. In the case of underexposure, the dark areas in the image are too deep and the texture and layers cannot be distinguished. Although the bright areas are relatively normal, the overall image brightness and contrast are low.

[0129] For example, Figure 1 A schematic diagram of the comparison between a normally exposed image and an underexposed image is given.

[0130] Figure 1 The following is a comparison diagram of a normal exposure image and an under-exposed image of a cake placed on a white table (light background 1). Figure 1 (a) shows the normal exposure image. Figure 1 (b) shows an underexposed image. Figure 1 (a) and Figure 1 (b) can be felt very intuitively Figure 1 The normal exposure image shown in (a) has normal brightness as a whole. Figure 1 In the underexposed image shown in (b), the light-colored background 1 appears dark in particular.

[0131] Overexposure refers to the phenomenon that the image is overexposed, resulting in the overall image being brighter and the details of the highlights being lost. In the case of overexposure, the bright areas in the image are too bright, appearing dead white, and it is impossible to distinguish the layers and textures, while the dark areas are relatively normal, but the overall image brightness and contrast are unbalanced.

[0132] For example, Figure 2 A schematic diagram of the comparison between a normally exposed image and an overexposed image is given.

[0133] Figure 2 The following is a comparison diagram of the normal exposure image and the over-exposure image of the bracelet placed on a black table (dark background 2). Figure 2 (a) shows the normal exposure image. Figure 2 (b) shows an overexposed image. Figure 2 (a) and Figure 2 (b) can be felt very intuitively Figure 2The normal exposure image shown in (a) shows normal brightness as a whole. Figure 2 In the overexposed image shown in (b), the dark background 2 appears brighter.

[0134] In image processing scenarios or some shooting scenarios, it is necessary to quantify (i.e., evaluate) the exposure status of the image to determine whether the image is underexposed or overexposed.

[0135] Common methods for quantifying image exposure include subjective evaluation methods. Subjective evaluation methods use people as observers to make subjective judgments on the exposure status of images, thereby obtaining quantitative results of images. Although subjective evaluation methods can directly reflect the visual experience of images, they are the most intuitive way to quantify exposure; however, there may be large differences between different observers in subjective evaluation methods, and there are problems such as strong subjectivity and subjective ambiguity. In addition, the entire process of subjective evaluation methods relies on manual labor, and the evaluation process is time-consuming and labor-intensive.

[0136] For example, Figure 3 A schematic diagram of the appropriate exposure images selected by different observers under a subjective evaluation method is given. Figure 3 middle, Figure 3 The image 1 shown in (a) is the appropriate exposure image selected by observer A (same as the normal exposure image); Figure 3 The image 2 shown in (b) is the appropriate exposure image selected by the observer B (same as the normal exposure image); Figure 3 The image 3 shown in (c) is the appropriate exposure image (same as the normal exposure image) selected by the observer C. Figure 3 You can feel it in Figure 3 (a) Figure 3 (b) and Figure 3 (c) is an image with different exposure states. That is to say, for the same image, the evaluation of "normal exposure" by observers A, B, and C is quite different. Exposure evaluation is subjective, and the exposure evaluation of the image is not accurate.

[0137] For the exposure quantization of an image, the final exposure quantization result of the image may be obtained by calculating the average of the exposure quantization values ​​of multiple observers.

[0138] For example, the calculation expression for the exposure quantization result MOS of an image is as follows:

[0139] Formula (1).

[0140] in, is the exposure quantization value of the i-th observer, where i is 1, 2, …k; is the ith observer.

[0141] In addition to the subjective evaluation method mentioned above, common image exposure quantification methods also include objective evaluation methods. Objective evaluation methods are divided into two types: with reference and without reference. Among them, with reference refers to the need for a reference reference image, that is, in the process of image exposure quantification, it is necessary to find a reference reference image to compare the image exposure state, so as to obtain the exposure quantification result of the image to be evaluated. Although this method can obtain the exposure quantification result of the image to be evaluated more objectively, the process of image exposure quantification is more troublesome due to the need to find a reference reference image. Without reference refers to objective exposure quantification that does not require a reference image as a reference image. Exemplarily, the image exposure quantification method without reference includes exposure prediction based on some preset statistical indicators, so as to obtain the exposure quantification result of the image to be evaluated. Among them, the statistical indicators may include business agreed indicators such as 18 degrees of gray. However, this type of method will have the problem of exposure prediction errors for some special scenes, so it is also necessary to repeatedly correct in the process of exposure quantification, resulting in inaccurate exposure quantification results and time-consuming and laborious. Among them, exemplarily, some special scenes include scenes containing large areas of light-colored content such as snow scenes, or scenes containing large areas of dark-colored content. For these special scenes, the image exposure quantization method without parameters may result in exposure prediction errors, requiring secondary exposure prediction correction, which has certain limitations.

[0142] For example, Figure 4 A before and after comparison of the corrected exposure of a snow scene is given. Figure 4 (a) shows an image that is considered to be appropriately exposed by the image exposure quantization method without parameters. Obviously, for this image with a large area of ​​snow, Figure 4 The brightness of the image shown in (a) is actually dark. Assume that, Figure 4 In the image 1 shown in (a), the exposure compensation is 0, the aperture value is F18, and the shutter speed is 1 / 250s. The exposure parameters of the shooting scene are corrected, and the exposure compensation is increased from 0 to 4 / 3, the aperture value remains unchanged at F18, and the shutter speed is reduced to 1 / 100s. Figure 4 (b) shows the image 2. Contrast Figure 4 (a) and Figure 4 (b) shows the difference between the exposure correction and the Figure 4 In the image 1 (a) shown in FIG. 1 , the snow scene is generally darker in brightness. After exposure correction, Figure 4 In the image 2 shown in (b), the snow scene has an appropriate brightness as a whole.

[0143] In addition, the parameter-free image exposure quantization method also includes other methods, such as using a quality assessment model to quantify the image exposure.

[0144] This exposure quantification method based on the quality assessment model, on the one hand, the quality assessment model usually requires manual labeling, and involves manual labeling. The labeling quantification methods and quantification values ​​are diverse, and it is difficult to obtain a unified measurement standard.

[0145] Table 1 below gives a comparison of subjective image evaluation metrics.

[0146] Table 1

[0147]

[0148] On the other hand, many quality assessment models are not designed for image exposure quantification. In the quality assessment process based on image quality assessment (IQA) datasets, IQA datasets usually involve different degradation types to simulate various quality problems that images may encounter in the real world. For example, common degradation types include blur, noise, compression distortions, and contrast distortions. If you want to use the IQA dataset to quantify image exposure, you need to fine-tune or migrate the quality assessment model and the IQA dataset, which makes the image exposure quantification process complicated.

[0149] Figure 5 A schematic diagram of the training pathway for a common quality assessment model is given.

[0150] The quality assessment data set (IQA data set) may include multiple sample images. The IQA data set is used as a training data set and input into the initial quality assessment model. The initial quality assessment model performs quality assessment based on one or more quantization strategies to obtain the initial predicted quality quantization value corresponding to the training data set. The Pearson linear correlation coefficient (PLCC) is calculated based on the initial predicted quality quantization value of the input data set. PLCC is used to measure the linear relationship between the initial predicted quality quantization value and the subjective quality quantization value. The quality assessment model is fine-tuned based on PLCC and the fine-tuning data set to obtain a fine-tuned quality assessment model. The fine-tuning data set includes multiple images in different exposure states. The fine-tuned quality assessment model can be used for exposure quantification. Thus, the training data set is input into the fine-tuned quality assessment model for inference to obtain the final exposure quantization result.

[0151] The training process of the entire quality assessment model requires obtaining images in different exposure states to fine-tune the quality assessment model. The acquisition of images in different exposure states requires a certain amount of time and procedures. The entire process is cumbersome, making the training process of the quality assessment model inefficient.

[0152] The embodiment of the present application provides an image exposure quantification method, which obtains the semantic similarity value between the input image and the preset prompt word through a trained image-text comparison model, wherein the preset prompt word includes fields describing different exposure states. The exposure state label of the input image can be determined based on the semantic similarity value. Among them, the image-text comparison model can learn the semantic relationship between the preset prompt word (text) and the image, thereby obtaining the semantic similarity value between the image and the preset prompt word. The exposure state label of the image obtained based on the semantic similarity value is objective and accurate. The subjective ambiguity of manual evaluation is avoided, and the entire process does not require the search for a reference benchmark image, nor does it require the formulation of complex multiple quantification strategies. While ensuring the accuracy of exposure quantification, it also reduces computing power.

[0153] Furthermore, the training process of the trained image-text comparison model used in the image exposure quantification method provided by the implementation of the present application can be modeled through sample images and training prompt words, so that the image-text comparison model has a more accurate understanding of the exposure of the image, and the similarity between the output image and the training prompt words is more accurate. Compared with the conventional image exposure quantification method based on the quality assessment model, the training process of the image-text comparison model provided by the implementation of the present application only requires adjusting the model parameters (such as model weights) and training prompt words, and does not require the formulation of complex quantization strategies. At the same time, it is also unnecessary to fine-tune the model based on a large number of images with different exposure states. While ensuring the accuracy of the output results of the image-text comparison model, the model training is simpler and the model training efficiency is higher.

[0154] The application scenario of the image exposure quantization method provided by the present application can be a pre-processing scenario of an image. For example, in an electronic device with a camera, the electronic device displays a preview image when starting the camera. The electronic device can perform exposure quantification based on the preview image, and determine whether exposure compensation is required for the preview image based on the exposure status label of the preview image. And, when the shooting scene does not change, when a shooting operation is received, the corresponding exposure compensation processing can also be performed on the shot image.

[0155] The application scenario of the image exposure quantization method provided in the embodiment of the present application can also be a post-processing scenario of an image. For example, in some electronic devices with image processing capabilities, the electronic devices can perform exposure quantization on a batch of images to be processed to obtain exposure status labels of the images to be processed. This embodiment does not limit the specific application scenario of the image exposure quantization method.

[0156] The image exposure quantization method provided in the embodiment of the present application can be applied to an electronic device with image processing capabilities. Optionally, the electronic device can have a shooting capability. Exemplarily, the electronic device can be a portable computer (such as a mobile phone), a tablet computer, a laptop computer, a personal computer (PC), a wearable electronic device (such as a smart watch), an augmented reality (AR) / virtual reality (VR) device, a car computer, a server, and other devices. The following embodiments do not impose any special restrictions on the specific form of the electronic device.

[0157] Figure 6 A schematic structural diagram of an electronic device 100 is shown.

[0158] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, a sensor module 180, a camera 193 and a display screen 194, etc.

[0159] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0160] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0161] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0162] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0163] In this embodiment, the processor 110 may serve as an executor of the image exposure quantization method, and be configured to perform exposure quantization on the input image and obtain an exposure quantization result of the input image.

[0164] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0165] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0166] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0167] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0168] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0169] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0170] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0171] In some embodiments, the electronic device may display, through the display screen 194 , a preview image after exposure compensation based on an exposure status label of the preview image, or a captured image after exposure compensation, and so on.

[0172] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0173] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0174] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0175] After the electronic device performs exposure quantification, when capturing a preview image or taking a picture through a camera, the electronic device can perform exposure compensation on the preview image or the taken image based on the exposure status tag, for example, reducing the door opening speed, or increasing the exposure compensation parameters, etc.

[0176] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0177] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0178] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0179] The embodiments of the present application do not limit the specific structure of the electronic device.

[0180] For the method of quantifying image exposure, in addition to the common image processing-related methods, the purpose of quantifying image exposure can also be achieved through the semantic understanding of images and texts, through the semantic understanding and semantic matching of image exposure and exposure prompt words. In the embodiment of the present application, a large language model (LLM) can be used to learn the characteristics of semantic features, and with the help of the text feature learning ability provided by the LLM, the image feature learning ability is further expanded on the text feature learning ability, and the image-text comparison model with this image-text feature learning ability is used to understand the image exposure state based on the exposure state prompt words, and obtain the result of image exposure quantification.

[0181] For example, Figure 7 A set of images with different exposure states are given. Among them, Figure 7 It includes 32 images of the same content under different exposure conditions. It can be seen intuitively that the exposure of the 32 images increases from image 1 to image 32. Prompt words are defined based on different exposure states. For example, prompt words can include underexposed, wellexposed, and overexposed. The prompt words and sample images are input into the image-text comparison model to calculate the similarity value between the image and the text.

[0182] Figure 8 A line diagram of the similarity values ​​between different prompt words and images is given. The horizontal axis of the line diagram is the image number, which includes 32 images from 1 to 32. The vertical axis of the line diagram is the similarity value.

[0183] from Figure 8 As can be seen from the figure, as the image number increases, the similarity between the image and the prompt word "overexposed" gradually increases. The similarity between the image and the prompt word "underexposed" gradually decreases. Figure 7 From the 32 images shown, we can also see that, from image 1 to image 32, as the number increases, the exposure of the image increases, and the image gradually changes from underexposure to overexposure. In other words, the image-text comparison model can learn the fact that as the image number increases, the image exposure also increases.

[0184] The image-text comparison model can capture information related to the exposure status of the image. The semantic features learned by the image-text comparison model can be used to infer quantitative results related to image quality from the image-text comparison results, including exposure quantitative results. Based on this, the image-text comparison model can be trained to obtain the similarity value between the image and the text describing the exposure status, and thus obtain the exposure status label of the image based on the similarity value.

[0185] In one embodiment, Fig. 9 A schematic diagram of the training path of a picture-text comparison model is given, which includes:

[0186] S101, an electronic device obtains a sample data set.

[0187] The sample data set may include multiple sample image groups. Each sample image group includes at least two images with different exposure states and the same shooting content. Fig. 9 As shown, the image numbered 1 and the image numbered 2 are a sample image group, the image numbered 3 and the image numbered 4 are a sample image group, the image numbered 5 and the image numbered 6 are a sample image group, and the image numbered 7 and the image numbered 8 are a sample image group.

[0188] In some embodiments, in order to make the model training more accurate, each sample image group may include at least three images with different exposure states and the same shooting content. That is, for the exposure states of underexposure, normal exposure, and overexposure, at least one image is required.

[0189] Each sample image in the sample data set may be an image generated by image processing, or may be an image captured under different exposures, which is not limited in this embodiment.

[0190] S102: The electronic device obtains a first training prompt word.

[0191] The first training prompt word may be a text used to describe the exposure state of the image. For example, the first training prompt word may be a word representing underexposure, normal exposure, and overexposure. For example, the first training prompt word may include a text including the keywords underexposed, wellexposed, and overexposed. The first training prompt word is a prompt word corresponding to the sample data set in this embodiment.

[0192] Exemplarily, the prompt words may be “this photo is underexposed”, “this foreground of photo is underexposed”, “this subject of photo is underexposed”, “this photo is wellexposed”, “this photo is overexposed”, and the like.

[0193] The electronic device can define prompt words based on actual scenarios, and the description range of all prompt words needs to include at least descriptions of the three states of underexposure, normal exposure, and overexposure.

[0194] In some other feasible embodiments, the representation of the exposure state of the image can be understood as the brightness of the image to a certain extent. The prompt word can also be other text representing the exposure state. For example, the prompt word can be "this image is too bright", "this image is too dark", "this image has appropriate brightness", etc.

[0195] The specific content and quantity of the prompt words are not limited in the embodiments of the present application.

[0196] S103, the electronic device inputs the sample data set and the first training prompt word into the image-text comparison model for inference, and obtains a similarity value between the sample image and the first training prompt word.

[0197] The image-text contrast model may be a contrastive language-image pre-training (CLIP) model, and the CLIP model may learn the semantic relationship between images and texts through natural language supervision. That is, the CLIP model may calculate the similarity value between each sample image and the first training prompt word, and the similarity value may reflect the exposure status of the sample image.

[0198] In this embodiment, the electronic device inputs the sample data set and the first training prompt word into the CLIP model to obtain the similarity value between the sample image in the sample data set and the first training prompt word. Wherein, in the case of including multiple first training prompt words, the CLIP model outputs the similarity value between the sample image and each first training prompt word.

[0199] In some feasible embodiments, the CLIP model can output the cosine similarity between the sample image and the preset prompt word. For example, if the first training prompt word includes underexposed, wellexposed, and overexposed, then the cosine similarity between the sample image and the preset prompt word is It can be expressed as:

[0200] Formula (2).

[0201] in, It can represent the cosine similarity (i.e., similarity value) between the sample image and the first training prompt word 1 (e.g., underexposed), It can represent the cosine similarity (i.e., similarity value) between the sample image and the first training prompt word 2 (e.g., wellexposed), It can represent the cosine similarity (ie, similarity value) between the sample image and the first training prompt word 3 (eg, overexposed).

[0202] The cosine similarity can be a specific value. For example, the similarity between sample image 1 and the first training prompt word 1 underexposed is 80%, the similarity between sample image 1 and the first training prompt word 2 wellexposed is 10%, and the similarity between sample image 1 and the first training prompt word 3 overexposed is 8%. In this way, the similarity value of the sample image corresponding to the first training prompt word is obtained.

[0203] S104: The electronic device determines a training label of the sample image based on the similarity value of the sample image.

[0204] In some embodiments, the electronic device may use the maximum cosine similarity as the exposure quantization value, and determine the training label of the sample image based on the first training prompt word corresponding to the maximum cosine similarity, wherein the training label refers to the exposure quantization label of the sample image output by the image-text comparison model.

[0205] In one possible approach, the maximum cosine similarity is used as the exposure quantization value It can be expressed as:

[0206] Formula (3).

[0207] For example, in the above S103, the maximum cosine similarity is 80%, and the cosine similarity of the first training prompt word 1 is used as the exposure quantization value of the sample image.

[0208] In a feasible manner, determining the training label of the sample image based on the first training prompt word corresponding to the maximum cosine similarity can be expressed as:

[0209] Formula (4).

[0210] For example, in the above S103, the maximum cosine similarity is 80%, and based on the first training prompt word 1 underexposed, the training label (ie, identification index) of the sample image is determined to be “underexposed”.

[0211] S105, the electronic device calculates the loss based on the training label of the sample image and the reference label corresponding to the sample image, and iteratively trains the image-text comparison model based on the loss value until the training conditions are met, thereby obtaining a trained image-text comparison model.

[0212] The reference label refers to a label that can represent the actual exposure state of the sample image. The reference label can be a manually annotated label or a label determined by an electronic device based on exposure parameters.

[0213] In this embodiment, the electronic device can calculate the loss between the training label of the sample image and the reference label corresponding to the sample image (for example, it can be a correlation loss), iteratively train the image-text comparison model based on the loss, adjust the model parameters and the training prompt words until the training conditions are met, thereby obtaining a trained image-text comparison model. The training condition may be that the loss gradually approaches 0, or that the number of iterations meets a threshold number. Model training can make the image-text comparison model more accurate in understanding the exposure of the image, and make the similarity between the output sample image and the training prompt words more accurate, so that the exposure status label of the image obtained based on the trained image-text comparison model is more accurate.

[0214] Compared with the conventional image exposure quantification method based on the quality assessment model, the image exposure quantification method provided in this embodiment does not need to formulate a complex quantization strategy, nor does it need to fine-tune the model based on the reference image. However, the trained image-text comparison model can still guarantee the accuracy of understanding the exposure status of the image. That is, the image exposure quantification method provided in this embodiment, while ensuring the accuracy of the output results of the image-text comparison model, only needs to adjust the model parameters (such as model weights) and training prompt words during the model training process, and does not need to fine-tune the entire model, so the model training is simpler and the model training efficiency is higher.

[0215] Correspondingly, the process of image exposure quantization of the input image based on the trained image-text contrast model can be referred to Fig.10 A flowchart of an image exposure quantification method is provided, including:

[0216] S201, the electronic device obtains an input image.

[0217] The input image may be an image to be quantized in different scenarios. For example, in a shooting scenario, the input image may be a preview image or a shot image. In an image processing scenario, the input image may be an image that needs to be quantized for exposure.

[0218] S202: The electronic device inputs the input image into the image-text comparison model to obtain a similarity value between the input image and a preset prompt word.

[0219] The electronic device stores a preset prompt word. The preset prompt word may be the same as the first training prompt word. The input image is input into the image-text comparison model, and the image-text comparison model may obtain the preset prompt word, calculate the similarity between the input image and the preset prompt word, and thereby obtain multiple similarity values ​​of the input image corresponding to the preset prompt word. The specific method of calculating the similarity value may refer to the embodiment provided in S103 of the model training embodiment, which will not be described in detail in this embodiment.

[0220] S203: The electronic device determines an exposure status label corresponding to the input image based on a similarity value between the input image and the preset prompt word.

[0221] The specific method of determining the exposure status label corresponding to the input image can refer to the embodiment provided in S104 in the model training embodiment, which will not be described in detail in this embodiment.

[0222] In this embodiment, the trained image-text comparison model can achieve semantic understanding of the correlation between the image and the preset prompt word. The method of image exposure quantification based on the image-text comparison model avoids the problems of inaccurate quantification and inconsistent standards caused by subjective ambiguity in manual quantitative evaluation, thereby improving the accuracy of image exposure quantification.

[0223] In some embodiments, the image to be quantized may be: Figure 4 As shown in the figure, the images have large areas of light-colored (such as white) content or large areas of dark-colored (such as black) content. For images of these special scenes, some image preprocessing can be performed on the input image / sample image, and then the image can be input into the image-text comparison model to calculate the similarity value, so as to obtain a more accurate exposure status label for the image.

[0224] In some feasible embodiments, in order to avoid interference of light-colored (such as white) or dark-colored (such as black) large-area background on image exposure quantization, subject detection may be performed on the subject of the image to obtain a local image corresponding to the subject. That is, the image preprocessing in this embodiment may be a process of detecting the subject in the global image and obtaining the subject image.

[0225] Thus, the electronic device can input the original image (i.e., the global image) and the prompt word into the image-text comparison model to obtain the similarity value between the global image and the prompt word. The main image (i.e., the local image) and the prompt word are input into the image-text comparison model to obtain the similarity value between the main image and the prompt word. The similarity value between the global image and the prompt word and the similarity value between the main image and the prompt word are combined to obtain the comprehensive similarity value of the image, thereby determining the exposure status label of the image based on the comprehensive similarity value.

[0226] In some embodiments, illustratively, Fig.11A schematic diagram of the training path of another image-text comparison model is given, which includes:

[0227] S301, the electronic device obtains a sample data set.

[0228] The sample data set may include multiple sample image groups. Each sample image group includes at least two images with different exposure states and the same shooting content. The sample images in the sample data set may also be called global images.

[0229] The specific method of obtaining the sample data set can be referred to the embodiment provided in S101, which will not be described in detail in this embodiment.

[0230] S302: The electronic device obtains a first training prompt word.

[0231] The first training prompt word may be a text used to describe the global image exposure state. The specific method of obtaining the first training prompt word may refer to the embodiment provided in the above S101, and will not be described in detail in this embodiment.

[0232] In this embodiment, exemplarily, the first training prompt word may be “this photo is underexposed”, “this foreground of photo is underexposed”, “this subject of photo is underexposed”, “this photo is wellexposed”, “this photo is overexposed”, and the like.

[0233] S303: The electronic device obtains a subject image corresponding to the sample image.

[0234] In this embodiment, the electronic device can perform saliency model inference on the sample image in the sample data set to obtain the subject image corresponding to the sample image. Among them, saliency model inference refers to analyzing and interpreting important features or areas (such as subject areas) in the input data (such as sample images) through a saliency model. The saliency model inference in this embodiment can refer to the saliency model inference in the traditional way, which will not be repeated here.

[0235] In this embodiment, relative to the entire sample image, the main image may also be referred to as a partial image.

[0236] like Fig.11As shown, after the saliency model inference, the main image numbered 1' corresponding to the sample image numbered 1, the main image numbered 2' corresponding to the sample image numbered 2, the main image numbered 3' corresponding to the sample image numbered 3, the main image numbered 4' corresponding to the sample image numbered 4, the main image numbered 5' corresponding to the sample image numbered 5, the main image numbered 6' corresponding to the sample image numbered 6, the main image numbered 7' corresponding to the sample image numbered 7, and the main image numbered 8' corresponding to the sample image numbered 8 can be obtained.

[0237] S304: The electronic device obtains a second training prompt word.

[0238] The second training prompt word may be a text used to describe the exposure state of the partial image. The specific method of obtaining the second training prompt word may also refer to the embodiment provided in the above S101, which will not be described in detail in this embodiment.

[0239] In this embodiment, the second training prompt word may be the same as the first training prompt word, or may be different, wherein the same includes the same semantics, or the same semantics and description.

[0240] In this embodiment, exemplarily, the second prompt word may be “this photo is underexposed”, “this subject of photo is underexposed”, “this image is wellexposed”, “this image is overexposed”, and the like.

[0241] S305: The electronic device inputs the sample data set and the first training prompt word into the image-text comparison model for inference, and obtains a first similarity value between the sample image and the first training prompt word.

[0242] In this embodiment, the electronic device inputs the sample data set and the first training prompt word into the CLIP model to obtain the similarity value between the sample image in the sample data set and the first training prompt word. Wherein, in the case of including multiple first training prompt words, the CLIP model outputs the similarity value between the sample image and each first training prompt word.

[0243] The specific method for calculating the first similarity value may refer to the embodiment provided in S103 above. For example, in this embodiment, the first similarity value obtained is .

[0244] S306: The electronic device inputs the subject image and the second training prompt word into the image-text comparison model for inference, and obtains a second similarity value between the subject image and the second training prompt word.

[0245] In this embodiment, the electronic device inputs the subject image and the second training prompt word into the CLIP model to obtain the similarity value between the subject image and the second training prompt word. Wherein, in the case of including multiple second training prompt words, the CLIP model outputs the similarity value between the sample image and each second training prompt word.

[0246] The specific method for calculating the second similarity value may refer to the embodiment provided in S103 above. For example, in this embodiment, the second similarity value obtained is .

[0247] S307: The electronic device obtains a third similarity value corresponding to the sample image according to the first similarity value and the second similarity value.

[0248] In this embodiment, the electronic device may define a first weight corresponding to the first similarity value and a second weight corresponding to the second similarity value.

[0249] The electronic device performs weighted calculation based on the first similarity value, the first weight, the second similarity value and the second weight to obtain a third similarity value of the sample image.

[0250] In this embodiment, the first similarity used for weighted calculation refers to the maximum value among the first similarities, and the second similarity used for weighted calculation refers to the maximum value among the second similarities.

[0251] The method for obtaining the third similarity value y can be expressed as:

[0252] Formula (5).

[0253] in, is the first weight, is the maximum first similarity value corresponding to the sample image; is the second weight, is the second largest similarity value corresponding to the local image.

[0254] S308: The electronic device determines a training label of the sample image based on the third similarity value of the sample image.

[0255] In general, the similarity of the sample image to different first training prompt words has the same trend as the similarity of the local image to different second training prompt words. That is, the target first training prompt word corresponding to the maximum first similarity value of the sample image and the target second training prompt word corresponding to the maximum second similarity value of the local image have consistency in representing the exposure state.

[0256] Based on this, the third similarity value after weighted calculation can represent the modified similarity value of the sample image to the target first training prompt word and the target second training prompt word.

[0257] Based on the third similarity value, the corresponding target first training prompt word, and the target second training prompt word, a training label of the sample image may be determined.

[0258] S309, the electronic device calculates the loss based on the training label of the sample image and the reference label corresponding to the sample image, and iteratively trains the image-text comparison model based on the loss value until the training conditions are met, thereby obtaining a trained image-text comparison model.

[0259] In this embodiment, the training process of the image-text comparison model can refer to the embodiment provided in the above S105, and this embodiment will not elaborate on it.

[0260] Compared with the conventional image exposure quantification method based on the quality assessment model, the image exposure quantification method provided in this embodiment does not need to formulate a complex quantification strategy, nor does it need to fine-tune the model based on the reference image. However, the trained image-text comparison model can still ensure the accuracy of understanding the exposure status of the image. That is, the image exposure quantification method provided in this embodiment only needs to adjust the model parameters (such as model weights) and training prompt words, and does not need to obtain image data of different exposure states for fine-tuning the model. While ensuring the accuracy of the output results of the image-text comparison model, the model training process is simpler and the model training efficiency is higher. Moreover, in this embodiment, the correlation learning between the local image and the training prompt words is introduced, so that the image-text comparison model can obtain a more accurate exposure quantification result for the entire image by quantifying the exposure of the subject for some special scenes, such as images with a large area of ​​light background or a large area of ​​dark background, so that the image exposure quantification is more accurate.

[0261] Correspondingly, the process of image exposure quantization of the input image based on the trained image-text contrast model can be referred to Fig.12 A flowchart of another image exposure quantization method is provided, comprising:

[0262] S401, the electronic device obtains an input image.

[0263] For details, please refer to the method provided in the above embodiment S201, which will not be described in detail in this embodiment.

[0264] S402: The electronic device performs saliency model inference on the input image to obtain a subject image corresponding to the input image.

[0265] For details, please refer to the method provided in S303 of the above embodiment, which will not be described in detail in this embodiment.

[0266] S403: The electronic device inputs the input image into the image-text comparison model to obtain a first similarity value of the input image corresponding to a preset prompt word.

[0267] The electronic device stores preset prompt words. The preset prompt words may include a first training prompt word and a second training prompt word. The input image is input into the image-text comparison model, and the image-text comparison model can obtain the preset prompt word, calculate the similarity between the input image and the preset prompt word, and thus obtain a plurality of first similarity values ​​of the input image corresponding to the preset prompt word. The specific method of calculating the first similarity value can refer to the embodiment provided in S305 of the model training embodiment, which will not be described in detail in this embodiment.

[0268] S404: The electronic device inputs the main image into the image-text comparison model to obtain a second similarity value of the main image corresponding to the preset prompt word.

[0269] The input image is input into the image-text comparison model, and the image-text comparison model can obtain a preset prompt word, calculate the similarity between the input image and the preset prompt word, and thus obtain a plurality of second similarity values ​​of the input image corresponding to the preset prompt word. The specific method of calculating the second similarity value can refer to the embodiment provided in S306 of the model training embodiment, and will not be described in detail in this embodiment.

[0270] S405: The electronic device calculates a third similarity value corresponding to the input image based on the first similarity value and the second similarity value.

[0271] Among them, the specific method of calculating the third similarity value can refer to the embodiment provided in S307 of the model training embodiment, and will not be described in detail in this embodiment.

[0272] S406: The electronic device determines an exposure status label corresponding to the input image based on the third similarity value.

[0273] Among them, the specific method of determining the exposure status label corresponding to the input image based on the third similarity value can refer to the embodiment provided in S308 of the model training embodiment, which is not described in detail in this embodiment.

[0274] In this embodiment, the trained image-text comparison model can realize the semantic understanding of the correlation between the global image of the input image and the preset prompt words, as well as the semantic understanding of the correlation between the subject image corresponding to the input image and the preset prompt words. For input images containing some special scenes, the semantic understanding of the correlation between the subject image and the preset prompt words can better determine the actual exposure state of the input image, thereby making the final exposure quantification label of the input image more accurate.

[0275] above Fig.10 as well as Fig.12 The image exposure quantification method applied by the given model can be understood as a scenario in which one image corresponds to multiple prompt words, and the exposure state of the image is determined by the similarity between the image and the multiple prompt words.

[0276] In some other feasible embodiments, the trained image-text comparison model can provide the ability to understand the semantics between the image and the preset prompt word. Based on this ability, a target image that matches the target prompt word can also be determined from a group of images. That is, the image exposure quantification method provided in the embodiment of the present application can also be used to query the target image that matches the target prompt word. For example, a group of images includes multiple images in different exposure states, and from the multiple images in different exposure states, a target image with "proper exposure" is queried.

[0277] In some embodiments, Fig.13 A flowchart of another image exposure quantization method is given, including:

[0278] S501, the electronic device obtains multiple input images.

[0279] The multiple input images include images in different exposure states, for example, at least two of an underexposed image, a normally exposed image, and an overexposed image. Fig.13 The images numbered 1 to 5 at different exposure states are shown.

[0280] S502: The electronic device performs saliency model inference on each input image to obtain a subject image corresponding to each input image.

[0281] For details, please refer to the method provided in S303 of the above embodiment, which will not be described in detail in this embodiment.

[0282] In this embodiment, main body images numbered 1' to 5' corresponding to images numbered 1 to 5 can be obtained.

[0283] S503: The electronic device inputs each input image into an image-text comparison model to obtain a fourth similarity value of each input image corresponding to the target prompt word.

[0284] The target prompt word may include a prompt word of “normal exposure”.

[0285] The present embodiment aims to select a target image that matches the prompt word from a plurality of input images based on the similarity value between the input image and the prompt word.

[0286] Here, each input image is input into the image-text comparison model, and the image-text comparison model can calculate the similarity between each input image and the target prompt word based on the target prompt word, thereby obtaining a fourth similarity value of each input image corresponding to the target prompt word.

[0287] S504: The electronic device inputs each subject image into an image-text comparison model to obtain a fifth similarity value of each subject image corresponding to the target prompt word.

[0288] Wherein, each subject image is input into the image-text comparison model, and the image-text comparison model can calculate the similarity between each subject image and the target prompt word based on the target prompt word, so as to obtain the fourth similarity value of each subject image corresponding to the target prompt word.

[0289] S505: The electronic device calculates a sixth similarity value corresponding to the input image based on the fourth similarity value of the input image and the fifth similarity value of the subject image corresponding to the input image.

[0290] Among them, the electronic device may perform weighted calculation on the fourth similarity value and the fifth similarity value to obtain a sixth similarity value. Alternatively, the electronic device may use the average of the fourth similarity value and the fifth similarity value as the sixth similarity value. Alternatively, the electronic device may use the larger similarity value between the fourth similarity value and the fifth similarity value as the sixth similarity value. Alternatively, the electronic device may also use the smaller similarity value between the fourth similarity value and the fifth similarity value as the sixth similarity value.

[0291] By using any method for calculating the sixth similarity value, the electronic device can obtain the sixth similarity value of each input image to the target prompt word determined by combining the similarity value of the global image and the similarity value of the main image.

[0292] For example, when the target prompt word is a prompt word containing "normal exposure", the fourth similarity value of the input image numbered 1 is 0.2655, and the fifth similarity value of the subject object numbered 1' corresponding thereto is 0.2457. For example, by weighted summation, the weight of the input image can be set to 0.6, and the weight corresponding to the subject image can be set to 0.4, then the sixth similarity value of the input image is 0.2655*0.6+0.2457*0.4=0.2575. In other words, the sixth similarity value between the input image numbered 1 and the target prompt word is 0.2575.

[0293] For example, the fourth similarity value of the input image numbered 4 is 0.8457, and the fifth similarity value of the subject object numbered 4' corresponding thereto is 0.8874. For example, by weighted summation, the weight of the input image can be set to 0.6, and the corresponding weight of the subject image can be set to 0.4, then the sixth similarity value of the input image is 0.8457*0.6+0.8874*0.4=0.8623. In other words, the sixth similarity value between the input image numbered 4 and the target prompt word is 0.8623.

[0294] For example, the fourth similarity value of the input image numbered 5 is 0.5412, and the fifth similarity value of the subject object numbered 5' corresponding thereto is 0.4474. For example, by weighted summation, the weight of the input image can be set to 0.6, and the corresponding weight of the subject image can be set to 0.4, then the sixth similarity value of the input image is 0.5412*0.6+0.4474*0.4=0.5036. In other words, the sixth similarity value between the input image numbered 5 and the target prompt word is 0.5036.

[0295] In this way, the electronic device can obtain the sixth similarity value between each input image and the target prompt word.

[0296] S506: The electronic device determines a target image based on a preset threshold strategy and a sixth similarity value of each input image.

[0297] Among them, in some actual demand scenarios, the target image refers to a normally exposed image.

[0298] In the scenario where the target prompt word is a prompt word including "normal exposure", the target image is the image with the highest similarity to the prompt word. The electronic device can obtain the image with the largest sixth similarity value as the target image. For example, in S505, the sixth similarity value of the input image numbered 4 is the maximum value, so the input image numbered 4 is the target image.

[0299] In a scenario where the target prompt word is a prompt word including “underexposure” or the target prompt word is a prompt word including “overexposure”, it is necessary to further obtain a target image that meets the normal exposure according to a threshold strategy.

[0300] In this embodiment, reference may be made to Figure 8A schematic diagram of the similarity values ​​between different prompt words and images is given. The intersection of the underexposure similarity value line and the overexposure similarity value line can be understood as the similarity value point corresponding to the normal exposure image. In other words, when the prompt word is a prompt word containing "underexposure", if the sixth similarity value of the image is within a numerical range corresponding to the intersection, the image can be used as the target image; when the prompt word is a prompt word containing "overexposure", if the sixth similarity value of the image is within a numerical range corresponding to the intersection, the image can be used as the target image.

[0301] Specifically, in some embodiments, when the target prompt word is a prompt word representing normal exposure, the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the first threshold is a normal exposure image. The exposure quantization label of the input image whose sixth similarity value is less than the first threshold is an abnormal exposure image.

[0302] When the target prompt word is a prompt word representing underexposure, the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the second threshold is an underexposed image. The sixth similarity value is less than the second threshold, and the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the third threshold is a normally exposed image, and the exposure quantization label of the input image whose sixth similarity value is less than the third threshold is an overexposed image.

[0303] When the target prompt word is a prompt word representing overexposure, the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the fourth threshold is an overexposed image. The sixth similarity value is less than the fourth threshold, and the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the fifth threshold is a normally exposed image, and the exposure quantization label of the input image whose sixth similarity value is less than the fifth threshold is an underexposed image.

[0304] It is understandable that the above application scenario can also be applied to the image exposure quantization method of the image contrast model without subject detection, that is, based on the fourth similarity value of the input image to the target prompt word and the threshold strategy corresponding to the target prompt word, the exposure quantization of the input image is performed, so as to determine the target image matching the target prompt word. The specific method can refer to the method for determining the target image matching the target prompt word based on the threshold strategy corresponding to the sixth similarity value and the target prompt word in the above embodiment, which will not be described in detail here.

[0305] Specifically, for example, when the target prompt word is a prompt word representing normal exposure, the first threshold may be 0.7.

[0306] The exposure quantization label of the input image whose sixth similarity value is greater than or equal to the first threshold is a normal exposure image (a target image matching the target prompt word). The exposure quantization label of the input image whose sixth similarity value is less than the first threshold is an abnormal exposure image.

[0307] like Fig.13 As shown, if the sixth similarity value of the input image No. 4 is 0.885, the similarity value of the input image is greater than the first threshold value 0.7, and the similarity values ​​of the other input images are all less than the first threshold value 0.7, then the exposure quantization label of the input image No. 4 is a normal exposure image, and the input image No. 4 is the target image that matches the target prompt word. The exposure quantization labels of the other input images are abnormal exposure images.

[0308] Specifically, for example, when the target prompt word is a prompt word representing underexposure, the second threshold and the third threshold can be based on Figure 8 The trend of the line chart shown is determined. Figure 8 As shown, the intersection of the underexposure line and the overexposure inflection point means the closest to normal exposure. The second threshold and the third threshold are taken based on the similarity value corresponding to the intersection. For example, the second threshold may be 0.55 and the third threshold may be 0.45.

[0309] In one case, the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the second threshold is an underexposed image, that is, a target image that matches the target prompt word.

[0310] like Fig.13 As shown, the sixth similarity value of the input image No. 1 is 0.987, the sixth similarity value of the input image No. 2 is 0.854, and the sixth similarity value of the input image No. 3 is 0.712. The sixth similarity values ​​of the input images No. 1-No. 3 are all greater than the second threshold value of 0.55. The exposure quantization labels of the input images No. 1-No. 3 are underexposed images, that is, target images that match the target prompt word.

[0311] In one case, the sixth similarity value is less than the second threshold value 0.55, and the exposure quantization label of the input image that is greater than or equal to the third threshold value 0.45 is a normal exposure image.

[0312] like Fig.13 As shown, the sixth similarity value of the input image No. 4 is 0.501, which is greater than the third threshold and less than the second threshold. Therefore, the exposure quantization label of the input image No. 4 is a normal exposure image.

[0313] In one case, the exposure quantization label of the input image whose sixth similarity value is less than the third threshold is an overexposed image.

[0314] like Fig.13 As shown, the sixth similarity value of the input image No. 5 is 0.337. The sixth similarity value of the input image No. 3 is less than the third threshold value 0.45. The exposure quantization label of the input image No. 3 is an overexposed image.

[0315] The second threshold and the third threshold may also form a threshold range, and the electronic device may also determine the image quantization label based on the threshold range. This embodiment does not limit the specific means used.

[0316] Specifically, for example, when the target prompt word is a prompt word representing overexposure, the fourth threshold and the fifth threshold can be based on Figure 8 The trend of the line chart shown is determined. Figure 8 As shown, the intersection of the underexposure line and the overexposure inflection point means the closest to normal exposure, and the fourth threshold and the fifth threshold are taken based on the similarity value corresponding to the intersection. For example, the fourth threshold may be 0.58, and the fifth threshold may be 0.43. In some embodiments, the fourth threshold may be the same as the second threshold, and the fifth threshold may also be the same as the third threshold.

[0317] In one case, the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the fourth threshold is an overexposed image.

[0318] like Fig.13 As shown, the sixth similarity value of the input image numbered 5 is 0.885, which is greater than the fourth threshold value of 0.58. Therefore, the exposure quantization label of the input image numbered 5 is an overexposed image, that is, a target image that matches the target prompt word.

[0319] In one case, the sixth similarity value is less than the fourth threshold value 0.58, and the exposure quantization label of the input image that is greater than or equal to the fifth threshold value 0.43 is a normal exposure image.

[0320] like Fig.13 As shown, the sixth similarity value of the input image No. 4 is 0.557, which is less than the fourth threshold value and greater than the fifth threshold value. Therefore, the exposure quantization label of the input image No. 4 is a normal exposure image.

[0321] In one case, the exposure quantization label of the input image whose sixth similarity value is less than the fifth threshold is an underexposed image.

[0322] like Fig.13 As shown, the sixth similarity value of the input image No. 1 is 0.112, the sixth similarity value of the input image No. 2 is 0.213, and the sixth similarity value of the input image No. 3 is 0.323. The sixth similarity values ​​of the input images No. 1-No. 3 are all less than the fifth threshold value of 0.43. The exposure quantization labels of the input images No. 1-No. 3 are underexposed images.

[0323] The second threshold and the third threshold may also form a threshold range, and the electronic device may also determine the image quantization label based on the threshold range. This embodiment does not limit the specific means used.

[0324] In this way, based on the difference of the target prompt words and the difference of the threshold strategies corresponding to the target prompt words, the target image can be determined based on the sixth similarity values ​​of each input image.

[0325] In this embodiment, the semantic understanding of the correlation between the image and the target prompt word of the trained image-text comparison model can also be used to realize the query of the target image based on the prompt word. For example, when the target image is a normal exposure image, the threshold strategy corresponding to the target prompt word for normal exposure in the method provided in the above embodiment can be used to determine the image with a similarity value greater than the normal exposure from multiple input images based on the similarity value between each input image and the target prompt word as the target image. The exposure quantification of the input image is obtained through the image-text comparison model, and the quantification result is objective and accurate, which can meet the image exposure quantification requirements in various scenarios.

[0326] In order to better illustrate the actual effect that can be achieved by using the image exposure quantization method provided in the embodiment of the present application. Fig.14 Several manually screened illustrations showing underexposure are given. Fig.14 The several examples given are all images of special scenes, for example, there is a large area of ​​light background (shopping mall) in the image; or there is a large area of ​​black background in the image.

[0327] In the exposure evaluation results of manual exposure evaluation, Fig.14 The images numbered 1 to 9 are all underexposed images.

[0328] Use images numbered 1 to 9 Fig.10 The provided image exposure quantization method performs exposure quantization judgment and determines the exposure status label of each image. The obtained results include that the image numbered 7 is an overexposed image, and the remaining images are normally exposed images.

[0329] Use images numbered 1 to 9 Fig.12 The provided image exposure quantization method performs exposure quantization judgment to obtain the exposure status label of each image. The result is that the image numbered 7 is an overexposed image; the remaining images are all underexposed images.

[0330] Fig.14 Among the 9 images provided, the actual exposure status is that image No. 7 is an overexposed image, and the remaining images are underexposed images.

[0331] Obviously, the exposure evaluation results obtained by manual exposure evaluation cannot accurately determine the overexposed images. Fig.10 The image exposure quantification method provided in the embodiment can at least identify the overexposed image No. 7. Fig.12The image exposure quantification method provided in the embodiment combines the exposure status of the whole and the subject to Fig.14 The images with light background or dark background are accurately quantified, and the exposure quantification results are obtained that the image numbered 7 is overexposed and the remaining images are underexposed images. Fig.12 The image exposure quantization method provided in the embodiment can effectively improve the accuracy of exposure quantization in special scenes (such as when there is a large area of ​​light background in the image (shopping mall); or when there is a large area of ​​black background in the image).

[0332] Fig.15 A possible structural diagram of the electronic device involved in the above embodiments is shown. Fig.15 The electronic device 1500 shown includes a processing module 1501 and a storage module 1502. In some other feasible embodiments, the electronic device 1500 may further include an image acquisition module 1503 and a display module 1504.

[0333] The processing module 1501 may be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The processor may include an application processor and a baseband processor. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0334] For example, the processing module 1501 may be as follows Figure 6 The processor 110 shown in FIG. 110; the image acquisition module 1503 may be as follows Figure 6 The camera 193 shown, the display module 1504 can be as follows Figure 6 The display screen 194 shown; the storage module 1502 can be as follows Figure 6 The internal memory 121 shown. The electronic device 1500 provided in the embodiment of the present application can be Figure 6 An electronic device 100 is shown.

[0335] The present application also provides a chip system (eg, a system on a chip (SoC)), such as Fig.16As shown, the chip system includes at least one processor 1601 and at least one interface circuit 1602. The processor 1601 and the interface circuit 1602 can be interconnected via lines. For example, the interface circuit 1602 can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit 1602 can be used to send signals to other devices (such as a processor 1601 or a camera of an electronic device). Exemplarily, the interface circuit 1602 can read instructions stored in the memory and send the instructions to the processor 1601. When the instructions are executed by the processor 1601, the electronic device can execute the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0336] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes each function or step executed by the electronic device 100 in the above-mentioned method embodiment.

[0337] The embodiment of the present application further provides a computer program product, which, when executed on a computer, enables the computer to execute the functions or steps executed by the electronic device 100 in the above method embodiment. For example, the computer may be the above electronic device 100.

[0338] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0339] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0340] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0341] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0342] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0343] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for quantifying image exposure, characterized in that: include: Acquire an input image; the input image includes one or more images to be subjected to exposure quantization; Inputting the input image into the image-text comparison model to obtain a semantic similarity value between the input image and a preset prompt word; The preset prompt word includes one or more description fields representing the exposure status; Based on the semantic similarity value of the input image corresponding to the preset prompt word, an exposure quantization label of the input image is determined; the exposure quantization label is used to characterize whether the input image is an underexposed image, a normally exposed image, or an overexposed image.

2. The method according to claim 1, characterized in that The input image includes an image to be subjected to exposure quantization, and the preset prompt words include a plurality of prompt words describing different exposure states; The step of inputting the input image into the image-text comparison model to obtain a semantic similarity value between the input image and a preset prompt word includes: Inputting the input image into the image-text comparison model, obtaining a first similarity value of the input image corresponding to each of the preset prompt words, and obtaining a plurality of the first similarity values; The step of determining the exposure quantization label of the input image based on the semantic similarity value includes: An exposure quantization label of the input image is determined based on the exposure state corresponding to the preset prompt word with the largest first similarity value.

3. The method according to claim 1, characterized in that The input image includes an image to be subjected to exposure quantization, and the preset prompt words include a plurality of prompt words describing different exposure states; The method further comprises: Performing saliency model inference on the input image to obtain a subject image corresponding to the input image; Inputting the subject image into the image-text comparison model to obtain a second similarity value of the subject image corresponding to each of the preset prompt words; The step of inputting the input image into the image-text comparison model to obtain a semantic similarity value between the input image and a preset prompt word includes: Inputting the input image into the image-text comparison model, obtaining a first similarity value of the input image corresponding to each of the preset prompt words, and obtaining a plurality of the first similarity values; The step of determining the exposure quantization label of the input image based on the semantic similarity value includes: Determining a third similarity value of the input image based on the first similarity value and the second similarity value; Based on the third similarity value, an exposure quantization label of the input image is determined.

4. The method according to claim 3, characterized in that The first similarity value includes semantic similarity values ​​of the input image corresponding to the plurality of preset prompt words, and the second similarity value includes semantic similarity values ​​of the subject image corresponding to the plurality of preset prompt words. The step of determining a third similarity value of the input image based on the first similarity value and the second similarity value comprises: For each of the preset prompt words, performing a weighted sum calculation on the first similarity value and the second similarity value to obtain a third similarity value corresponding to each of the preset prompt words, and obtaining a plurality of the third similarity values; The step of determining the exposure quantization label of the input image based on the third similarity value includes: An exposure quantization label of the input image is determined based on the exposure state corresponding to the preset prompt word with the largest third similarity value.

5. The method according to claim 3, characterized in that: The first similarity value includes semantic similarity values ​​of the input image corresponding to the plurality of preset prompt words, and the second similarity value includes semantic similarity values ​​of the subject image corresponding to the plurality of preset prompt words. The step of determining a third similarity value of the input image based on the first similarity value and the second similarity value comprises: Obtaining a first similarity value with a maximum value and a second similarity value with a maximum value; When the preset prompt word corresponding to the largest first similarity value is consistent with the preset prompt word corresponding to the largest second similarity value, weighted summing the largest first similarity value and the largest second similarity value is performed to obtain the third similarity value; The step of determining the exposure quantization label of the input image based on the third similarity value includes: An exposure quantization label of the input image is determined based on the exposure state corresponding to the preset prompt word corresponding to the third similarity value.

6. The method according to claim 5, characterized in that The method further comprises: If the preset prompt word corresponding to the largest first similarity value is inconsistent with the preset prompt word corresponding to the largest second similarity value, the largest second similarity value is used as the third similarity value.

7. The method according to any one of claims 1 to 6, characterized in that The preset prompt words include a first prompt word, a second prompt word and a third prompt word, the first prompt word is a prompt word representing underexposure, the second prompt word is a prompt word representing normal exposure, and the third prompt word is a prompt word representing overexposure.

8. The method according to claim 1, characterized in that: The input image includes a plurality of images to be subjected to exposure quantization, and the preset prompt word includes a target prompt word of a target exposure state; The step of inputting the input image into the image-text comparison model to obtain a semantic similarity value between the input image and a preset prompt word includes: Inputting each of the input images into the image-text comparison model, and obtaining a fourth similarity value of each of the input images corresponding to the target prompt word; The step of determining the exposure quantization label of the input image based on the semantic similarity value includes: Based on the fourth similarity value of each of the input images and a preset threshold strategy, an exposure quantization label of each of the input images is determined.

9. The method according to claim 8, characterized in that The preset threshold strategy is related to the target prompt word, The preset threshold strategy includes: When the target prompt word is a prompt word representing normal exposure, the exposure quantization label of the input image whose fourth similarity value is greater than or equal to the first threshold is a normal exposure image; the exposure quantization label of the input image whose fourth similarity value is less than the first threshold is an abnormal exposure image; When the target prompt word is a prompt word representing underexposure, the exposure quantization label of the input image whose fourth similarity value is greater than or equal to the second threshold is an underexposed image, the fourth similarity value is less than the second threshold, and the exposure quantization label of the input image whose fourth similarity value is greater than or equal to the third threshold is a normally exposed image, and the exposure quantization label of the input image whose fourth similarity value is less than the third threshold is an overexposed image; When the target prompt word is a prompt word representing overexposure, the exposure quantization label of the input image whose fourth similarity value is greater than or equal to a fourth threshold is an overexposed image, the fourth similarity value is less than the fourth threshold, and the exposure quantization label of the input image whose fourth similarity value is greater than or equal to a fifth threshold is a normally exposed image, and the exposure quantization label of the input image whose fourth similarity value is less than the fifth threshold is an underexposed image.

10. The method according to claim 1, characterized in that The input image includes a plurality of images to be subjected to exposure quantization, and the preset prompt word includes a target prompt word of a target exposure state; The method further comprises: Performing saliency model inference on each of the input images to obtain a subject image corresponding to each of the input images; Inputting each of the subject images into the image-text comparison model to obtain a fifth similarity value of each of the subject images corresponding to the target prompt word; The step of inputting the input image into the image-text comparison model to obtain a semantic similarity value between the input image and a preset prompt word includes: Inputting each of the input images into the image-text comparison model, and obtaining a fourth similarity value of each of the input images corresponding to the target prompt word; The step of determining the exposure quantization label of the input image based on the semantic similarity value includes: determining a sixth similarity value of the input images based on a fourth similarity value corresponding to each of the input images and a fifth similarity value of the subject image corresponding to the input image; Based on the sixth similarity value of each of the input images and a preset exposure threshold, an exposure quantization label of each of the input images is determined.

11. The method according to claim 10, characterized in that The determining, based on the fourth similarity value corresponding to each of the input images and the fifth similarity value of the subject image corresponding to the input image, a sixth similarity value of the input image comprises: taking an average value of the fourth similarity value corresponding to each of the input images and the fifth similarity value of the subject image corresponding to the input image as a sixth similarity value of the input image; or, A value of a weighted sum of the fourth similarity value corresponding to each of the input images and the fifth similarity value of the subject image corresponding to the input image is used as a sixth similarity value of the input image.

12. The method according to claim 10 or 11, characterized in that: The preset threshold strategy is related to the target prompt word, The preset threshold strategy includes: When the target prompt word is a prompt word representing normal exposure, the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the first threshold is a normal exposure image; the exposure quantization label of the input image whose sixth similarity value is less than the first threshold is an abnormal exposure image; When the target prompt word is a prompt word representing underexposure, the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the second threshold is an underexposed image, the sixth similarity value is less than the second threshold, and the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the third threshold is a normally exposed image, and the exposure quantization label of the input image whose sixth similarity value is less than the third threshold is an overexposed image; When the target prompt word is a prompt word representing overexposure, the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the fourth threshold is an overexposed image, the sixth similarity value is less than the fourth threshold, and the exposure quantization label of the input image whose sixth similarity value is greater than or equal to the fifth threshold is a normally exposed image, and the exposure quantization label of the input image whose sixth similarity value is less than the fifth threshold is an underexposed image.

13. A method for training a graphic model for image exposure quantification, characterized in that: The method further comprises: Acquire multiple sample images; Inputting the plurality of sample images into an initial image-text comparison model, obtaining a first training similarity value between each of the sample images and a plurality of first training prompt words; the plurality of first training prompt words include prompt words representing different exposure states; determining a training label for the sample image based on an exposure state corresponding to a first training prompt word having the largest first training similarity value; Calculate the loss based on the training label of the sample image and the reference label of the sample image; the reference label is the actual exposure state of the sample image, and the reference label is used to characterize whether the sample image is an underexposed image, a normally exposed image, or an overexposed image; Based on the loss, the initial image-text comparison model is iteratively trained until the initial image-text comparison model meets the training conditions, thereby obtaining the image-text comparison model.

14. The method according to claim 13, characterized in that The method further comprises: Performing saliency model inference on the multiple sample images to obtain a sample subject image corresponding to each of the sample images; Inputting the sample subject image into an initial image-text comparison model, obtaining a second training similarity value between each sample subject image and a plurality of second training prompt words; the plurality of second training prompt words include prompt words representing different exposure states; The determining of the training label of the sample image based on the exposure state corresponding to the first training prompt word with the largest first training similarity value includes: For each sample image, obtaining a first training similarity value with a maximum value and a second training similarity value with a maximum value of a sample subject image corresponding to the sample image; In a case where a first training prompt word corresponding to a first training similarity value having a maximum value of the sample image and a second training prompt word corresponding to a second training similarity value having a maximum value represent the same exposure state, determining a third training similarity value based on the maximum first training similarity value and the maximum second training similarity value; A training label of the sample image is determined based on the first training prompt word or the second training prompt word corresponding to the third training similarity value.

15. The method according to claim 14, characterized in that The step of determining the training label of the sample image based on the exposure state corresponding to the first training prompt word having the largest first training similarity value further includes: If the preset prompt word corresponding to the largest first similarity value is inconsistent with the preset prompt word corresponding to the largest second similarity value, the largest second similarity value is used as the third training similarity value; A training label of the sample image is determined based on the first training prompt word or the second training prompt word corresponding to the third training similarity value.

16. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 12, and / or to implement the steps of the method according to any one of claims 13 to 15.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the steps of the method described in any one of claims 1 to 12, and / or implements the steps of the method described in any one of claims 13 to 15.

18. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the steps of the method described in any one of claims 1 to 12, and / or implements the steps of the method described in any one of claims 13 to 15.

Images