Image processing method and apparatus therefor

Abstract

The application discloses an image processing method and device, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a first image, wherein the first image comprises at least one shooting object; identifying each shooting object in the at least one shooting object; determining the light supplement intensity information of each shooting object according to the area image corresponding to each shooting object, wherein the light supplement intensity information of each shooting object is the intensity information of light supplement to the shooting object; and supplementing light to the corresponding shooting object in the at least one shooting object respectively according to the light supplement intensity information, so as to obtain a second image.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and specifically relates to an image processing method and apparatus. Background Technology

[0002] With the increasing popularity of electronic devices, users have higher and higher demands for the imaging technology of electronic devices, especially portrait mode. Users expect to take portrait photos with more atmosphere and closer to the effect of SLR cameras.

[0003] Light and shadow are a key dimension for evaluating the aesthetics of portrait photography. Because natural light is limited and influenced by many factors during image capture, it is difficult to strictly control. Therefore, professional photographers use specialized lighting equipment to create optimal lighting effects based on the ambient natural light, resulting in portrait photos with good light and shadow. However, this method increases hardware costs. Summary of the Invention

[0004] The purpose of this application is to provide an image processing method and apparatus that can solve the problem of excessively high hardware costs when acquiring portrait photos with good portrait lighting effects.

[0005] In a first aspect, embodiments of this application provide an image processing method, the method comprising:

[0006] Acquire a first image, wherein the first image includes at least one subject being photographed;

[0007] Identify each of the at least one shooting objects;

[0008] Based on the area image corresponding to each of the photographed objects, the fill light intensity information of each of the photographed objects is determined, and the fill light intensity information of each of the photographed objects is the intensity information of the fill light applied to the photographed objects;

[0009] Based on the supplementary light intensity information, supplementary light is applied to the corresponding subject in the at least one shooting subject to obtain a second image.

[0010] Secondly, embodiments of this application provide an image processing apparatus, the apparatus comprising:

[0011] A first acquisition module is used to acquire a first image, wherein the first image includes at least one photographed object;

[0012] The first determining module is used to identify each of the at least one shooting objects;

[0013] The second determining module is used to determine the supplementary light intensity information of each of the shooting objects based on the area image corresponding to each shooting object, wherein the supplementary light intensity information of each shooting object is the intensity information of supplementing light to the shooting object;

[0014] The third determining module is used to apply supplementary lighting to the corresponding shooting objects among the at least one shooting objects according to the supplementary lighting intensity information to obtain the second image.

[0015] Thirdly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0016] Fourthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0017] Fifthly, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0018] In this embodiment, by identifying each subject in a first image including at least one subject, a region image corresponding to each subject can be obtained. For each subject, based on the region image corresponding to the subject, supplementary lighting intensity information for supplementing the subject can be determined. For each subject, supplementary lighting is applied based on the supplementary lighting intensity information for supplementing the subject, resulting in a second image after supplementing the subject. Thus, the image processing method provided in this embodiment can achieve portrait photos with good portrait lighting effects without adding other supplementary lighting devices to supplement the subject. This not only reduces hardware costs but also solves the problem of poor portrait lighting effects caused by insufficient lighting conditions or limitations of electronic device hardware. As a result, high-quality portrait photos are achieved at low cost, while improving the success rate and aesthetics of electronic devices. Attached Figure Description

[0019] Figure 1 This is a schematic flowchart of an image processing method provided in some embodiments of this application;

[0020] Figure 2 This is a schematic diagram illustrating the determination of supplementary light intensity information for supplementing light to a target object, provided by some embodiments of this application;

[0021] Figure 3These are schematic diagrams illustrating the collection of the first training samples provided in some embodiments of this application;

[0022] Figure 4 This is a schematic diagram of the data processing flow for the first training sample provided in some embodiments of this application;

[0023] Figure 5 This is a schematic flowchart of an image processing method provided in some embodiments of this application;

[0024] Figure 6 These are schematic diagrams illustrating the structure of an image processing apparatus according to some embodiments of this application;

[0025] Figure 7 These are schematic diagrams illustrating the structure of an electronic device according to some embodiments of this application;

[0026] Figure 8 These are schematic diagrams illustrating the hardware structure of an electronic device according to some embodiments of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0028] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and are not limited in number; for example, a first object can be one or N objects. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0029] As described in the background section, existing solutions require high hardware costs to obtain portrait photos with good lighting effects. To address this issue, this application provides an image processing method and apparatus. By identifying each subject in a first image including at least one subject, a region image corresponding to each subject can be obtained. For each subject, based on the region image corresponding to that subject, supplementary lighting intensity information can be determined. For each subject, supplementary lighting is applied based on the supplementary lighting intensity information, resulting in a second image after supplementary lighting for each subject. Thus, the image processing method provided by this application eliminates the need for additional supplementary lighting equipment to obtain portrait photos with good lighting effects. This not only reduces hardware costs but also solves the problem of poor portrait lighting effects caused by insufficient lighting conditions or limitations of electronic device hardware. This achieves high-quality portrait photos at low cost while improving the success rate and aesthetics of electronic devices.

[0030] The technical solution of this application embodiment can be applied to scenarios where portrait photos with poor lighting effects are processed to obtain portrait photos with better lighting effects.

[0031] The image processing method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0032] Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of this application. The subject executing the image processing method can be an electronic device, which may be, but is not limited to, a personal computer (PC), a smartphone, a tablet computer, or a personal digital assistant (PDA).

[0033] like Figure 1 As shown, the image processing method provided in this application embodiment may include S110-S140.

[0034] S110, Obtain the first image.

[0035] The first image can be an image captured using an electronic device with image acquisition capabilities, or it can be an image stored locally or downloaded from the Internet. The first image may include at least one subject being photographed. The subject being photographed can be anything being photographed, such as a person.

[0036] S120, Identify each of at least one shooting object.

[0037] In some embodiments of this application, each photographed object can be identified from the first image, and a region image corresponding to each photographed object can be obtained.

[0038] In some embodiments of this application, in order to accurately obtain the region image corresponding to each shooting object, S120 may specifically include:

[0039] Obtain the position information of each captured object in the mask image of the first image;

[0040] Based on the position information of each subject, each subject is extracted from the mask image of the first image to obtain the region image corresponding to each subject.

[0041] The mask image of the first image can be an image obtained by extracting the foreground image from the first image.

[0042] In some embodiments of this application, the position information of each shooting object in the mask image of the first image can be obtained respectively, and then each shooting object can be extracted from the mask image of the first image according to the position information of each shooting object, thereby obtaining the region image corresponding to each shooting object.

[0043] In some embodiments of this application, the first image can be processed using image processing techniques to obtain a mask image of the first image. For example, the first image can be converted into a binary image through threshold segmentation, and a threshold can be set to distinguish the foreground and background, thereby obtaining the mask image of the first image. Alternatively, it can be obtained through edge detection, such as using an edge detection algorithm (e.g., the Canny edge detection algorithm) to identify edges in the first image, and then using the edge information to generate a mask, thereby obtaining the mask image of the first image. Region growing can also be used to obtain the mask image of the first image. The specific method used to obtain the mask image of the first image can be set according to user needs and is not limited in the embodiments of this application.

[0044] In the embodiments of this application, by obtaining the position information of each shooting object in the mask image of the first image, and then extracting each shooting object from the mask image of the first image according to the position information of each shooting object, the region image corresponding to each shooting object can be accurately obtained.

[0045] S130. Determine the fill light intensity information for each subject based on the area image corresponding to each subject.

[0046] The fill light intensity information for each subject can be the intensity information of the fill light applied to that subject.

[0047] In some embodiments of this application, to improve the efficiency of determining the fill light intensity information for each subject, S130 may specifically include:

[0048] Select the target subject from each subject;

[0049] Based on the image of the area corresponding to the target subject, determine the fill light intensity information for each subject.

[0050] The target subject can be a subject selected from at least one of the first images.

[0051] In some embodiments of this application, one subject can be selected from at least one subject as the target subject, and then the supplementary lighting intensity information for each subject can be determined based on the area image corresponding to the target subject.

[0052] In the embodiments of this application, by selecting one subject from at least one subject as the target subject, and then using the area image of the target subject to determine the supplementary lighting intensity information for each subject, there is no need to process the area image of each subject, thus improving the efficiency of determining the supplementary lighting intensity information for each subject.

[0053] In some embodiments of this application, to improve the flexibility of selecting the target subject, the step of selecting the target subject from each subject may specifically include:

[0054] If the number of objects captured in the first image is 1, then the object captured is determined to be the target object.

[0055] If the number of subjects in the first image is greater than 1, the target subject is determined based on the size information of the face region of each subject.

[0056] In some embodiments of this application, the selection of the target subject can vary depending on the number of subjects in the first image. Specifically, if there is only one subject in the first image, that subject can be identified as the target subject. If there is more than one subject in the first image, the target subject can be determined from multiple subjects based on the size information of the face region of each subject.

[0057] In the embodiments of this application, different methods are selected to determine the target shooting object based on the number of shooting objects in the first image, thereby improving the flexibility of target shooting object selection.

[0058] In some embodiments of this application, when the number of subjects in the first image is greater than one, in order to prevent the lighting effects between subjects in the image after supplemental lighting from changing too drastically, determining the target subject based on the size information of the face region of each subject may specifically include:

[0059] Select the largest size information from the size information of the face region of each subject;

[0060] The subject corresponding to the largest size information is taken as the target subject.

[0061] Among them, the maximum size information can be the largest size information among the size information of the face area of ​​each subject being photographed.

[0062] In some embodiments of this application, the maximum size information can be selected from the size information of the face region of each subject, and then the subject corresponding to the maximum size information can be used as the target subject. This is because the target subject occupies the largest space in the first image. Therefore, the target subject can be used as a reference to determine the supplementary lighting intensity information for other subjects based on the supplementary lighting intensity information for supplementary lighting. In this way, the light and shadow effects between subjects in the supplemented image will not change too much.

[0063] In the embodiments of this application, by taking the target subject as the subject corresponding to the largest size information in the size information of the face region of each subject, the light and shadow effects between the subjects in the image after supplementary lighting will not change too abruptly, thereby obtaining a second image with better light and shadow effects.

[0064] In some embodiments of this application, in order to accurately determine the fill light intensity information of each subject, the step of determining the fill light intensity information of each subject based on the area image corresponding to the target subject may specifically include:

[0065] The image of the area corresponding to the target object is converted to grayscale to obtain the grayscale image of the area.

[0066] The brightness values ​​of each pixel in the grayscale region image are sorted in ascending order to obtain a brightness value sequence;

[0067] The first normalized luminance value sequence is obtained based on the position of each luminance value in the luminance value sequence.

[0068] The second normalized luminance value sequence is obtained based on each luminance value in the luminance value sequence and the position of each luminance value in the luminance value sequence.

[0069] Based on the first normalized luminance value sequence and the second normalized luminance value sequence, determine the fill light intensity information of the target shooting object;

[0070] Based on the fill light intensity information of the target subject, determine the fill light intensity information for each subject.

[0071] The brightness value sequence can be a sequence obtained by sorting the brightness values ​​of each pixel in the grayscale region image in ascending order.

[0072] The first normalized luminance value sequence can be a sequence composed of luminance values ​​normalized to each luminance value, obtained based on the position of each luminance value in the luminance value sequence.

[0073] The second normalized luminance value sequence can be a sequence composed of luminance values ​​normalized from each luminance value in the luminance value sequence and the position of each luminance value in the luminance value sequence.

[0074] In some embodiments of this application, the concept of the Gini coefficient can be used to calculate the light ratio of the target object, and the light ratio can be used as the supplementary lighting intensity information for supplementing the target object.

[0075] Specifically, one approach is to first convert the area image corresponding to the target object into grayscale, obtaining a grayscale image. This involves transforming a three-channel image of the target object's area into a one-channel image. Since the target object's area image is an RGB image, which is a three-channel image, converting it to grayscale is necessary. For example, if the target object's area image is (512, 512, 3), converting it to grayscale will result in an image of (512, 512).

[0076] Then, the brightness values ​​of each pixel in the grayscale region image are sorted in ascending order to obtain a brightness value sequence. This sequence is a 1x1, 1x2, 1x2 sequence of brightness values, where each value is arranged in ascending order. For example, the grayscale region image (512, 512) contains 512 × 512 = 262,144 pixels. The brightness values ​​of these 262,144 pixels are then sorted in ascending order to form the brightness value sequence w(j) = {a1, a2, a3, ... a...}. 262144}

[0077] Then, based on the position of each brightness value in the brightness value sequence, the first normalized brightness value sequence x can be obtained according to the following formula (1). i :

[0078]

[0079] In the above formula (1), i is the position of each brightness value in the brightness value sequence, i∈(1,2,…262144), and the initial value of i is 1.

[0080] In other words, first let i = 1, and according to formula (1), get the value of x1. Then let i = 2, and according to formula (1), get the value of x2. Then let i = 3, and so on, until let i = 262144, and according to formula (1), get x. 262144 The value of x1, x2, ..., x 262144 The sequence is composed to obtain the first normalized luminance value sequence, in which each value is between (0,1).

[0081] Based on each luminance value in the luminance value sequence and its position in the sequence, the second normalized luminance value sequence y can be obtained according to the following formula (2). i :

[0082]

[0083] According to the above formula (2), the sum of the first i brightness values ​​can be obtained by taking the quotient of the sum of each brightness value in the brightness value sequence. i .

[0084] In other words, let i = 1 first, and according to formula (1), we get The result is Then let i = 1, and according to formula (1), we get The result is Then let i = 3, and so on, until i = 262144. According to formula (2), we get y. 262144 The values ​​of y1, y2, ..., y 262144 The sequence is composed to obtain the second normalized luminance value sequence, in which each value is between (0,1).

[0085] Then, based on the first and second normalized luminance value sequences, the supplementary lighting intensity information for the target subject can be obtained. Specifically, each value in the first normalized luminance value sequence can be matched one-to-one with each value in the second normalized luminance value sequence according to the order, that is, x1 corresponds to y1, x2 corresponds to y2, and so on. Figure 2 As shown, each value in the first normalized luminance value sequence is used as the x-axis, and each value in the second normalized luminance value sequence is used as the y-axis. Figure 2In the diagram, curve 21 is the curve fitted by the second normalized brightness value sequence, and the area between curve 21 and the horizontal axis is S. B Since every value in the first normalized luminance value sequence and every value in the second normalized luminance value sequence are between (0,1), the area of ​​the triangle enclosed by the first and second normalized luminance value sequences can be calculated. Then, based on this area and S... B The area S can be calculated. A Then, according to the following formula (3), the supplementary lighting intensity information G for the target object can be obtained:

[0086]

[0087] Then, based on the fill light intensity information of the target subject, the fill light intensity information for each subject can be determined.

[0088] In the embodiments of this application, the supplementary lighting intensity information for each subject is determined by a software method, saving hardware costs. In addition, by calculating the light ratio of the target subject, the supplementary lighting intensity information for the target subject can be obtained, and then the supplementary lighting intensity information for each subject can be obtained. This allows for accurate determination of the supplementary lighting intensity information for each subject, improving the accuracy of the determination of the supplementary lighting intensity information for each subject.

[0089] In some embodiments of this application, when the number of subjects in the first image is greater than one, in order to prevent the light and shadow effects between the subjects in the image after supplemental lighting from changing too drastically, and to improve the efficiency of determining the supplemental lighting intensity information of each subject, the step of determining the supplemental lighting intensity information of each subject based on the supplemental lighting intensity information of the target subject may specifically include:

[0090] The fill light intensity information of the target subject is determined to be the fill light intensity information corresponding to each subject.

[0091] In some embodiments of this application, after obtaining the supplementary lighting intensity information of the target subject, the supplementary lighting intensity information of the target subject can be determined as the supplementary lighting intensity information for each subject separately. This ensures that the portrait of each subject in the same image has the same supplementary lighting intensity information, so as not to be too abrupt.

[0092] In the embodiments of this application, by determining the supplementary lighting intensity information of the target subject as the supplementary lighting intensity information for each subject separately, the lighting and shadow effects between the subjects in the image after supplementary lighting are not too abrupt. Furthermore, by directly determining the supplementary lighting intensity information of the target subject as the supplementary lighting intensity information for each subject separately, there is no need to repeatedly calculate the supplementary lighting intensity information for other subjects, thus improving the efficiency of determining the supplementary lighting intensity information for each subject.

[0093] It should be noted that, in order to further improve the lighting effect of the second image, the lighting intensity information of each subject can also be determined using the method described above for determining the lighting intensity information of the target subject. This can further improve the lighting effect of the second image. Specifically, whether the lighting intensity information of each subject is calculated and determined sequentially, or whether the lighting intensity information of the target subject is used as the lighting intensity information of other subjects, can be selected according to user needs and is not limited in this embodiment.

[0094] S140. Apply supplementary lighting to the corresponding subject in at least one of the shooting objects according to the supplementary lighting intensity information to obtain the second image.

[0095] The second image can be an image with better lighting effects obtained by supplementing the lighting of each subject in the first image.

[0096] In some embodiments of this application, in order to improve the supplementary lighting efficiency for each subject, S140 may specifically include:

[0097] Based on the fill light intensity information of each subject, determine the fill light intensity information image of the first image;

[0098] The image data of the first image, the image data of the mask image of the first image, the image data of the skin image of at least one subject in the first image, and the image data of the supplementary light intensity information image are stitched together to obtain the first image data vector;

[0099] The first image data vector is input into the supplementary lighting model, and supplementary lighting is applied to each subject based on the supplementary lighting model to obtain the second image data vector.

[0100] The second image is determined based on the second image data vector.

[0101] Among them, the supplementary light intensity information image can be an image of supplementary light intensity information obtained based on the supplementary light intensity information of each subject being photographed.

[0102] At least one subject's skin image can be an image of the skin regions of each subject. For example, in the first image, there is the upper body region of subject 1 and the upper body region of subject 2. Subject 1 is wearing a short-sleeved shirt, and subject 2 is wearing a long-sleeved shirt and sunglasses. In the first image, the facial skin of subject 1 and the skin of the arm exposed outside the short-sleeved shirt can be seen. The facial skin of subject 2, excluding the skin covered by sunglasses, is the other facial skin. The image composed of the facial skin of subject 1, the skin of the arm exposed outside the short-sleeved shirt of subject 1, and the other facial skin of subject 2, excluding the skin covered by sunglasses, is the skin image of at least one subject.

[0103] The first image data vector can be an image data vector formed by stitching together the image data of the first image, the image data of the mask image of the first image, the image data of the skin image of at least one subject in the first image, and the image data of the supplementary light intensity information image. For example, the first image data vector can be a data vector of [1,5,1024,1024].

[0104] The second image data vector can be the image data vector output by inputting the first image data vector into the supplementary lighting model. This second image data vector can be a data vector of [1, 3, 1024, 1024].

[0105] The supplementary lighting model can be a pre-trained model used to supplement the lighting of each subject to obtain a second image data vector. This supplementary lighting model can be, but is not limited to, a neural network model based on deep learning, a support vector machine model, or a decision tree model.

[0106] In some embodiments of this application, a supplementary light intensity information image of a first image can be obtained based on the supplementary light intensity information of each photographed object. Then, the image data of the first image, the image data of the mask image of the first image, the image data of the skin image of at least one photographed object in the first image, and the image data of the supplementary light intensity information image can be stitched together to obtain a first image data vector. The first image data vector is then input into the supplementary light model to obtain a second image data vector. The second image can then be determined based on the second image data vector.

[0107] It should be noted that before stitching together the image data of the first image, the image data of the mask image of the first image, the image data of the skin image of at least one subject in the first image, and the image data of the supplementary light intensity information image, the weights of the first image, the mask image of the first image, the skin image, and the supplementary light intensity information image can be set respectively. Then, according to the respective weights of the first image, the mask image of the first image, the skin image, and the supplementary light intensity information image, the image data of the first image, the image data of the mask image of the first image, the image data of the skin image of at least one subject in the first image, and the image data of the supplementary light intensity information image are stitched together.

[0108] The specific weights of the first image, the mask image of the first image, the skin image, and the supplementary light intensity information image can be set based on prior experience or other reference factors. The specific weight values ​​of the first image, the mask image of the first image, the skin image, and the supplementary light intensity information image can be set by the user according to their needs, and are not limited in this embodiment.

[0109] In the embodiments of this application, a supplementary lighting model is used to supplement the lighting of each subject, thereby improving the supplementary lighting efficiency of each subject.

[0110] It should be noted that since the supplementary lighting model processes image data, that is, it processes graphics, the format of the data it processes should conform to the data format that the graphics processor can process. That is, the first image data and the second image data vector mentioned above can both be the first vector format corresponding to the graphics processor. In other words, the first vector format can be a vector format that the graphics processor can process, such as a tensor format.

[0111] In some embodiments of this application, to avoid the need for the user to adjust the obtained image due to its format and size not meeting requirements, the step of determining the second image based on the second image data vector may specifically include:

[0112] The format of the second image data vector is converted to obtain the second image data vector in the second vector format;

[0113] The second image data vector in the second vector format is upsampled to obtain the second image.

[0114] The second vector format can be the vector format corresponding to the processor, such as NumPy format.

[0115] In some embodiments of this application, since the number of channels and size information of the second image data vector output by the supplementary lighting model are different from those of the first image, it is necessary to adjust the number of channels and size information of the second image data vector to be the same as those of the first image. However, this process is performed in the processor. Since the vector format of the data that the processor can process is different from the vector format of the data processed by the graphics processor, it is first necessary to convert the format of the second image data vector to a second vector format that the processor can process. Then, the second image data vector in the second vector format is upsampled to obtain a second image with the same size information and number of channels as the first image.

[0116] In the embodiments of this application, by converting the format of the second image data vector to obtain a second image data vector in a second vector format, and then upsampling the second image data vector in the second vector format, a second image with the same size information and number of channels as the first image can be obtained. This ensures that the obtained second image with better lighting and shadow effects is consistent with the size information of the first image, so that the user does not need to adjust the obtained image because the format and size of the obtained image with better lighting and shadow effects do not meet the requirements, thus simplifying the user operation.

[0117] In some embodiments of this application, in order to improve the training efficiency of the preset supplementary lighting model, before inputting the first image data vector into the supplementary lighting model, supplementing the lighting for each subject based on the supplementary lighting model, and obtaining the second image data vector, the method described above may further include:

[0118] Obtain multiple first training samples;

[0119] Based on the brightness values ​​of each pixel in the region image corresponding to each sample object in the first sample image, the sample supplementary light intensity information image of the first sample image is determined;

[0120] The image data of the first sample image, the image data of the mask image of the first sample image, the image data of the skin image of each sample object in the first sample image, and the image data of the sample supplementary light intensity information image are stitched together to obtain the first sample image data vector;

[0121] The image data of the second sample image is downsampled to obtain the second sample image data vector;

[0122] The second training sample is obtained based on the first sample image data vector and the second sample image data vector;

[0123] The pre-set supplementary lighting model is trained based on multiple second training samples to obtain the supplementary lighting model.

[0124] The first training sample can be an initially acquired sample used to train a preset supplementary lighting model.

[0125] Each of the first training samples mentioned above may include a first sample image and a second sample image. The first sample image may include at least one sample shooting object. The sample shooting object may be a shooting object used to obtain samples for training the preset supplementary lighting model, such as a person. The sample shooting object may be the same as or different from the shooting object in the first image in S110. This application embodiment does not limit this.

[0126] The first sample image mentioned above can be an image captured before supplemental lighting is applied to the sample subject, and the second sample image can be an image captured after supplemental lighting is applied to the sample subject. That is, the second sample image can be an image with perfect lighting and shadow effects.

[0127] In one example, reference Figure 3 Before using the fill light 31 to illuminate the sample object 32, the image acquisition device 33 can be used to capture the sample object 32 to obtain a first sample image. Then, the sample object 32 is kept still, the fill light 31 is turned on, and the fill light 31 is used to illuminate the sample object 32. Then, the image acquisition device 33 is used to capture the sample object 32 to obtain a second sample image.

[0128] Following the above process, different shooting objects can be used to collect several pairs of first and second sampled images to obtain multiple training samples.

[0129] It should be noted that, in Figure 3 In this embodiment, the number of fill lights 31, sample subjects 32, and image acquisition devices 33 are merely examples and not limitations. That is, the number of fill lights 31, sample subjects 32, and image acquisition devices 33 can be set according to user needs and are not limited in this application embodiment.

[0130] It should be noted that before using the fill light and image acquisition equipment, the fill light intensity had been adjusted based on prior experience, and the placement angle, color temperature, camera parameters, etc. of the image acquisition equipment had also been adjusted.

[0131] For each first training sample, the supplementary lighting intensity information image of the first sample image can be an image of supplementary lighting intensity information for each sample object obtained by supplementing lighting to each sample object based on the brightness values ​​of each pixel in the region image corresponding to each sample object in the first sample image. The specific determination of this supplementary lighting intensity information image is the same as the determination method of the supplementary lighting intensity information image in the above embodiment, and will not be repeated here.

[0132] The first sample image data vector can be a data vector obtained by stitching together the image data of the first sample image, the image data of the mask image of the first sample image, the image data of the skin image of each sample subject in the first sample image, and the image data of the sample supplementary lighting intensity information image. The method for determining the first sample image data vector can refer to the method for determining the first image data vector in the above embodiments, and will not be repeated here.

[0133] The second sample image data vector can be an image data vector obtained by downsampling the image data of a second sample image, and the dimension of the second sample image data vector is the same as the dimension of the first sample image data vector. Specifically, the dimensions of the second sample image data vector and the first sample image data vector are consistent with the dimensions of the second image data vector and the first image data vector in the above embodiment.

[0134] The second training sample can be the final training sample obtained after processing the first training sample to train the preset supplementary lighting model. The second training sample can be a training sample obtained based on the first sample image data vector and the second sample image data vector.

[0135] The preset supplementary lighting model can be a pre-set supplementary lighting model that has not yet been trained. After training the preset supplementary lighting model, the above-mentioned supplementary lighting model can be obtained.

[0136] In some embodiments of this application, for each first training sample, a corresponding second training sample can be obtained through the above processing method, and then the second training sample corresponding to each first training sample is used to train the preset supplementary lighting model to obtain the above supplementary lighting model.

[0137] In the embodiments of this application, by processing each initially acquired first training sample, multiple second training samples based on image data vectors can be obtained. Then, based on the multiple second training samples, a preset supplementary lighting model can be trained to obtain the supplementary lighting model. In this way, the preset supplementary lighting model is trained using image data vectors instead of directly using images, which avoids increasing the processing burden of the preset supplementary lighting model due to excessively large images. Training the preset supplementary lighting model using image data vectors reduces the computational power of the preset supplementary lighting model and improves the training efficiency of the preset supplementary lighting model.

[0138] In some embodiments of this application, in order to improve the training effect of the preset supplementary lighting model, after obtaining multiple first training samples, the method described above may further include:

[0139] Multiple first training samples are preprocessed to obtain multiple preprocessed first training samples;

[0140] The step of determining the sample fill light intensity information image of the first sample image based on the brightness values ​​of each pixel in the region image corresponding to each sample object in the first sample image may specifically include:

[0141] For each of the multiple pre-processed first training samples, the sample supplementary lighting intensity information image of the first sample image is determined based on the brightness value of each pixel in the region image corresponding to each sample object in the first sample image of the first training sample.

[0142] In some embodiments of this application, after obtaining multiple first training samples, the quality of the first training samples may be poor, so preprocessing is required. Specific preprocessing may include, but is not limited to, removing first training samples with large deviations between the first sample image and the second sample image, and adjusting first training samples with small deviations between the first sample image and the second sample image.

[0143] In some embodiments of this application, there may be a large deviation between the first sample image and the second sample image of some training samples among multiple first training samples. For example, the movement of the shooting object, the movement of the image acquisition device, the untimely transmission of the signal, the failure to turn on the fill light in time, the incorrect setting of the image acquisition device parameters, the incorrect setting of the fill light parameters, etc., can cause a large deviation between the first sample image and the second sample image. Therefore, it is necessary to remove or process these training samples in order to obtain better training samples.

[0144] Specifically, we can first remove the first training samples from the multiple first training samples, specifically those with significant discrepancies between the first and second sample images. For example, for a particular training sample, after acquiring the first sample image, movement of the subject or the image acquisition device can cause a significant deviation in the position of the subject in the acquired second sample image relative to the first sample image. Consequently, the position and brightness values ​​of each pixel in the first sample image will differ significantly from those in the second sample image. This leads to a mismatch between the first and second sample image data vectors obtained from the first training sample, thus affecting the training effect on the preset supplementary lighting model. Therefore, it is necessary to remove this first training sample.

[0145] After removing the first training samples with significant deviations from the second sample images from a pool of multiple first training samples, the remaining first training samples can be processed as follows: Figure 4 The data processing flow shown is used for processing. Figure 4 As shown, the data processing flow for processing the remaining first training samples includes S410-S440.

[0146] S410. Determine whether the first training sample meets the preset training conditions. If yes, execute S440; otherwise, execute S420.

[0147] The first training sample in S410 refers to the first training sample remaining after removing the first training sample image and the second training sample image with a large deviation from the multiple first training samples obtained.

[0148] The aforementioned preset training conditions can be conditions for whether a first training sample can be directly used to train a preset supplementary lighting model. These preset training conditions may include, but are not limited to, whether the quality of the first sample image and the second sample image in the remaining first training samples meets the requirements. For example, whether the pixels in the first sample image and the pixels in the second sample image are aligned. This is because when acquiring the first sample image and the second sample image in a certain first training sample, the sample shooting object remains stationary, and only supplementary lighting is added. Therefore, for the first sample image and the second sample image in a certain first training sample, the pixels in the two images should be aligned.

[0149] S420. Register and align the first sample image and the second sample image.

[0150] In the remaining first training samples, if the deviation between the first sample image and the second sample image in a certain first training sample is small, the deviation can be adjusted by a registration and alignment algorithm without removing the first training sample. For example, for a certain first training sample, after acquiring the first sample image, when acquiring the second sample image, the subject blinked, meaning the subject's eyes are closed in the second sample image, but open in the first sample image. The deviation between the first and second sample images is small, so the deviation can be corrected by a registration and alignment algorithm without removing the first training sample.

[0151] S430. Remove the black border regions from the first and second sample images after registration and alignment.

[0152] Images that have undergone registration and alignment algorithms may have black borders. Therefore, black borders can be cropped using a cropping algorithm.

[0153] S440. Calculate the mask image of the first sample image and the mask image of the second sample image.

[0154] After preprocessing the first training sample, the mask image of the first sample image and the mask image of the second sample image can be calculated. The mask images of the first sample image and the mask images of the second sample image are used to train the subsequent preset lighting model.

[0155] In the embodiments of this application, the problematic first training sample can be preprocessed to obtain a preprocessed first training sample, which can improve the training accuracy of the preset supplementary lighting model.

[0156] In some embodiments of this application, if the subject is wearing glasses, a hard shadow will be cast on the subject's face when the image is captured. This hard shadow cannot be solved by supplementary lighting. Therefore, when training the preset supplementary lighting model, parameters for shadow degradation of the data can be added to improve the generalization ability of the preset supplementary lighting model.

[0157] In addition, due to the parameter settings of the preset lighting model, there may be a problem that the image data vector obtained after processing by the preset lighting model is blurry. Therefore, when training the preset lighting model, parameters for blurring and degradation of the data can be added to improve the generalization ability of the preset lighting model.

[0158] It should be noted that when setting the parameters for data shadow degradation and data haziness degradation, the parameters for data shadow degradation and data haziness degradation can be set according to a certain ratio. The specific ratio can be set by the user based on their experience, and is not limited in this embodiment.

[0159] In some embodiments of this application, since model training requires a large number of training samples for different situations, such as if the object in the first image is not upright but rotated at a certain angle, if the preset supplementary lighting model is not trained using training samples for this situation, the resulting supplementary lighting model cannot process the first image, or the processing effect on the first image is poor. Therefore, in order to increase the robustness of the supplementary lighting model, data augmentation can be performed on multiple preprocessed first training samples to obtain a larger number of training samples containing more situations. Specific data augmentation can include rotating, mirroring, and color adjusting the first and second sample images in the first training samples.

[0160] It should be noted that when performing data augmentation on a certain first training sample, the first sample image and the second sample image in the first training sample need to be augmented in the same way. For example, if the first sample image in a certain first training sample is rotated 60° to the left, the second sample image in the first training sample also needs to be rotated 60° to the left.

[0161] It should be noted that different proportions of the preprocessed first training samples can be selected for data augmentation in different ways. For example, if there are 10 first training samples, 30% of the first training samples can be selected for rotation data augmentation, 30% can be selected for mirroring data augmentation, and 40% can be selected for color adjustment data augmentation. The proportion of the first training samples selected for different augmentation methods can be set by the user according to their needs, and is not limited in this embodiment.

[0162] In some embodiments of this application, in order to improve the efficiency of supplementing light for images taken under poor lighting conditions, the step of training a preset supplementing light model based on multiple sets of second training samples to obtain a supplementing light model may specifically include:

[0163] For each second training sample, the second training sample is input into the preset supplementary lighting model to obtain the predicted image data vector;

[0164] Based on the predicted image data vector and the second sample image data vector, determine the loss function value of the preset supplementary lighting model;

[0165] If the loss function value does not meet the preset conditions, the model parameters of the preset supplementary lighting model are adjusted, and the preset supplementary lighting model with adjusted model parameters is trained using multiple second training samples until the loss function value meets the preset conditions, thus obtaining the supplementary lighting model.

[0166] The predicted image data vector can be obtained by inputting the second training sample into the preset illumination model, processing the first sample image data vector based on the preset illumination model, and outputting the image data vector after illumination of the image represented by the first sample image data vector.

[0167] The preset condition can be a pre-set condition for the pre-illuminated model to stop training, and the condition that the loss function value must meet. For example, the preset condition can be that the loss function value is less than a certain threshold.

[0168] In some embodiments of this application, for each second training sample, the second training sample can be input into a preset supplementary lighting model, and a predicted image data vector can be output. Then, based on the predicted image data vector and the second sample image data vector, the loss function value of the preset supplementary lighting model can be determined. If the loss function value does not meet the preset conditions, the model parameters of the preset supplementary lighting model can be adjusted. Then, multiple second training samples are used to continue training the preset supplementary lighting model with adjusted parameters until the loss function value meets the preset conditions. At this point, the training of the preset supplementary lighting model can be stopped, thereby obtaining the supplementary lighting model.

[0169] In the embodiments of this application, by continuously training the preset supplementary lighting model using multiple second training samples, a supplementary lighting model can be obtained. This model can then be used to process the first image to obtain the second image, thereby improving the efficiency of supplementing lighting for images taken under poor lighting conditions.

[0170] In some embodiments of this application, in order to improve the robustness of the preset supplementary lighting model, after obtaining multiple first training samples, the method described above may further include:

[0171] For each first training sample, a sample shadow mask image and a sample specular reflection image are obtained based on the first sample image and the second sample image in the first training sample, respectively.

[0172] The step of obtaining the second training sample based on the first sample image data vector and the second sample image data vector may specifically include:

[0173] The second training sample is obtained based on the first sample image data vector, the second sample image data vector, the sample shadow mask image, the sample specular reflection image, the second sample image, and the first sample image.

[0174] The sample shadow mask image can be a mask image obtained from the first sample image and the second sample image to characterize the shadow area of ​​the first sample image relative to the second sample image.

[0175] The sample specular reflection image can be an image obtained from the first sample image and the second sample image, used to characterize the specular reflection of the human figure in the first sample image relative to the second sample image.

[0176] In some embodiments of this application, for each first training sample, a sample shadow mask image and a sample specular reflection image can be obtained respectively based on the first sample image and the second sample image in the first training sample. The sample shadow mask image and the sample specular reflection image can roughly reflect the approximate distribution of shadows on the portrait.

[0177] Specifically, for each first training sample, a sample shadow mask image S representing the shadow region of the first sample image relative to the second sample image can be obtained according to the following formula (4) based on the first sample image and the second sample image:

[0178] S = max(min(1-I) in gray / I gt ),0) (4)

[0179] In the above formula (4), I in gray I is the grayscale image of the first sample image. gt This is the grayscale image of the second sample image.

[0180] For each first training sample, the sample specular reflection image D, which characterizes the specular reflection of the first sample image relative to the second sample image, can be obtained according to the following formula (5) based on the first sample image and the second sample image:

[0181] D = max(min(1-I) gt / I in gray ),0) (5)

[0182] In the above formula (5), I in gray I is the grayscale image of the first sample image. gt This is the grayscale image of the second sample image.

[0183] The sample shadow mask image and sample specular reflection image obtained above can be used to supervise the shadow area or the area with low brightness value in the portrait image (first sample image and second sample image). For the preset lighting model, the ability to identify the details and structural information of the dark area or strong shadow area in the portrait image is weak. Therefore, the sample shadow mask image obtained by formula (4) above, which is used to characterize the shadow area of ​​the first sample image relative to the second sample image, and the sample specular reflection image obtained by formula (5) above, which is used to characterize the portrait specular reflection of the first sample image relative to the second sample image, together with the data vector of the first sample image and the data vector of the second sample image, can be used as the second training sample to train the preset lighting model, thereby enhancing the preset lighting model's ability to identify the details and structural information of the dark area or strong shadow area in the portrait image.

[0184] In the embodiments of this application, for each first training sample, a sample shadow mask image and a sample specular reflection image are obtained respectively based on the first sample image and the second sample image in the first training sample. Then, the sample shadow mask image and the sample specular reflection image are also used as second training samples for training the preset lighting model. This can enhance the preset lighting model's ability to capture details and structural information of dark or strong shadow areas in portrait images, thereby improving the robustness of the preset lighting model.

[0185] In some embodiments of this application, in order to improve the generalization ability and robustness of the preset supplementary lighting model, the step of inputting the second training sample into the preset supplementary lighting model to obtain the predicted image data vector may specifically include:

[0186] Input the second training sample into the preset supplementary lighting model to obtain the predicted image data vector and the predicted image;

[0187] The step of determining the loss function value of the preset supplementary lighting model based on the predicted image data vector and the second sample image data vector includes:

[0188] Based on the predicted image, the sample shadow mask image, the sample specular reflection image, and the second sample image, determine the first loss function value of the preset supplementary lighting model;

[0189] Based on the predicted image data vector and the second sample image data vector, determine the second loss function value of the preset supplementary lighting model;

[0190] Based on the first loss function value and the second loss function value, the loss function value of the preset supplementary lighting model is determined.

[0191] The predicted image can be the image after supplementing the first sample image with supplementary lighting, which is output after the second training sample is input into the preset supplementary lighting model and the first sample image is processed based on the preset supplementary lighting model.

[0192] The first loss function value can be the loss function value of a preset supplementary lighting model obtained based on the predicted image, the sample shadow mask image, the sample specular reflection image, the second sample image, and the first sample image.

[0193] The second loss function value can be the loss function value of the preset supplementary lighting model obtained based on the predicted image data vector and the second sample image data vector.

[0194] In some embodiments of this application, a first loss function value of the preset supplementary lighting model can be obtained based on the predicted image, the sample shadow mask image, the sample specular reflection image, the second sample image, and the first sample image. A second loss function value of the preset supplementary lighting model can be obtained through the predicted image data vector and the second sample image data vector. Then, the loss function value of the preset supplementary lighting model is determined by combining the first loss function value and the second loss function value, thereby comprehensively evaluating the training of the preset supplementary lighting model.

[0195] In the embodiments of this application, the ability of the preset lighting model to identify details and structural information of dark or strong shadow areas in portrait images can be enhanced by using the predicted image, sample shadow mask image, sample specular reflection image, second sample image and first sample image. The ability of the preset lighting model to illuminate portrait images can be enhanced by using the predicted image data vector and the second sample image data vector. The generalization ability and robustness of the preset lighting model can be improved by using a variety of different loss function values.

[0196] In some embodiments of this application, the first loss function value described above may include a first sub-loss function value and a second sub-loss function value.

[0197] The first sub-loss function value mentioned above can be the loss function value of a preset supplementary lighting model obtained based on the predicted image, the second sample image, and the sample shadow mask image. This first sub-loss function value can be used to characterize the difference between the pixels of the predicted image and the pixels of the second sample image.

[0198] The second sub-loss function value can be the loss function value of the preset supplementary lighting model obtained based on the predicted image, the second sample image, and the sample specular reflection image. This second sub-loss function value can be used to characterize the difference between the feature information of the predicted image and the feature information of the second sample image.

[0199] To improve the generalization ability and robustness of the preset supplementary lighting model, the first loss function value of the preset supplementary lighting model is determined based on the predicted image, the sample shadow mask image, the sample specular reflection image, and the second sample image. Specifically, this may include:

[0200] Based on the predicted image, the second sample image, and the sample shadow mask image, determine the value of the first sub-loss function of the preset supplementary lighting model;

[0201] Based on the predicted image, the second sample image, and the sample specular reflection image, determine the value of the second sub-loss function of the preset supplementary lighting model.

[0202] In some embodiments of this application, a first sub-loss function L, characterizing the difference between pixels in the prediction image and pixels in the second sample image, can be calculated based on the prediction image, the second sample image, and the sample shadow mask image, according to the following formula (6). pixel :

[0203]

[0204] In formula (6) above, x is the second sample image. For the predicted image, S is the sample shadow mask image obtained by the above formula (4).

[0205] The second sub-loss function value L, which characterizes the difference between the feature information of the predicted image and the feature information of the second sample image, can be calculated according to the following formula (7) based on the predicted image, the second sample image, and the sample specular reflection image. vgg :

[0206]

[0207] In the above formula (7), D is the sample specular reflection image obtained by the above formula (5), and VGG(·) is the feature information captured based on the VGG network model.

[0208] In the embodiments of this application, the first sub-loss function value of the preset supplementary lighting model is determined by the predicted image, the second sample image, and the sample shadow mask image, and the second sub-loss function value of the preset supplementary lighting model can be determined by the predicted image, the second sample image, and the sample specular reflection image. In this way, by using a variety of different loss function values, the generalization ability and robustness of the preset supplementary lighting model can be improved.

[0209] In some embodiments of this application, the second loss function value described above may include a third sub-loss function value and a fourth sub-loss function value.

[0210] The aforementioned third sub-loss function value can be the loss function value of the preset supplementary lighting model obtained from the predicted image data vector and the second sample image data vector. This third sub-loss function value can be used to characterize the realism of the predicted image data vector.

[0211] The fourth sub-loss function value can be the loss function value of the preset supplementary lighting model obtained from the predicted image data vector and the second sample image data vector. This fourth sub-loss function value can be used to characterize the similarity between the predicted image data vector and the second sample image data vector.

[0212] To improve the generalization ability and robustness of the preset supplementary lighting model, the determination of the first loss function value of the preset supplementary lighting model based on the predicted image data vector and the second sample image data vector may specifically include:

[0213] Based on the predicted image data vector and the second sample image data vector, the values ​​of the third and fourth sub-loss functions of the preset supplementary lighting model are determined.

[0214] In some embodiments of this application, a Generative Adversarial Network (GAN) can be added to the preset supplementary lighting model to improve the realism of the predicted image data vector. Specifically, the third sub-loss function value L, which characterizes the realism of the predicted image data vector, can be obtained according to the following formula (8) based on the predicted image data vector and the second sample image data vector. adv :

[0215]

[0216] In Equation (8), GAN(·) is an algorithm based on the GAN network used to determine the realism of the generated image.

[0217] In some embodiments of this application, a model for calculating structural similarity (SSIM) can be added to the preset lighting model to improve the realism of the output preset image. Specifically, the fourth sub-loss function value L, which characterizes the similarity between the predicted image data vector and the second sample image data vector, can be obtained according to the following formula (9). ssim :

[0218]

[0219] In formula (8), SSIM(·) is the algorithm used to calculate structural similarity.

[0220] In the embodiments of this application, by predicting the image data vector and the second sample image data vector, a third sub-loss function value used to characterize the realism of the predicted image data vector and a fourth sub-loss function value used to characterize the similarity between the predicted image data vector and the second sample image data vector can be obtained in the preset supplementary lighting model. In this way, by using multiple different loss function values, the generalization ability and robustness of the preset supplementary lighting model can be improved.

[0221] After obtaining the first, second, third, and fourth sub-loss function values, the first, second, third, and fourth sub-loss function values ​​can be weighted and summed to obtain the loss function value of the preset supplementary lighting model. The specific weights corresponding to the first, second, third, and fourth sub-loss function values ​​can be set by the user according to their needs, and are not limited in this embodiment.

[0222] To better understand the image processing method of the embodiments of this application, the embodiments of this application are described in detail with specific scenarios.

[0223] like Figure 5 As shown, the image processing method provided in this application embodiment may include S510-S570.

[0224] S510, Obtain the first image.

[0225] S520. Calculate the mask image of the first image.

[0226] S530. Determine whether the number of subjects in the first image is 1. If yes, execute S540; otherwise, execute S550.

[0227] S540. Determine the fill light intensity information for each subject based on the mask image of the first image.

[0228] In this S540, the specific calculation of the fill light intensity information of each shooting object can refer to the method for determining the fill light intensity information of the target shooting object in the above embodiment, and will not be repeated here.

[0229] S550: Obtain the maximum size information of the subject corresponding to the subject whose face region size information is obtained for each subject.

[0230] S560. Use the supplementary lighting model to supplement the light on the subject in the first image to obtain the second image data vector.

[0231] In this S560, based on the determined supplementary lighting intensity information for the subject being photographed, the first image data vector can be calculated according to the calculation method of the first image data vector in the above embodiment. Then, the first image data vector is input into the supplementary lighting model, and the subject in the first image is supplemented with light based on the supplementary lighting model to obtain the second image data vector.

[0232] S570. Post-process the second image data vector to obtain the second image.

[0233] In S570, the process of processing the second image data vector to obtain the second image can be referred to in the above embodiment, and will not be described again here.

[0234] The image processing method provided in this application can be executed by an image processing device. This application uses an image processing device executing the image processing method as an example to illustrate the image processing device provided in this application.

[0235] Figure 6 This is a schematic diagram illustrating the structure of an image processing apparatus according to an exemplary embodiment. Figure 6 As shown, the image processing apparatus 600 may include:

[0236] The first acquisition module 610 is used to acquire a first image, wherein the first image includes at least one photographed object;

[0237] The first determining module 620 is used to identify each of the at least one shooting objects;

[0238] The second determining module 630 is used to determine the supplementary light intensity information of each of the shooting objects based on the area image corresponding to each shooting object, wherein the supplementary light intensity information of each shooting object is the intensity information of supplementing light to the shooting object;

[0239] The third determining module 640 is used to apply supplementary lighting to the corresponding shooting objects among the at least one shooting objects according to the supplementary lighting intensity information to obtain a second image.

[0240] In this embodiment, by identifying each subject in a first image including at least one subject, a region image corresponding to each subject can be obtained. For each subject, based on the region image corresponding to the subject, supplementary lighting intensity information for supplementing the subject can be determined. For each subject, supplementary lighting is applied based on the supplementary lighting intensity information for supplementing the subject, resulting in a second image after supplementing the subject. Thus, the image processing method provided in this embodiment can achieve portrait photos with good portrait lighting effects without adding other supplementary lighting devices to supplement the subject. This not only reduces hardware costs but also solves the problem of poor portrait lighting effects caused by insufficient lighting conditions or limitations of electronic device hardware. As a result, high-quality portrait photos are achieved at low cost, while improving the success rate and aesthetics of electronic devices.

[0241] In some embodiments of this application, the second determining module 630 may include:

[0242] The first selection submodule is used to select the target subject from each shooting object;

[0243] The first determining submodule is used to determine the fill light intensity information of each of the shooting objects based on the area image corresponding to the target shooting object.

[0244] In some embodiments of this application, the first selection submodule may include:

[0245] The first selection unit is used to determine the object being photographed as the target object when the number of objects being photographed in the first image is 1.

[0246] The second selection unit is used to determine the target subject based on the size information of the face region of each subject when the number of subjects in the first image is greater than 1.

[0247] In some embodiments of this application, the first determining submodule may include:

[0248] The first determining unit is used to convert the area image corresponding to the target shooting object to grayscale to obtain the grayscale area image.

[0249] The second determining unit is used to sort the brightness values ​​of each pixel in the grayscale region image in ascending order to obtain a brightness value sequence.

[0250] The third determining unit is used to obtain a first normalized brightness value sequence based on the position of each brightness value in the brightness value sequence.

[0251] The fourth determining unit is used to obtain a second normalized brightness value sequence based on each brightness value in the brightness value sequence and the position of each brightness value in the brightness value sequence.

[0252] The fifth determining unit is used to determine the supplementary lighting intensity information for the target object based on the first normalized brightness value sequence and the second normalized brightness value sequence;

[0253] The sixth determining unit is used to determine the fill light intensity information of each shooting object based on the fill light intensity information of the target shooting object.

[0254] In some embodiments of this application, the third determining module 640 may include:

[0255] The second determining submodule is used to determine the fill light intensity information image of the first image based on the fill light intensity information of each shooting object;

[0256] The third determining submodule is used to stitch together the image data of the first image, the image data of the mask image of the first image, the image data of the skin image of at least one subject in the first image, and the image data of the supplementary light intensity information image to obtain a first image data vector.

[0257] The fourth determining submodule is used to input the first image data vector into the supplementary lighting model, and to supplement the lighting for each shooting object based on the supplementary lighting model to obtain the second image data vector;

[0258] The fifth determination submodule is used to determine the second image based on the second image data vector.

[0259] The image processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0260] The image processing device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0261] The image processing apparatus provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0262] Optionally, such as Figure 7 As shown, this application embodiment also provides an electronic device 700, including a processor 701 and a memory 702. The memory 702 stores a program or instructions that can run on the processor 701. When the program or instructions are executed by the processor 701, they implement the various steps of the above-described image processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0263] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0264] Figure 8 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0265] The electronic device 800 includes, but is not limited to, components such as: radio frequency unit 801, network module 802, audio output unit 803, input unit 804, sensor 805, display unit 806, user input unit 807, interface unit 808, memory 809, and processor 810.

[0266] Those skilled in the art will understand that the electronic device 800 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 810 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 8 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0267] The processor 810 is configured to acquire a first image, the first image including at least one subject; identify each of the at least one subject; determine the fill light intensity information of each subject based on the region image corresponding to each subject, the fill light intensity information of each subject being the intensity information for filling light on the subject; and apply fill light to the corresponding subject in the at least one subject according to the fill light intensity information to obtain a second image.

[0268] Thus, by identifying each subject in a first image that includes at least one subject, a region image corresponding to each subject can be obtained. For each subject, based on the region image corresponding to that subject, supplementary lighting intensity information for supplementing the subject can be determined. For each subject, supplementary lighting is applied based on the supplementary lighting intensity information for that subject, resulting in a second image after supplementing the subject. Thus, the image processing method provided in this application embodiment can obtain portrait photos with good portrait lighting effects without adding other supplementary lighting devices to supplement the subject. This not only reduces hardware costs but also solves the problem of poor portrait lighting effects caused by insufficient lighting conditions or limitations of electronic device hardware. As a result, high-quality portrait photos are achieved at low cost, while improving the success rate and aesthetics of electronic devices.

[0269] Optionally, the processor 810 is further configured to select a target subject from each subject being photographed; and to determine the fill light intensity information of each subject based on the area image corresponding to the target subject being photographed.

[0270] Thus, by selecting one subject from at least one subject as the target subject, and then using the area image of the target subject to determine the supplementary lighting intensity information for each subject, there is no need to process the area image of each subject, thereby improving the efficiency of determining the supplementary lighting intensity information for each subject.

[0271] Optionally, the processor 810 is further configured to determine the photographed object as a target photographed object when the number of photographed objects in the first image is 1; and to determine the target photographed object based on the size information of the face region of each photographed object when the number of photographed objects in the first image is greater than 1.

[0272] Thus, by selecting different methods based on the number of objects in the first image to determine the target object, the flexibility in selecting the target object is improved.

[0273] Optionally, the processor 810 is further configured to: convert the region image corresponding to the target object to grayscale to obtain a grayscale region image; sort the brightness values ​​of each pixel in the grayscale region image in ascending order to obtain a brightness value sequence; obtain a first normalized brightness value sequence based on the position of each brightness value in the brightness value sequence; obtain a second normalized brightness value sequence based on each brightness value in the brightness value sequence and its position in the brightness value sequence; determine the fill light intensity information of the target object based on the first normalized brightness value sequence and the second normalized brightness value sequence; and determine the fill light intensity information of each object based on the fill light intensity information of the target object.

[0274] In this way, by using software methods to determine the supplementary lighting intensity information for each subject, hardware costs are saved. In addition, by calculating the light ratio of the target subject, the supplementary lighting intensity information for the target subject can be obtained, and then the supplementary lighting intensity information for each subject can be obtained separately. This allows for precise determination of the supplementary lighting intensity information for each subject, improving the accuracy of determining the supplementary lighting intensity information for each subject.

[0275] Optionally, the processor 810 is further configured to: determine a supplementary light intensity information image of the first image based on the supplementary light intensity information of each subject; concatenate the image data of the first image, the image data of the mask image of the first image, the image data of the skin image of at least one subject in the first image, and the image data of the supplementary light intensity information image to obtain a first image data vector; input the first image data vector into a supplementary light model, and supplement light to each subject based on the supplementary light model to obtain a second image data vector; and determine a second image based on the second image data vector.

[0276] In this way, by using a fill light model to provide fill light for each subject, the fill light efficiency for each subject is improved.

[0277] It should be understood that, in this embodiment, the input unit 804 may include a graphics processing unit (GPU) 8041 and a microphone 8042. The GPU 8041 processes image data of still images or videos obtained by an image capture device (such as a color camera) in video capture mode or image capture mode. The display unit 806 may include a display panel 8061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 807 includes at least one of a touch panel 8071 and other input devices 8072. The touch panel 8071 is also called a touch screen. The touch panel 8071 may include a touch detection device and a touch controller. Other input devices 8072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0278] The memory 809 can be used to store software programs and various data. The memory 809 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 809 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 809 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0279] Processor 810 may include one or more processing units; optionally, processor 810 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 810.

[0280] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0281] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0282] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0283] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0284] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0285] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0286] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0287] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: Acquire a first image, wherein the first image includes at least one subject being photographed; Identify each of the at least one shooting objects; Based on the area image corresponding to each of the photographed objects, the fill light intensity information of each of the photographed objects is determined, and the fill light intensity information of each of the photographed objects is the intensity information of the fill light applied to the photographed objects; Based on the supplementary light intensity information, supplementary light is applied to the corresponding shooting objects in the at least one shooting object to obtain a second image; The step of determining the fill light intensity information for each of the photographed objects based on the region image corresponding to each of the photographed objects includes: The image of the region corresponding to the target object is converted to grayscale to obtain a grayscale image of the region, wherein the target object is selected from each target object. The brightness values ​​of each pixel in the grayscale region image are sorted in ascending order to obtain a brightness value sequence; The first normalized luminance value sequence is obtained by dividing the position of each luminance value in the luminance value sequence by the number of luminance values ​​in the luminance value sequence. The first i brightness values ​​in the brightness value sequence are summed to obtain a sum value; the second normalized brightness value sequence is obtained based on the quotient of the sum value and the sum of each brightness value in the brightness value sequence, where the initial value of i is 1, and each value in the first normalized brightness value sequence corresponds one-to-one with each value in the second normalized brightness value sequence according to the order. Using each first normalized brightness value in the first normalized brightness value sequence as the abscissa, and each second normalized brightness value in the second normalized brightness value sequence as the ordinate, and fitting each value in the second normalized brightness value sequence to obtain a fitting curve; Calculate the area of ​​the triangle formed by the first normalized brightness value sequence and the second normalized brightness value sequence; and determine the quotient of the area enclosed by the fitted curve and the horizontal axis and the area of ​​the triangle as the fill light intensity information of the target object. Based on the fill light intensity information of the target shooting object, determine the fill light intensity information of each shooting object.

2. The method according to claim 1, characterized in that, The step of determining the fill light intensity information for each of the photographed objects based on the region image corresponding to each of the photographed objects includes: Select the target subject from each subject; Based on the area image corresponding to the target object, determine the fill light intensity information for each target object.

3. The method according to claim 2, characterized in that, The step of selecting the target subject from each subject includes: If the number of objects captured in the first image is 1, then the captured object is determined to be the target object. If the number of subjects in the first image is greater than 1, the target subject is determined based on the size information of the face region of each subject.

4. The method according to any one of claims 1-3, characterized in that, The step of applying supplementary lighting to corresponding subjects in at least one of the shooting subjects according to the supplementary lighting intensity information to obtain a second image includes: Based on the fill light intensity information of each subject, determine the fill light intensity information image of the first image; The image data of the first image, the image data of the mask image of the first image, the image data of the skin image of at least one subject in the first image, and the image data of the supplementary light intensity information image are stitched together to obtain the first image data vector; The first image data vector is input into the supplementary lighting model, and supplementary lighting is applied to each subject based on the supplementary lighting model to obtain the second image data vector. The second image is determined based on the second image data vector.

5. An image processing apparatus, characterized in that, The device includes: A first acquisition module is used to acquire a first image, wherein the first image includes at least one photographed object; The first determining module is used to identify each of the at least one shooting objects; The second determining module is used to determine the supplementary light intensity information of each of the shooting objects based on the area image corresponding to each shooting object, wherein the supplementary light intensity information of each shooting object is the intensity information of supplementing light to the shooting object; The third determining module is used to apply supplementary lighting to the corresponding shooting objects in the at least one shooting object according to the supplementary lighting intensity information to obtain the second image; The second determining module is specifically used for: The image of the region corresponding to the target object is converted to grayscale to obtain a grayscale image of the region, wherein the target object is selected from each target object. The brightness values ​​of each pixel in the grayscale region image are sorted in ascending order to obtain a brightness value sequence; The first normalized luminance value sequence is obtained by dividing the position of each luminance value in the luminance value sequence by the number of luminance values ​​in the luminance value sequence. The first i brightness values ​​in the brightness value sequence are summed to obtain a sum value; the second normalized brightness value sequence is obtained based on the quotient of the sum value and the sum of each brightness value in the brightness value sequence, where the initial value of i is 1, and each value in the first normalized brightness value sequence corresponds one-to-one with each value in the second normalized brightness value sequence according to the order. Using each first normalized brightness value in the first normalized brightness value sequence as the abscissa, and each second normalized brightness value in the second normalized brightness value sequence as the ordinate, and fitting each value in the second normalized brightness value sequence to obtain a fitting curve; Calculate the area of ​​the triangle formed by the first normalized brightness value sequence and the second normalized brightness value sequence; and determine the quotient of the area enclosed by the fitted curve and the horizontal axis and the area of ​​the triangle as the fill light intensity information of the target object. Based on the fill light intensity information of the target shooting object, determine the fill light intensity information of each shooting object.

6. The apparatus according to claim 5, characterized in that, The second determining module includes: The first selection submodule is used to select the target subject from each shooting object; The first determining submodule is used to determine the fill light intensity information of each of the shooting objects based on the area image corresponding to the target shooting object.

7. The apparatus according to claim 6, characterized in that, The first selection submodule includes: The first selection unit is used to determine the object being photographed as the target object when the number of objects being photographed in the first image is 1. The second selection unit is used to determine the target subject based on the size information of the face region of each subject when the number of subjects in the first image is greater than 1.

8. The apparatus according to any one of claims 5-7, characterized in that, The third determining module includes: The second determining submodule is used to determine the fill light intensity information image of the first image based on the fill light intensity information of each shooting object; The third determining submodule is used to stitch together the image data of the first image, the image data of the mask image of the first image, the image data of the skin image of at least one subject in the first image, and the image data of the supplementary light intensity information image to obtain a first image data vector. The fourth determining submodule is used to input the first image data vector into the supplementary lighting model, and to supplement the lighting for each shooting object based on the supplementary lighting model to obtain the second image data vector; The fifth determination submodule is used to determine the second image based on the second image data vector.

Images