Model training method and device

Abstract

The present application discloses a model training method and device thereof, belonging to the field of image processing technology. F groups of images are collected, each group of images includes character images collected under the same soft light intensity, with different light source information or character posture information, different groups of images correspond to different soft light intensities, and F is an integer greater than 1; based on the F groups of images, a first model is trained until a convergence condition is reached to obtain a light and shadow reconstruction model.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to a model training method and a device thereof. Background Art

[0002] With the continuous development of terminal technology, users have higher and higher requirements for the image processing performance of electronic devices. For example, users require electronic devices to be able to reshape the light and shadow of images.

[0003] In the related art, electronic devices can adjust the light source position and light source intensity of an image through a light and shadow reconstruction model to achieve light and shadow reconstruction of the image. The light and shadow reconstruction model is trained by collecting sample images with different light positions and different light intensities in a light field system composed of multiple point light sources.

[0004] However, according to the above method, since the light emitted by the point light source is hard light, the shadow boundary of the light and shadow reshaping image output by the light and shadow reshaping model in the related art is relatively hard, resulting in poor light and shadow processing effect of the electronic device on the image. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a model training method and a device thereof, which can improve the processing effect of light and shadow processing of images by electronic devices.

[0006] In a first aspect, an embodiment of the present application provides a model training method, which includes: collecting F groups of images, each group of images including character images collected under the same soft light intensity but with different light source information or character posture information, different groups of images corresponding to different soft light intensities, and F being an integer greater than 1; based on the F groups of images, training a first model until the convergence condition is reached to obtain a light and shadow reconstruction model.

[0007] In the second aspect, the embodiment of the present application provides a model training device, which includes: an acquisition module and a training module. The acquisition module is used to acquire F groups of images, each group of images includes character images acquired under the same soft light intensity, with different light source information or character posture information, different groups of images correspond to different soft light intensities, and F is an integer greater than 1; the training module is used to train the first model based on the F groups of images acquired by the acquisition module until the convergence condition is reached to obtain a light and shadow reconstruction model.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0012] In an embodiment of the present application, the electronic device can collect F groups of images, and based on the F groups of images, train the first model until the convergence condition is reached to obtain a light and shadow reconstruction model. Among them, each group of images in the F groups of images includes character images with different light source information or character posture information collected under the same soft light intensity, different groups of images correspond to different soft light intensities, and F is an integer greater than 1. Through this scheme, since the electronic device can collect F groups of images, and the F groups of images correspond to different soft light intensities, and the soft light intensities are different, the light quality and shadow of the image are different, so the first model is trained based on the F groups of images, so that the first model can learn the light and shadow effects of images under different light qualities and shadows, that is, the trained light and shadow reconstruction model has the image processed into an image under any soft light intensity, and the processed image has the real light and shadow effect under the soft light intensity, so as to improve the light and shadow processing effect of the image.

[0013] Furthermore, since each group of images includes images of characters with different character postures or light source information, the light and shadow reconstruction model can perform light and shadow reconstruction processing on images with any character posture and light source information, thereby improving the versatility and generalization of the light and shadow reconstruction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flowchart of a model training method provided in an embodiment of the present application;

[0015] Figure 2A is a schematic diagram of a light source system and an image captured in the light source system provided in an embodiment of the present application;

[0016] Figure 2B is a schematic diagram of a portrait image used in a model training method provided in an embodiment of the present application;

[0017] Figure 2C is a shadow schematic diagram of a portrait image used in a model training method provided in an embodiment of the present application;

[0018] Figure 2D is a schematic diagram of highlights of a portrait image used in a model training method provided in an embodiment of the present application;

[0019] Figure 3A is a structural schematic diagram of a light source system provided in an embodiment of the present application;

[0020] Figure 3B This is a schematic diagram of light positions that can be provided by a light source system provided in an embodiment of the present application;

[0021] Figure 3C is a structural schematic diagram of a light source system provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of a portrait image involved in a model training method provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of a portion of a hard shadow image involved in a model training method provided in an embodiment of the present application;

[0024] Figure 6 A schematic diagram of the architecture of a light and shadow reshaping model provided in an embodiment of the present application;

[0025] Figure 7 A schematic diagram of diffuse reflection properties of a portrait corresponding to a set of images with minimum soft light intensity is shown;

[0026] Figure 8 It is a schematic diagram of a portrait image, a grayscale image, and a normal line of a portrait surface involved in a model training method provided in an embodiment of the present application;

[0027] Fig. 9A A schematic diagram of a light position transformation module in a light and shadow reconstruction model provided in an embodiment of the present application;

[0028] Fig. 9B Schematic diagram of the image before and after the light position transformation;

[0029] Fig. 9C A schematic diagram of the structure of a light and shadow weight module and a softening module in a light and shadow reconstruction model provided in an embodiment of the present application;

[0030] Fig. 10A is a schematic diagram of a hard shadow portrait image involved in a model training method provided in an embodiment of the present application;

[0031] Fig. 10B is a schematic diagram of a soft light portrait image involved in a model training method provided in an embodiment of the present application;

[0032] Fig.11A schematic diagram of the structure of a model training device provided in an embodiment of the present application;

[0033] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0034] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0036] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0037] The model training method and device provided in the embodiments of the present application are described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0038] The model training method and device provided in the embodiments of the present application can be applied to the scene of reshaping light and shadow of images.

[0039] The model training method provided in the embodiment of the present application can be executed by a model training device, which can be an electronic device, or a functional module and an entity module in the electronic device that can implement the model training method. The specific method can be determined according to actual use requirements, and the embodiment of the present application is not limited. The following takes the execution of the model training method by an electronic device as an example to illustrate the model training method provided in the embodiment of the present application.

[0040] The present application embodiment provides a model training method, Figure 1 The flowchart of the model training method provided in the embodiment of the present application is shown. Figure 1 As shown, the model training method provided in the embodiment of the present application may include steps 101 to 102.

[0041] Step 101: An electronic device collects F groups of images.

[0042] Among them, each group of images in the F groups of images includes character images with different light source information or character posture information collected under the same soft light intensity, different groups of images correspond to different soft light intensities, and F is an integer greater than 1.

[0043] In some embodiments of the present application, the electronic device can collect F groups of images in a light source system with adjustable soft light intensity. In actual implementation, the collection of F groups of images can also be implemented in other systems. The embodiments of the present application are not limited to this.

[0044] It can be understood that the above light source system is a light source system that conforms to the aesthetics of portrait photography.

[0045] In some embodiments of the present application, the above-mentioned light source system with adjustable soft light intensity may include P evenly distributed point light sources, and a transparent medium layer located in a first regular polyhedron with the P point light sources as vertices, the shape of the transparent medium layer is the same as that of the first regular polyhedron, and the vertices of the transparent medium layer are aligned with the vertices of the first regular polyhedron, and P is an integer greater than 1.

[0046] The soft light intensity of the light source system is negatively correlated with the transparency of the light-transmitting medium layer, and the transparency of the light-transmitting medium layer changes with the change of the electrical information applied to the light-transmitting medium layer.

[0047] In some embodiments of the present application, each of the P point light sources can change brightness and color.

[0048] In some embodiments of the present application, the light source system may further include: a plurality of support rods, which constitute the edges of the first regular polyhedron, so that the P point light sources may be distributed on the vertices of the first regular polyhedron.

[0049] In some embodiments of the present application, "the vertices of the light-transmitting medium layer are aligned with the vertices of the first regular polyhedron" may include: each face in the light-transmitting medium layer is parallel to the face adjacent to the face in the first regular polyhedron. In other words, the light-transmitting medium layer is aligned with the vertices of the first regular polyhedron, and the edges and faces are parallel.

[0050] In some embodiments of the present application, the transparency of the light-transmitting medium layer is negatively correlated with the light transmittance t of the light-transmitting medium layer. It can be understood that the higher the transparency of the light-transmitting medium layer, the smaller the light transmittance t of the light-transmitting medium layer. The light transmittance t of the light-transmitting medium layer can indicate the transparency of the light-transmitting medium layer.

[0051] In some embodiments of the present application, the transparency of the light-transmitting medium layer may change as the electrical information applied to the light-transmitting medium layer changes.

[0052] For example, the transparency of the light-transmitting medium layer can increase with the increase of the voltage applied to the light-transmitting medium layer, that is, the two are positively correlated. Specifically, when the voltage applied to the light-transmitting medium layer is larger, the transparency of the light-transmitting medium layer is larger, and when the applied voltage reaches the maximum, the transparency of the light-transmitting medium layer reaches the maximum, such as a completely transparent state (at this time, the transparency coefficient t=0); when the voltage applied to the light-transmitting medium layer is disconnected, the transparency of the light-transmitting medium layer is the minimum, and it can be an atomized state (at this time, the transparency coefficient t=1).

[0053] In some embodiments of the present application, the light from the point light source can generate light of different soft light intensities after passing through the transparent medium layer with different transparencies. The greater the transparency of the transparent medium layer, the harder the light from the point light source after passing through the transparent medium layer, that is, the stronger the sense of direction of the light; the smaller the transparency of the transparent medium layer, the softer the light from the point light source after passing through the transparent medium layer, that is, the weaker the directionality of the light or the larger the light surface.

[0054] It should be noted that the increase in the value of the transparency coefficient t of the light-transmitting medium layer indicates that the transparency of the light-transmitting medium layer is gradually decreasing. For example, when the transparency coefficient t=0, it can be indicated that the transparency of the light-transmitting medium layer is the largest, that is, the light-transmitting medium layer is completely transparent, and the light of the point light source is the hardest after passing through the light-transmitting medium layer; when the transparency coefficient t=1, it can be indicated that the transparency of the light-transmitting medium layer is the smallest, and the light of the point light source is the softest after passing through the light-transmitting medium layer. In this way, by changing the light transmittance (i.e. transparency) of the transparent medium layer, the effect image of the portrait under soft light of different intensities can be obtained, and the t value from 0-1 represents a gradual decrease in the light transmittance.

[0055] It should be noted that the soft light intensity of the light source system is different due to the different light transmittance of the transparent medium layer. Therefore, the soft light intensity of the light source system can be adjusted by adjusting the transparency coefficient t of the transparent medium layer, that is, the transparency coefficient t corresponds to the soft light intensity one by one, and the larger the transparency coefficient t, the larger the soft light intensity of the light source system.

[0056] In the above light source system, by changing the transparency of the transparent medium layer, the light of the point light source can form a surface light source with different softness after passing through the transparent medium layer. For example, the light can be switched from hard light to large-area soft light through the transparent medium layer.

[0057] For example, Figure 2A , which are schematic diagrams of the light source system when the transparency coefficients of the light-transmitting medium layer are t1, t2 and t3, and schematic diagrams of portrait images taken in the light source system under the corresponding transparency coefficients, wherein the values ​​of t1, t2 and t3 increase in sequence. Figure 2AIt can be seen that the smaller the transparency coefficient, the harder the light shining on the subject, making the shadow boundary of the captured image more obvious and abrupt; similarly, the larger the transparency coefficient, the softer the light shining on the subject, making the shadow boundary of the captured image softer and the image quality better. In this way, by changing the light transmittance of the light-transmitting medium layer, the effects of portraits under soft light of different intensities can be obtained, and the larger the t value, the smaller the light transmittance.

[0058] In some embodiments of the present application, when t=0, the image captured in the light source system is a hard shadow image; when t=1, the image captured in the light source system is a full soft light image.

[0059] In some embodiments of the present application, the light-transmitting medium layer may be formed by splicing multiple pieces of polymer dispersed liquid crystal (PDLC) glass.

[0060] In some embodiments of the present application, the sizes of the multiple pieces of PDLC glass may be the same, wherein the size of the PDLC glass may include: shape and area.

[0061] In some embodiments of the present application, the transparency of the plurality of PDLC glasses can be adjusted individually or uniformly.

[0062] For example, each piece of PDLC glass may be provided with an electric information regulating device, so that the voltage applied to each piece of PDLC glass can be regulated by the electric information regulating device of each piece of PDLC glass, so as to regulate the transparency of each piece of PDLC glass. It is understood that in this case, the transparency of the multiple pieces of PDLC glass may be the same or different.

[0063] For example, multiple PDLC glasses can share one electrical information adjustment device, so that the voltage applied to the PDLC glass can be uniformly adjusted by the electrical information adjustment device to adjust the transparency of the PDLC glass. It can be understood that in this case, the transparency of the multiple PDLC glasses is the same.

[0064] It can be understood that PDLC glass can switch between a light scattering state and a transparent state under the action of an electric field. Specifically, when no voltage is applied, the optical axes of the liquid crystal droplets are disordered, the light is strongly scattered, and the glass appears opaque or translucent. After an electric field is applied, the refractive index of the liquid crystal particles matches that of the matrix, and the light can pass through the glass, presenting a transparent state.

[0065] In this way, since the transparent medium layer in the light source system is composed of multiple pieces of PDLC glass, and the PDLC glass can switch between the light scattering state and the transparent state under the action of the electric field, the light source system can simulate a lighting environment with different soft light intensities by applying different electrical information to the transparent medium layer, thereby not only improving the lighting performance of the light source system, but also improving the convenience and accuracy of adjusting the soft light intensity of the light source system.

[0066] In this way, in the light source system with adjustable soft light intensity, since the transparent medium layer is located in the first regular polyhedron formed by P point light sources and has the same shape as the first regular polyhedron, and the vertices of the transparent medium layer are aligned with the P point light sources, when the electronic device shoots a person in the transparent medium layer, the light source system can simulate the lighting environment in various real scenes such as indoors, outdoors, during the day, and at night by adjusting the transparency of the transparent medium layer. Therefore, it can ensure that the electronic device captures multiple groups of portrait images with different soft light intensities, thereby ensuring that the light and shadow reconstruction model trained based on these portrait images learns the light and shadow effects of portraits under different soft light levels. In this way, the light and shadow processing ability of the light and shadow reconstruction model for the image can be improved, and the light and shadow processing effect of the image can be improved.

[0067] In some embodiments of the present application, F groups of images may correspond to F soft light intensities, that is, the F groups of images correspond to F transparency coefficients of the transparent medium layer, and the values ​​of the F transparency coefficients are different, that is, different groups of images correspond to different transparency coefficients t.

[0068] For example, the values ​​of the F transparency coefficients are: 0, 0.2, 0.4, 0.6, 0.8, 1.0.

[0069] In some embodiments of the present application, the light source information may include at least one of the following: light source position, light intensity, light color, etc.

[0070] In some embodiments of the present application, "each group of images includes character images with different light source information or character posture information captured under the same soft light intensity" may include: the images in each group of images have the same soft light intensity, and the light source information or character posture information of any two images in the group of images is different.

[0071] In some embodiments of the present application, the above-mentioned “the electronic device acquires F groups of images in a light source system with adjustable soft light intensity” may include:

[0072] For the j-th group of images in the F-group of images, when the light-transmitting medium layer is at the j-th transparency:

[0073] When the subject is in the kth person posture among W person postures, an image of the subject is captured when each of the multiple point light sources is lit, and P images corresponding to the kth person posture are obtained, where W, j and k are positive integers.

[0074] When the subject is in the k+1th posture among W postures, when each of the multiple point light sources is lit, an image of the subject is captured to obtain P images corresponding to the k+1th posture, until P images corresponding to the Wth posture are obtained, and the jth group of images is obtained.

[0075] It should be noted that the photographed object may be located in a space enclosed by the light-transmitting medium layer, such as in the center of the light source system. Furthermore, the electronic device may collect the above-mentioned group F images when located in the space. Figure 3A As shown in the figure, it is a schematic diagram of the structure of the light source system. Figure 3C As shown, it is a schematic diagram of a photographed subject 61 standing in the center of a light source system 62 .

[0076] In some embodiments of the present application, when different point light sources in the above light source system are turned on, lighting scenes under different light positions can be realized, such as Figure 3B The lighting scenes shown are backlit scene, frontlit scene, bottom-lit scene, and top-lit scene.

[0077] In the embodiment of the present application, the light from the point light source can be irradiated onto the photographed object through the light-transmitting medium layer.

[0078] In some embodiments of the present application, P point light sources may be lit up in sequence within a preset time. Furthermore, the electronic device may capture P images within the preset time, that is, the P point light sources correspond to the P images one by one. The preset time may be 1s, 1.2s, 1.3s, 0.8s, etc.

[0079] For example, assuming P=100, the preset duration is 1s, and t=0, the light source system can control 100 point light sources to light up in sequence, so that the electronic device can take an image of the subject when each point light source is lit. In this way, the electronic device can collect 100 portrait images with hard shadows within 1s.

[0080] In some embodiments of the present application, the lighting order of the P point light sources may be controlled by an algorithm or may be preset by a user.

[0081] In some embodiments of the present application, in a lighting process of P point light sources, each point light source can maintain the same light intensity. It can be understood that in a lighting process of P point light sources, each point light source is lit once.

[0082] In some embodiments of the present application, when one of the P point light sources is turned on, the other point light sources remain off.

[0083] In some embodiments of the present application, when one of the P point light sources is lit, at least some of the point light sources in the P point light sources that are different from the one point light source may also remain lit, and the brightness (or light intensity) of the at least some of the point light sources is less than the brightness (or light intensity) of the one point light source. That is, the one point light source serves as the main light source, and the at least some of the point light sources may serve as auxiliary light sources.

[0084] It can be understood that "when the subject is in the kth character posture among W character postures, when each point light source among the multiple point light sources is lit, an image of the subject is captured to obtain P images corresponding to the kth character posture", may include: when the character posture of the subject is the kth character posture, the light source system can control the multiple point light sources to light up in sequence, and the electronic device can capture an image of the subject when each point light source is lit, to obtain P images corresponding to the kth character posture.

[0085] For example, Figure 4 As shown, Figure 4 The portrait image captured by the electronic device when the point light source located on the left side of the photographed object is lit and the transparency coefficient of the transparent medium layer is t = 0. It can be seen that the portrait image is a hard shadow image of the scene captured under the hard shadow lighting condition.

[0086] In some embodiments of the present application, when the transparency of the transparent medium layer is the maximum, a hard shadow dataset of the photographed object can be obtained. The hard shadow dataset can be used for training the first model.

[0087] In some implementations of the present application, the electronic device can adopt the single variable principle in the light source system to perform single variable switching acquisition of light position switching, soft light intensity switching, and character posture switching, thereby obtaining a data set including different soft light intensities, different light source positions, and different character postures, that is, the above-mentioned F group of images. In other words, the above-mentioned F group of images includes all-light position switching images of portrait images, images with different soft light intensities, and images with different character postures.

[0088] The following describes the process of an electronic device collecting the above F groups of images with reference to specific examples.

[0089] In some embodiments of the present application, a light source system that conforms to the aesthetics of portrait photography can be provided, wherein the hardware acquisition part of the system is a lighting experimental platform (lightstage) based on variable light glass (i.e., PDLC glass), and all components in the platform (such as point light sources and variable light glass) are electronically controlled elements with extremely fast responses, which can be switched multiple times in a short period of time. For example, P point light sources can be lit one by one within 1 second, so that the electronic device can capture images of the same portrait under different light positions within 1 second, thereby simulating user alignment data in various real-life scenes such as indoors, outdoors, during the day, and at night.

[0090] The light source system provided in the embodiment of the present application may include a polyhedral lightstage frame composed of multiple support rods, and a regular polyhedral controllable glass curtain (i.e., the above-mentioned light-transmitting medium layer) located inside the frame. A point light source that can change brightness and color is provided on each node (i.e., vertex) of the frame. The regular polyhedral controllable glass curtain is spliced ​​by multiple pieces of PDLC glass whose transmittance can be adjusted by electrical signals. The electronic device collects the above-mentioned F group of images, which may include the following steps:

[0091] Step 01: Figure 3C As shown, during the initial shooting, the subject can be located in the center of the shooting studio (i.e., the light source system), and the electronic device is located in front of the subject. Then, the transparency of the PDLC glass can be adjusted to be completely transparent (i.e., t=0), and then the point light sources are lit one at a time, and the light of the lit point light sources can pass through the PDLC glass and shine on the subject located inside the PDLC glass to simulate a hard shadow lighting scene, so that the electronic device can capture an image of the subject, for example, Figure 3C In the shooting scene shown in FIG. 1 , when the point light source on the left face of the subject is lit, the electronic device can capture a hard shadow image, such as Figure 4 shown.

[0092] It can be understood that the lights of each node on the framework are turned on one by one at an interval of 10ms, and each light maintains the same light intensity. Within 1s, 100 lights are turned on one by one, so that the electronic device can take 100 image sets (photo_pair) of the user (i.e., the photographed object) under different light positions within 1s. The 100 images can constitute a hard shadow image set of the user in the posture p1. For example, Figure 5 FIG. 1 shows some images in the hard shadow image set. By analogy, the electronic device can respectively shoot the hard shadow image sets of the object in W postures through the above method to obtain the final hard shadow data set.

[0093] Step 02, the user still stands in the center of the studio, and then takes data sets at t of 0.2, 0.4, 0.6, 0.8 and 1.0 respectively, to obtain data sets under 5 different soft light intensities. It can be understood that the shooting process of the 5 image sets is the same as the above step 01. It can be understood that the hard shadow data in step 01 and the 5 data sets shot in step 02 can be collectively referred to as softened data sets. It can be understood that the softened data set is the above group F images.

[0094] It should be noted that the above steps 01 and 02 are illustrated by taking all images corresponding to t by changing the light position and the person's posture in sequence while keeping t unchanged. In actual implementation, the order of change of the above variables is not limited. For example, the light position and the person's posture can be kept unchanged, and within the first preset time, t is taken as 0, 0.2, 0.4, 0.6, 0.8, and 1.0 in sequence, so that the electronic device can take 6 groups of images corresponding to the 6 t values, and the light position and the person's posture of the 6 groups of images are the same. Then, within the second preset time, the light position or the person's posture is changed, and then t is taken as 0, 0.2, 0.4, 0.6, 0.8, and 1.0 in sequence, so that the electronic device can take 6 groups of images corresponding to the 6 t values; and so on, until the 6 groups of images under the last light position and the last person's posture are collected, the above soft light data set can be obtained.

[0095] For example, when the subject is in the kth figure posture among W figures, when the transparency coefficient corresponding to the transparency of the transparent medium layer is t=0, the light of each point light source is turned on one by one at an interval of 10ms, and each light maintains the same light intensity. Within 1s, if 100 lights in the light source system are turned on one by one, then an image of the subject is taken when each light is turned on one by one, and 100 images of the subject in different light positions in the kth figure posture can be obtained. Figure 5 As shown, Figure 5 The hard shadow image of the photographed object when some point light sources are lit in sequence is shown. In the embodiment of the present application, the relevant description when the photographed object is in the k+1th person posture among the W person postures refers to the relevant description in the above embodiment, and in order to avoid repetition, it is not repeated here.

[0096] In this way, when the light-transmitting medium layer is at the jth transparency, since images of the subject under different light source information and different character postures can be collected separately, a group of images with the same soft light intensity but different character postures and light source information can be obtained. This can improve the content richness of the F group of images, thereby improving the light and shadow reconstruction ability of the light and shadow reconstruction model trained based on the F group of images.

[0097] Step 102: The electronic device trains the first model based on the F groups of images until a convergence condition is reached to obtain a light and shadow reconstruction model.

[0098] In some embodiments of the present application, the convergence conditions may include at least one of the following: the error between the predicted value and the true value is less than a preset threshold, the weight change between two iterations is less than a preset threshold, the model reaches the maximum number of iterations, the loss function no longer changes significantly, and the performance of the training set and the validation set no longer improves significantly.

[0099] In some embodiments of the present application, the light reconstruction model can be used to perform light reconstruction processing on the image.

[0100] Among them, the light and shadow reshaping processing may include at least one of the following: light position transformation processing and softening processing.

[0101] The light position transformation process may include at least one of light source position adjustment and light intensity adjustment.

[0102] In some embodiments of the present application, the first model may be any possible model such as a neural network model, an artificial intelligence model, etc.

[0103] In some embodiments of the present application, the first model may include at least one of the following: a light position transformation module, a light and shadow weight module, and a softening module. The light position transformation module may be used to change the light source information of the image, the light and shadow weight module may be used to obtain a light and shadow weight map in the image, wherein the light and shadow weight map may indicate highlight areas and shadow areas to be optimized in the image, and the softening module may be used to process highlight areas and shadow areas in the image to obtain a softened image.

[0104] It can be understood that the model architecture of the light and shadow reshaping model is the same as the model architecture of the first model.

[0105] In some embodiments of the present application, Figure 6 As shown, it is a schematic diagram of the framework of the light and shadow reconstruction model provided by the embodiment of the present application. It can be seen that after the portrait image is input into the first model, the first model can first re-render the portrait image through the light position transformation module to obtain the image after the light source position is switched, and then soften the light and shadow of the image after the light source position is switched through the light and shadow weight module and the softening module to obtain the light and shadow image. Among them, the parameters of the light position transformation module can be obtained by training based on the full light switching data of the transparent medium with t=1 or 0.2; the parameters of the soft light module and the light and shadow weight module can be obtained by training based on the variable light simulation of various hard light images as input data and the soft light image as the true value data.

[0106] In some embodiments of the present application, during the light and shadow reconstruction model training process of the electronic device, the training order of the light position transformation module, the light and shadow weight module and the softening module in the first model can be determined according to the specific usage situation, and the embodiments of the present application are not limited to this.

[0107] It can be understood that the above-mentioned F group of images includes user data (i.e., portrait images) in different postures, different light positions and different soft light intensities. Therefore, the trained light and shadow reconstruction model can perform more realistic and comprehensive light and shadow reconstruction processing on portrait images taken in any outdoor scene.

[0108] In some embodiments of the present application, the above step 102 can be specifically implemented through the following step 102A.

[0109] Step 102A: The electronic device uses M groups of images in F groups of images as input data and N groups of images in F groups of images as label data, trains a first model until a convergence condition is reached, and obtains a light and shadow reconstruction model.

[0110] The soft light intensity corresponding to the M group of images is less than the soft light intensity corresponding to the N group of images, M+N=F, and F, M and N are positive integers.

[0111] It can be understood that the transparency coefficient t corresponding to the M group of images is smaller than the transparency coefficient t corresponding to the N group of images.

[0112] In some embodiments of the present application, the M groups of images may be referred to as training data, and the N groups of images may be used as true value data, reference data, or label data.

[0113] In some embodiments of the present application, the M group of images may be images with t less than or equal to 0.4, and the t value corresponding to the M group of images may be specifically determined according to the transparency coefficient corresponding to the F group of images.

[0114] In some embodiments of the present application, the M group of images may include at least one of the following: an image corresponding to a transparency coefficient of 0, and an image corresponding to a transparency coefficient of 0.2. In other words, the M group of images may include an image with at least one of t values ​​of 0 and 0.2.

[0115] In some embodiments of the present application, N groups of images correspond to N soft light intensities, that is, N groups of images correspond to N transparency coefficient t values.

[0116] In some embodiments of the present application, the above-mentioned N groups of images may include images in which the value of t is at least one of 0.6, 0.8 and 1.0.

[0117] In some embodiments of the present application, the default label data may be the portrait image set collected when t = 1, that is, the full soft light portrait image is used as the label data. Of course, in order to improve the light and shadow reshaping ability of the light and shadow reshaping model, the soft light intensity or soft light degree preferred by the user can also be controlled by adjusting the size of t.

[0118] In this way, since the transparency coefficient can characterize the soft light intensity of the image, and the soft light intensity of the image is different, the light quality and shadow of the image are different, so N groups of images with larger transparency coefficients are used as label data, and M groups of images with smaller transparency coefficients are used as input data. This can enable the light and shadow reconstruction model to learn the light and shadow effects of portraits under different soft light levels, thereby improving the light and shadow reconstruction ability of the light and shadow reconstruction model.

[0119] The model training method provided in an embodiment of the present application is described below by taking an example where an electronic device trains a first model based on an image in M ​​groups of images.

[0120] In some embodiments of the present application, for each image in the M groups of images, the electronic device may execute the following steps 102A1 and 102A2 until a model convergence condition is reached to obtain a light and shadow reconstruction model.

[0121] Step 102A1, the electronic device inputs the ith image in the M groups of images into the first model, and through the softening module in the first model, based on the first light and shadow weight image corresponding to the ith image and the soft light intensity corresponding to the N groups of images, performs N softening processes on the ith image to obtain N softened images of the ith image.

[0122] The value range of i is [1, M*Q], Q is the number of images in each group of images, and i and Q are positive integers.

[0123] In some embodiments of the present application, the first light and shadow weight image may indicate, in the ith image, highlight areas that need to be darkened and shadow areas that need to be softened and brightened.

[0124] In some embodiments of the present application, the first light and shadow weight image may include a highlight weight image S and a shadow weight image D.

[0125] The highlight weight image S may indicate a highlight area that needs to be darkened, and the shadow weight image D may indicate a shadow area that needs to be softened and brightened.

[0126] For example, Figure 2B It is a portrait image of the subject in a lighting scene with top death light. Figure 2C for Figure 2B The shadow weight image D of the image shown, Figure 2D for Figure 2B The highlight weight image S of the image shown in .

[0127] In some embodiments of the present application, the soft light intensity corresponding to the N groups of images can be represented by the transparency coefficients corresponding to the N groups of images. That is, in actual implementation, the electronic device can obtain N softened images of the i-th image under N transparency coefficients t based on the i-th image, the first light and shadow weight image, and the N transparency coefficients t corresponding to the N groups of images through a softening module. The N groups of images correspond one to one to the N transparency coefficients. That is, the softening degree or soft light intensity of the N softened images is different.

[0128] For example, the values ​​of the transparency coefficients corresponding to N (N=3) groups of images may be 0.6, 0.8, and 1.0, so that the electronic device can perform three softening processes on the i-th image based on the three transparency coefficients through the softening module to obtain a softened image of the i-th image under the soft light intensities represented by the three transparency coefficients. It can be understood that the softening module has different softening degrees for the i-th image based on different coefficients.

[0129] It can be understood that each transparency coefficient corresponds to a degree of image softening, and different transparency coefficients correspond to different degrees of image softening.

[0130] Step 102A2: The electronic device adjusts parameters of the softening module based on the light and shadow parameters of the N softened images and the light and shadow parameters of the N first images.

[0131] The N first images include images whose first information matches the i-th image in the N groups of images, the N first images correspond one-to-one to the N groups of images, and the first information may include portrait posture information and light source information.

[0132] In some embodiments of the present application, the light and shadow parameters for softening the image may include at least one of the following: a darkening parameter for the highlight area, a softening and brightening parameter for the shadow area, color temperature, clarity, and saturation, etc.

[0133] In some embodiments of the present application, the electronic device can calculate a loss value based on each first image and the softened image corresponding to the first image, so that the electronic device can calculate N loss values ​​based on N first images and N softened images, and then the electronic device can adjust the parameters of the softening module respectively based on the N loss values.

[0134] It can be understood that the soft light intensity corresponding to the first image matches the soft light intensity of the softened image corresponding to the first image, such as the transparency coefficients corresponding to the two are the same.

[0135] In some embodiments of the present application, the electronic device can determine N loss values ​​based on the light and shadow parameters of a softened image and the light and shadow parameters of a first image, and adjust the parameters of the softening module based on the N loss values. If all of the N loss values ​​are less than or equal to the preset threshold, the parameter adjustment of the softening module is terminated; if at least one of the N loss values ​​is greater than the preset threshold, the above step 102A1 is performed again until the calculated loss value is less than or equal to the preset threshold. Among them, the t value of the softened image compared each time is the same as the t value of the first image.

[0136] In some embodiments of the present application, using the loss value to adjust the parameters of the softening module is one possible implementation method. In actual implementation, the parameters of the softening module can also be adjusted in other ways, for example, the first model reaches the maximum number of iterations, the loss function no longer changes significantly, etc. The embodiments of the present application are not limited to this.

[0137] It should be noted that, in the above embodiment, for each input data, N label data with different soft light intensities are taken as an example to adjust the parameters of the softening module.

[0138] In actual implementation, the parameters of the softening module can be adjusted with each image in the M group of images and with label data of a soft light intensity until the convergence condition is reached. Then, the parameters of the softening module can be adjusted with each image in the M group of images and with label data of another soft light intensity until the convergence condition is reached.

[0139] For example, first, take M sets of images as input data, and take the image at t=1 as true value data, and adjust the parameters of the softening module until the convergence condition is reached. Then, take M sets of images as input data, and take the image at t=0.8 as true value data, and adjust the parameters of the softening module until the convergence condition is reached.

[0140] In some embodiments of the present application, the t values ​​of the N first images can be 0.6, 0.8, or 1.0, so that the softening module can learn the light and shadow effects of the portrait under different degrees of soft light. In this way, the softening module after adjusting the parameters can soften the image into a softened image with t=0.6, 0.8 or 1.0, thereby improving the flexibility of image softening processing.

[0141] In this way, since the electronic device can guide the softening module to perform N softening processes on the ith image based on the soft light intensities corresponding to N groups of images based on the first light and shadow weight image corresponding to the ith image, N softened images of the ith image can be obtained, and the parameters of the softening module can be adjusted based on the light and shadow parameters of the N softened images and the light and shadow parameters of the N first images. Therefore, the softening module after adjusting the parameters can soften the image into softened images corresponding to different soft light intensities, thereby improving the flexibility of softening the image.

[0142] In some embodiments of the present application, before the above-mentioned step 102A1, the model training method provided in the embodiment of the present application may include the following steps 103 to 105.

[0143] Step 103: The electronic device determines a second light and shadow weight image corresponding to the i-th image based on the second image through the light and shadow weight module in the first model.

[0144] The second image is an image corresponding to the maximum soft light intensity in the N groups of images, and the first information matches the i-th image.

[0145] In some embodiments of the present application, the second image corresponds to the maximum soft light intensity in the N groups of images, that is, the second image is an image with full soft light intensity when t=1.0.

[0146] Step 104: The electronic device adjusts the weight of the second light and shadow weight image based on the facial feature information of the portrait in the i-th image to obtain the first light and shadow weight image.

[0147] In some embodiments of the present application, the second light and shadow weight image may be the initial light and shadow weight image of the i-th image. The second light and shadow weight image may include a highlight weight image S' and a shadow weight image D'. The highlight weight image S' represents the actual distribution of highlights in the i-th image, and the shadow weight image D' may represent the actual distribution of shadows in the i-th image.

[0148] In some embodiments of the present application, the highlight weight image S and the shadow weight image D for solving the first light and shadow weight image can be obtained using formula (1) and formula (2) respectively.

[0149] S=f(max(min(1–It1 / I,1),0)) (1)

[0150] D=f(max(min(1–I / It1,1),0)) (2)

[0151] Among them, I represents the i-th image currently input to the light and shadow weight module, which can be the image at t=0 or t=0.2, I can represent the hard shadow image that needs to be optimized, It1 represents the full soft light image with the same first information as the i-th image and t=1.0, and the f function represents the adaptive processing algorithm function of the facial features.

[0152] In some embodiments of the present application, the electronic device can reduce the brightness of some areas in the highlight weight image S', such as the forehead, nose wings, etc., based on the facial feature information of the portrait in the i-th image, to obtain the highlight weight image S, so as to avoid the softening module from darkening the image area corresponding to the partial area in the i-th image, that is, retaining the highlights in the corresponding area in the softened image of the i-th image, such as retaining the highlights in the forehead area in the softened image.

[0153] Similarly, the electronic device can soften and brighten some areas in the shadow weight image D', such as the eyebag area and the bottom of the nose, based on the facial feature information of the portrait in the i-th image, to obtain the shadow weight image D, so as to avoid the softening module brightening the image area corresponding to the partial area in the i-th image, that is, the corresponding area in the softened image of the i-th image retains the shadow, such as the eyebag area in the softened image retains the shadow. In other words, the electronic device can adjust the weight of the second light and shadow weight image based on the facial feature information of the portrait in the i-th image to obtain the first light and shadow weight image.

[0154] It is understandable that in aesthetic photography, users have two requirements for the softened part of the portrait: controlling the degree of softening and the softened part. Too high a softening degree will result in the complete removal of shadows and the loss of three-dimensionality of the character. Too low a softening degree will result in too heavy shadows, affecting the appearance. In certain areas of the facial features, such as bags under the eyes and the bridge of the nose, shadows need to be retained to reflect the three-dimensional effect. In this way, based on the facial feature information of the portrait in the i-th image, the second light and shadow weight image is weighted to obtain the first light and shadow weight image, which can make the first light and shadow weight image more in line with the user's requirements for portrait softening.

[0155] In some embodiments of the present application, in the area of ​​the facial features of the first light and shadow weight image, the electronic device can perform local adjustment of the light and shadow weight, and the local adjustment strategy can be specified in advance by the user.

[0156] In some embodiments of the present application, the electronic device may perform local adjustment of the light and shadow weights in the area of ​​the facial features of the first light and shadow weight image, and the local adjustment strategy may be based on the adjustment performed by the electronic device according to the second image.

[0157] Step 105: The electronic device adjusts parameters of the light and shadow weight module based on the first light and shadow weight image and the second light and shadow weight image.

[0158] In some embodiments of the present application, the electronic device can control the degree and location of softening according to the user's requirements for portrait softening in aesthetic photography, and adjust the parameters of the light and shadow weight module based on the first light and shadow weight image and the second light and shadow weight image.

[0159] For other descriptions on adjusting the parameters of the light and shadow weight module, refer to the related descriptions on adjusting the parameters of the softening module in the above embodiment, which will not be repeated here to avoid repetition.

[0160] It should be noted that, in the embodiment of the present application, step 102A may include step 102A1, steps 103 to 105, and step 102A.

[0161] In some embodiments of the present application, the execution order of step 105 and step 102A1 is not limited.

[0162] In this way, the electronic device can determine the second light and shadow weight image corresponding to the i-th image based on the second image through the light and shadow weight module, and can adjust the weight of the second light and shadow weight image based on the facial features information of the portrait in the i-th image to obtain the first light and shadow weight image, and then the electronic device can adjust the parameters of the light and shadow weight module based on the first light and shadow weight image and the second light and shadow weight image. Since the electronic device can adjust the weight of the second light and shadow weight image based on the facial features information of the portrait in the i-th image to obtain the first light and shadow weight image, the first light and shadow weight image can guide the softening module of the first model to soften the i-th image. Therefore, it can avoid the problem that the electronic device softens the portrait image too much, resulting in the complete removal of the shadow and the loss of the three-dimensional sense of the character, and it can also avoid the problem that the electronic device softens the portrait image too little, resulting in the shadow being too heavy and affecting the appearance. Thereby, the portrait image can retain shadows in certain areas of the facial features, such as the eye bags and the bridge of the nose, reflecting the three-dimensional sense of the portrait image, and improving the processing effect of the electronic device on the light and shadow reshaping of the image.

[0163] In some embodiments of the present application, in the above step 102, "the electronic device trains the first model based on F groups of images" can be specifically implemented through the following steps 106 and 107.

[0164] Step 106: The electronic device inputs the first group of images corresponding to the minimum soft light intensity in the F group of images into the first model, and calculates the light position transformation parameters of each image in the first group of images through the light position transformation module in the first model.

[0165] The light position transformation parameters include: portrait texture feature information, portrait normal feature information and light source information.

[0166] It can be understood that the first group of images corresponding to the minimum soft light intensity in the F group of images is a group of images at t=0.

[0167] In some embodiments of the present application, the first group of images corresponding to the minimum soft light intensity in the F group of images includes: a group of images with the same character posture and different light source information, and a group of images with different character postures and the same light source information.

[0168] In some embodiments of the present application, the electronic device may calculate the light position transformation parameters of each image in the first group of images by using the re-rendering equation in the light position transformation module in the first model based on the first group of images.

[0169] In some embodiments of the present application, the re-rendering equation formula may be the following formula (3).

[0170] L o (v) = ∫ Ω f(v,l)L i (l)<n·l> dl (3)

[0171] Among them, f(v,l) represents the physical properties of the portrait surface, which can be represented by the bidirectional reflectance distribution function (BRDF) model; Li(l) represents the light intensity corresponding to each image in the light source system, which can be represented by the intensity of the point light source; n represents the normal information of the portrait surface; v represents the angle information of the electronic device shooting; l represents the direction of the light irradiated by the point light source; Ω represents the integral around the incident point of the portrait.

[0172] In some embodiments of the present application, in the above formula (3), f(v, l) and n are unknown integral terms, and other terms can be obtained by measuring the physical position and light intensity of the point light source distribution of the light source system.

[0173] In some embodiments of the present application, the portrait model is based on the BRDF principle, and f(v,l) satisfies the following formula (4).

[0174] f(v,l)=fd(v,l)+fs(v,l) (4)

[0175] Among them, fd(v,l) represents the diffuse reflection term of the portrait, and fs(v,l) represents the highlight term of the portrait.

[0176] In some embodiments of the present application, the diffuse reflection term fd(v,l) of the portrait can describe the texture features and skin color features of the portrait surface, and is independent of the shooting position of the electronic device. The expression formula of fd(v,l) can be expressed by formula (5).

[0177] fd(v,l)=σ / π (5)

[0178] In some embodiments of the present application, the portrait highlight term fs(v,l) can be described by the Cook-Forrance model, and the expression formula of fs(v,l) can be expressed by formula (6), and fs(v,l) is equivalent to f(l,v).

[0179]

[0180] Among them, h is a half-way vector, which can be represented by v and l, so that the highlight term fs(v,l) can be represented by the vectors l, v, n. l represents the input direction of light from each point light source in the light source system, v represents the shooting angle of the electronic device, l and v are known vectors, and the highlight term fs(v,l) is related to the normal vector n of the portrait. Therefore, the above formula (3) can be simplified to the following formula (7).

[0181] I=R(n,σ,E) (7)

[0182] Among them, I represents the generated 2D image, n represents the normal of the portrait, σ represents the surface texture albedo of the portrait, and E represents the light source information.

[0183] It can be understood that formula (7) includes light position transformation parameters: σ represents the texture feature information of the portrait, n represents the normal feature information of the portrait, and E represents the light source information.

[0184] The light source information represented by E may include at least one of the following: light intensity, light position, light color, etc.

[0185] In some embodiments of the present application, σ may represent the diffuse reflection property item of the portrait.

[0186] In some embodiments of the present application, the electronic device can perform weighted summation on the first group of images corresponding to the minimum soft light intensity in the F group of images to obtain a texture image of the portrait under uniform lighting, and use the texture image as the diffuse reflection attribute item σ of the portrait. σ can be solved using the following formula (8).

[0187] σ=∑w i *I i (8)

[0188] Among them, w i represents the weight of each image in the first group of images. The electronic device can obtain it by calculating the distance between the portrait and the point light source. i Represents a portrait image captured when each point light source is lit.

[0189] For example, Figure 7 As shown, Figure 7The diffuse reflection properties of the portrait corresponding to a set of images with minimum soft light intensity are shown.

[0190] In some embodiments of the present application, the electronic device can use photometric stereo to solve the normal vector n of the portrait surface. The photometric stereo can be calculated using formula (9).

[0191] I=σ·n T S=σ·S T n (9)

[0192] Wherein, σ is the texture amount of the portrait surface in the above formula (8); I is the captured portrait image; S represents the light source direction vector corresponding to the image, which can be replaced by the vector of the point light source and the person.

[0193] In some embodiments of the present application, the normal line n of the portrait surface can be solved by the least square method, and the expression of the normal line n of the portrait surface can be expressed by the following formula (10).

[0194] n=(S T S) -1 S T Iσ -1 (10)

[0195] Among them, I represents the portrait image, S represents the light source direction vector corresponding to image I, σ represents the texture amount of the portrait surface, and S T A vector representing the transposed vector of the light source direction.

[0196] For example, Figure 8 As shown, Figure 8 This is a schematic diagram of the portrait image and the solved portrait normal geometry. Figure 8 The first image in is a portrait image. Figure 8 The second image is a grayscale image of a portrait image. Figure 8 The third image is a schematic diagram of the normal lines of the portrait surface of the portrait image.

[0197] In some embodiments of the present application, in the first group of images in the F group of images corresponding to the minimum soft light intensity, the position and direction of the light source of each image are known, and the electronic device can expand the light source into an environment map (HDRImap) through the sky box method to obtain the expression formula of the light source E.

[0198] In some embodiments of the present application, the electronic device can perform calculations using formulas (7) to (10) on each image in the first group of images to obtain the light position transformation parameters of each image in the first group of images, that is, to obtain n, σ, and E of all true value information of light position reconstruction.

[0199] In some embodiments of the present application, Fig. 9AAs shown, the electronic device can input images in the first group of images with different character postures and the same light source information into the first model, calculate the light position transformation parameters of each image in the first group of images with different character postures and the same light source information through the light position transformation module 30 in the first model, and re-render the image based on the light position transformation parameters to obtain an image with the light source position changed, that is, a light position transformed image.

[0200] Step 107: The electronic device adjusts the parameters of the light level transformation module based on the light level transformation parameter and the target light level transformation parameter corresponding to the light level transformation parameter.

[0201] In some embodiments of the present application, the target light position transformation parameters corresponding to the light position transformation parameters can be understood as: the light position transformation parameters corresponding to the target image having the same character posture information as the input image and the same soft light intensity as the input image.

[0202] In the embodiment of the present application, the electronic device may use the light level transformation parameters obtained in the above step 106 as target light level transformation parameters of the light level transformation module.

[0203] In some embodiments of the present application, the electronic device can input one of the first group of images into the light position transformation module in the first model, and then the electronic device can output an image after light position transformation, compare the light position transformation parameters of the image with the corresponding target light position transformation parameters, and adjust the parameters of the light position transformation module.

[0204] In this way, since the electronic device can input the first group of images corresponding to the minimum soft light intensity in the F group of images into the first model, calculate the light position transformation parameters of each image in the first group of images through the light position transformation module in the first model, and adjust the parameters of the light position transformation module based on the light position transformation parameters and the target light position transformation parameters corresponding to the light position transformation parameters, the electronic device can accurately adjust the parameters of the light position transformation module, and the light position transformation module is a part of the first model, thereby improving the accuracy of the first model training.

[0205] In the model training method provided in some embodiments of the present application, since the electronic device can collect F groups of images in a light source system with adjustable soft light intensity, and the F groups of images correspond to different soft light intensities, and the soft light intensities are different, the light quality and shadow of the image are different; therefore, training the first model based on the F groups of images can enable the first model to learn the light and shadow effects of images under different light qualities and shadows, that is, the trained light and shadow reshaping model has the ability to process the image into an image under any soft light intensity, and the processed image has the real light and shadow effect under the soft light intensity, so as to improve the light and shadow processing effect of the image.

[0206] Furthermore, since each group of images includes images of characters with different character postures or light source information, the light and shadow reconstruction model can perform light and shadow reconstruction processing on images with any character posture and light source information, thereby improving the versatility and generalization of the light and shadow reconstruction model.

[0207] In some embodiments of the present application, after the above step 102, the model training method provided by the embodiment of the present application may include the following step 108.

[0208] Step 108: The electronic device inputs the first portrait image into the light and shadow reconstruction model, performs light and shadow reconstruction processing on the first portrait image, and outputs a light and shadow image of the first portrait image.

[0209] It can be understood that since the light and shadow reconstruction model is trained based on images with different soft light intensities, the light and shadow reconstruction model can perform more realistic and effective light and shadow reconstruction processing on the first portrait image, so that the light and shadow image of the first portrait image has a more realistic light and shadow effect.

[0210] For example, Fig. 10A shows a hard shadow input image at t=0, Fig. 10B The model after light and shadow reconstruction is shown. Fig. 10A The image shown is a light and shadow image after light and shadow reconstruction processing.

[0211] In some embodiments of the present application, the above step 108 can be specifically implemented through the following step 108A.

[0212] Step 108A: The electronic device softens the first portrait image through a softening module in the light and shadow reconstruction model based on the first soft light intensity and the third light and shadow weight image corresponding to the first portrait image, and obtains and outputs a light and shadow image of the first portrait image.

[0213] The first soft light intensity may be greater than the soft light intensity corresponding to the first portrait image.

[0214] In some embodiments of the present application, the first soft light intensity corresponds to a transparency coefficient t=1.

[0215] In some embodiments of the present application, the first soft light intensity may be a default soft light intensity of the softening module, or may be a soft light intensity preferred by the user, such as a soft light intensity selected by the user.

[0216] It can be understood that the soft light intensity selection range of the softening module is determined by the soft light intensities corresponding to the above-mentioned N groups of images.

[0217] In this way, since the first portrait image can be softened based on the first soft light intensity through the softening module in the light and shadow reconstruction model, the light and shadow image of the first portrait image can be made to have light and shadow parameters corresponding to the first soft light intensity, thereby improving the light and shadow reconstruction effect of the light and shadow reconstruction model.

[0218] In some embodiments of the present application, before the above step 108, the model training method provided in the embodiment of the present application may include the following step 109.

[0219] Step 109: The electronic device determines a third light and shadow weight image corresponding to the first portrait image based on the facial feature information of the portrait in the first portrait image through the light and shadow weight module in the light and shadow reconstruction model.

[0220] In this way, since the light and shadow weight module can determine the third light and shadow weight image corresponding to the first portrait image based on the facial feature information of the portrait in the first portrait image, it can ensure that the highlight area and shadow area indicated by the third light and shadow weight image are more consistent with the shadow and highlight distribution in the real lighting environment, and then the softening module can soften the first portrait image more accurately based on the third light and shadow weight image, so that the light and shadow effect of the light and shadow image output by the light and shadow reconstruction model is more realistic.

[0221] In some embodiments of the present application, the above step 108 can be specifically implemented through the following steps 108B and 108C.

[0222] Step 108B: The electronic device performs light position transformation processing on the first portrait image through the light position transformation module in the light and shadow reconstruction model to obtain a first light position transformation image corresponding to the first portrait image.

[0223] Step 108C: The electronic device softens the first light position transformation image through the softening module in the light and shadow reconstruction model based on the first soft light intensity and the fourth light and shadow weight image corresponding to the first light position transformation image, to obtain and output a light and shadow image of the first portrait image.

[0224] In some embodiments of the present application, before executing step 108C, the electronic device may first calculate a fourth light and shadow weight image of the first light position transformation image based on the facial feature information of the portrait in the first light position transformation image through a light and shadow weight module in the light and shadow reconstruction model.

[0225] For example, Fig. 9C This is a schematic diagram of the connection architecture of the light and shadow weight module and the softening module in the light and shadow reshaping model. Fig. 9CAs shown, the light position change map I2 processed by the light position transformation module can be input into the light and shadow weight module 31. The light and shadow weight module 31 can calculate the highlight image S and weight image D of the light position change map I2 based on the facial feature information of the light position change map I2. Then, the softening module 32 can perform softening processing on the light position change map I2 based on the highlight image S, the weight image D and the transparency coefficient t, and output the softened map, i.e., the light and shadow map.

[0226] In some embodiments of the present application, the electronic device can perform light position transformation processing on the first portrait image through the light position transformation module in the light and shadow reconstruction model, and obtain the image of the first portrait image under the new light source E' through the re-rendering equation, thereby realizing the switching of the light source position and obtaining the first light position transformed image corresponding to the first portrait image.

[0227] The re-rendering equation for solving the first light position transformation image corresponding to the first portrait image can be expressed by the following formula (11).

[0228] I=R(n,σ,E') (11)

[0229] Wherein, I represents the first light position transformation image, R() represents the re-rendering algorithm, n is the normal quantity of the portrait surface, σ represents the surface texture of the portrait, and E' represents the transformed light source.

[0230] For example, Fig. 9B As shown, E is the light source position in the first portrait image, and E' is the light position re-rendered by the light position transformation module, that is, the light source position in the first light position transformation image.

[0231] In this way, since the electronic device can first perform light position transformation processing on the first input image through the light position transformation module in the light and shadow reconstruction model, and then soften the image after the light position transformation through the softening module based on the first soft light intensity and the fourth light and shadow weight image corresponding to the first light position transformation image, that is, the light and shadow reconstruction model can perform all-round light and shadow reconstruction on the first input image, thereby improving the light and shadow effect of the final light and shadow image.

[0232] The model training method provided in the embodiment of the present application can be executed by a model training device. In the embodiment of the present application, the model training device executing the model training method is taken as an example to illustrate the model training device provided in the embodiment of the present application.

[0233] like Fig.11As shown, the embodiment of the present application further provides a model training device 1500, which may include: an acquisition module 1501 and a training module 1502. The acquisition module 1501 may be used to acquire F groups of images, each group of images including character images acquired under the same soft light intensity with different light source information or character posture information, different groups of images correspond to different soft light intensities, and F is an integer greater than 1; the training module 1502 may be used to train a first model based on the F groups of images acquired by the acquisition module 1501, until a convergence condition is reached, and a light and shadow reconstruction model is obtained.

[0234] In some embodiments of the present application, the above-mentioned training module 1502 can be specifically used to use M groups of images in the above-mentioned F groups of images as input data and N groups of images in the above-mentioned F groups of images as label data to train the first model until the convergence condition is reached to obtain a light and shadow reconstruction model.

[0235] The soft light intensity corresponding to the M group of images is less than the soft light intensity corresponding to the N group of images, M+N=F, and F, M and N are positive integers.

[0236] In some embodiments of the present application, the above-mentioned training module 1502 can be specifically used to: input the i-th image in the M groups of images into the first model, and through the softening module in the first model, based on the first light and shadow weight image corresponding to the i-th image and the soft light intensity corresponding to the N groups of images, perform N softening processes on the i-th image to obtain N softened images of the i-th image, where the value range of i is [1, M*Q], Q is the number of images in each group of images, and i and Q are positive integers; and adjust the parameters of the softening module based on the light and shadow parameters of the N softened images and the light and shadow parameters of the N first images.

[0237] The N first images include images whose first information matches the i-th image in the N groups of images, the N first images correspond one-to-one to the N groups of images, and the first information includes portrait posture information and light source information.

[0238] In some embodiments of the present application, the above-mentioned device may also include: a determination module. The determination module may be used to determine the second light and shadow weight image corresponding to the i-th image through the light and shadow weight module in the first model based on the second image before the above-mentioned training module 1502 performs N softening processes on the i-th image based on the first light and shadow weight image corresponding to the i-th image and the soft light intensity corresponding to the N groups of images to obtain N softened images of the i-th image through the softening module in the first model, wherein the second image is an image corresponding to the maximum soft light intensity in the N groups of images and the first information matches the i-th image; the above-mentioned training module 1502 may be specifically used to adjust the weight of the second light and shadow weight image based on the facial feature information of the portrait in the i-th image to obtain the first light and shadow weight image, and adjust the parameters of the light and shadow weight module based on the first light and shadow weight image and the second light and shadow weight image.

[0239] In some embodiments of the present application, the above-mentioned training module 1502 can be specifically used to: input the first group of images corresponding to the minimum soft light intensity in the F group of images into the first model, and calculate the light position transformation parameters of each image in the first group of images through the light position transformation module in the first model; based on the light position transformation parameters and the target light position transformation parameters corresponding to the light position transformation parameters, adjust the parameters of the light position transformation module.

[0240] The light position transformation parameters include: portrait texture feature information, portrait normal feature information and light source information.

[0241] In some embodiments of the present application, the above-mentioned light source system includes P uniformly distributed point light sources, and a light-transmitting medium layer located in a first regular polyhedron with the P point light sources as vertices, the shape of the light-transmitting medium layer is the same as that of the first regular polyhedron, and the vertices of the light-transmitting medium layer are aligned with the vertices of the first regular polyhedron, and P is an integer greater than 1.

[0242] The soft light intensity of the light source system is negatively correlated with the transparency of the light-transmitting medium layer, and the transparency of the light-transmitting medium layer changes with the change of the electrical information applied to the light-transmitting medium layer.

[0243] In some embodiments of the present application, the light-transmitting medium layer is formed by splicing multiple pieces of polymer dispersed liquid crystal (PDLC) glass.

[0244] In some embodiments of the present application, the acquisition module 1501 may be specifically used to: for the j-th group of images in the F-th group of images, when the light-transmitting medium layer is at the j-th transparency:

[0245] When the subject is in the kth person posture among W person postures, an image of the subject is captured when each of the multiple point light sources is lit, and P images corresponding to the kth person posture are obtained. The value range of k is [1, W], and W, j and k are positive integers.

[0246] When the subject is in the k+1th posture among W postures, when each of the multiple point light sources is lit, an image of the subject is captured to obtain P images corresponding to the k+1th posture, until P images corresponding to the Wth posture are obtained, and the jth group of images is obtained.

[0247] In some embodiments of the present application, the apparatus further includes a processing module. The processing module may be configured to input the first portrait image into the light and shadow reconstruction model after the training module 1502 obtains the light and shadow reconstruction model, perform light and shadow reconstruction processing on the first portrait image, and output a light and shadow image of the first portrait image.

[0248] In some embodiments of the present application, the above-mentioned processing module can be specifically used to soften the first portrait image through the softening module in the light and shadow reshaping model based on the first soft light intensity and the third light and shadow weight image corresponding to the first portrait image, to obtain and output the light and shadow image of the first portrait image, wherein the first soft light intensity is greater than the soft light intensity corresponding to the first portrait image.

[0249] In some embodiments of the present application, the above-mentioned device may further include: a determination module. The determination module may be used to determine the third light and shadow weight image corresponding to the first portrait image through the light and shadow weight module in the light and shadow reconstruction model based on the facial feature information of the portrait in the first portrait image before the above-mentioned processing module softens the first portrait image through the softening module in the light and shadow reconstruction model based on the third light and shadow weight image and the first soft light intensity corresponding to the first portrait image to obtain and output the light and shadow image of the first portrait image.

[0250] In some embodiments of the present application, the above-mentioned processing module can be specifically used to: perform light position transformation processing on the first portrait image through the light position transformation module in the light and shadow reconstruction model to obtain a first light position transformation image corresponding to the first portrait image; and perform softening processing on the first light position transformation image based on the first soft light intensity and the fourth light and shadow weight image corresponding to the first light position transformation image through the softening module in the light and shadow reconstruction model to obtain and output the light and shadow image of the first portrait image.

[0251] In an embodiment of the present application, since the model training device can collect F groups of images, and the F groups of images correspond to different soft light intensities, and the soft light intensities are different, the light quality and shadow of the image are different, therefore, training the first model based on the F groups of images can enable the first model to learn the light and shadow effects of images under different light qualities and shadows, that is, the trained light and shadow reconstruction model has the ability to process the image into an image under any soft light intensity, and the processed image has the real light and shadow effect under the soft light intensity, so as to improve the light and shadow processing effect of the image.

[0252] The model training device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtualreality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personalcomputer, PC), a television (television, FV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.

[0253] The model training device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0254] The model training device provided in the embodiment of the present application can implement each process implemented in the above method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0255] Alternatively, if Fig.12 As shown, an embodiment of the present application also provides an electronic device 1600, including a processor 1601 and a memory 1602, wherein the memory 1602 stores a program or instruction that can be executed on the processor 1601, and when the program or instruction is executed by the processor 1601, each step of the above-mentioned model training method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0256] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0257] Fig.13 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0258] The electronic device 1700 includes but is not limited to: a radio frequency unit 1701, a network module 1702, an audio output unit 1703, an input unit 1704, a sensor 1705, a display unit 1706, a user input unit 1707, an interface unit 1708, a memory 1709, and a processor 1710 and other components.

[0259] Those skilled in the art will appreciate that the electronic device 1700 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1710 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.13 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0260] The processor 1710 can be used to collect F groups of images, each group of images includes character images with different light source information or character posture information collected under the same soft light intensity, different groups of images correspond to different soft light intensities, and F is an integer greater than 1; the processor 1710 can be used to train the first model based on the F groups of images collected by the processor 1710 until the convergence condition is reached to obtain a light and shadow reconstruction model.

[0261] In some embodiments of the present application, the processor 1710 can be specifically used to use M groups of images in the above-mentioned F groups of images as input data and N groups of images in the above-mentioned F groups of images as label data, train the first model until the convergence condition is reached, and obtain the light and shadow reconstruction model.

[0262] The soft light intensity corresponding to the M group of images is less than the soft light intensity corresponding to the N group of images, M+N=F, and F, M and N are positive integers.

[0263] In some embodiments of the present application, the processor 1710 can be specifically used to: input the i-th image in the M groups of images into the first model, and through the softening module in the first model, based on the first light and shadow weight image corresponding to the i-th image and the soft light intensity corresponding to the N groups of images, perform N softening processes on the i-th image to obtain N softened images of the i-th image, where the value range of i is [1, M*Q], Q is the number of images in each group of images, and i and Q are positive integers; and adjust the parameters of the softening module based on the light and shadow parameters of the N softened images and the light and shadow parameters of the N first images.

[0264] The N first images include images whose first information matches the i-th image in the N groups of images, the N first images correspond one-to-one to the N groups of images, and the first information includes portrait posture information and light source information.

[0265] In some embodiments of the present application, the processor 1710 can be used to perform N softening processes on the i-th image through the softening module in the first model based on the first light and shadow weight image corresponding to the i-th image and the soft light intensity corresponding to N groups of images, and before the processor 1710 obtains N softened images of the i-th image, determine the second light and shadow weight image corresponding to the i-th image through the light and shadow weight module in the first model based on the second image, where the second image is an image corresponding to the maximum soft light intensity in the N groups of images, and the first information matches the i-th image; the processor 1710 can specifically be used to weight adjust the second light and shadow weight image based on the facial feature information of the portrait in the i-th image to obtain the first light and shadow weight image, and adjust the parameters of the light and shadow weight module based on the first light and shadow weight image and the second light and shadow weight image.

[0266] In some embodiments of the present application, the processor 1710 can be specifically used to: input the first group of images corresponding to the minimum soft light intensity in the F group of images into the first model, and calculate the light position transformation parameters of each image in the first group of images through the light position transformation module in the first model; adjust the parameters of the light position transformation module based on the light position transformation parameters and the target light position transformation parameters corresponding to the light position transformation parameters.

[0267] The light position transformation parameters include: portrait texture feature information, portrait normal feature information and light source information.

[0268] In some embodiments of the present application, the above-mentioned light source system includes P uniformly distributed point light sources, and a light-transmitting medium layer located in a first regular polyhedron with the P point light sources as vertices, the shape of the light-transmitting medium layer is the same as that of the first regular polyhedron, and the vertices of the light-transmitting medium layer are aligned with the vertices of the first regular polyhedron, and P is an integer greater than 1.

[0269] The soft light intensity of the light source system is negatively correlated with the transparency of the light-transmitting medium layer, and the transparency of the light-transmitting medium layer changes with the change of the electrical information applied to the light-transmitting medium layer.

[0270] In some embodiments of the present application, the light-transmitting medium layer is formed by splicing multiple pieces of polymer dispersed liquid crystal (PDLC) glass.

[0271] In some embodiments of the present application, the processor 1710 may be specifically configured to: for the jth group of images in the above F groups of images, when the light-transmitting medium layer is at the jth transparency:

[0272] When the subject is in the kth person posture among W person postures, an image of the subject is captured when each of the multiple point light sources is lit, and P images corresponding to the kth person posture are obtained. The value range of k is [1, W], and W, j and k are positive integers.

[0273] When the subject is in the k+1th posture among W postures, when each of the multiple point light sources is lit, an image of the subject is captured to obtain P images corresponding to the k+1th posture, until P images corresponding to the Wth posture are obtained, and the jth group of images is obtained.

[0274] In some embodiments of the present application, the processor 1710 can be used to input the first portrait image into the light and shadow reconstruction model after the processor 1710 obtains the light and shadow reconstruction model, perform light and shadow reconstruction processing on the first portrait image, and output a light and shadow image of the first portrait image.

[0275] In some embodiments of the present application, the processor 1710 can be specifically used to soften the first portrait image through the softening module in the light and shadow reconstruction model based on the first soft light intensity and the third light and shadow weight image corresponding to the first portrait image, to obtain and output the light and shadow image of the first portrait image, wherein the first soft light intensity is greater than the soft light intensity corresponding to the first portrait image.

[0276] In some embodiments of the present application, the processor 1710 can be used to soften the first portrait image through the softening module in the light and shadow reconstruction model based on the third light and shadow weight image and the first soft light intensity corresponding to the first portrait image, and before the processor 1710 obtains and outputs the light and shadow image of the first portrait image, determine the third light and shadow weight image corresponding to the first portrait image through the light and shadow weight module in the light and shadow reconstruction model based on the facial feature information of the portrait in the first portrait image.

[0277] In some embodiments of the present application, the processor 1710 can be specifically used to: perform light position transformation processing on the first portrait image through the light position transformation module in the light and shadow reconstruction model to obtain a first light position transformation image corresponding to the first portrait image; and perform softening processing on the first light position transformation image based on the first soft light intensity and the fourth light and shadow weight image corresponding to the first light position transformation image through the softening module in the light and shadow reconstruction model to obtain and output the light and shadow image of the first portrait image.

[0278] In an embodiment of the present application, since the electronic device can collect F groups of images, and the F groups of images correspond to different soft light intensities, and the soft light intensities are different, the light quality and shadow of the image are different, therefore, training the first model based on the F groups of images can enable the first model to learn the light and shadow effects of images under different light qualities and shadows, that is, the trained light and shadow reconstruction model has the ability to process the image into an image under any soft light intensity, and the processed image has the real light and shadow effect under the soft light intensity, so as to improve the light and shadow processing effect of the image.

[0279] It should be understood that in the embodiment of the present application, the input unit 1704 may include a graphics processor (Graphics Processing Unit, GPU) 17041 and a microphone 17042, and the graphics processor 17041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1706 may include a display panel 17061, and the display panel 17061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1707 includes a touch panel 17071 and at least one of other input devices 17072. The touch panel 17071 is also called a touch screen. The touch panel 17071 may include two parts: a touch detection device and a touch controller. Other input devices 17072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0280] The memory 1709 can be used to store software programs and various data. The memory 1709 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1709 may include a volatile memory or a non-volatile memory, or the memory 1709 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1709 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0281] The processor 1710 may include one or more processing units; optionally, the processor 1710 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1710.

[0282] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned model training method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0283] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0284] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned model training method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0285] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0286] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned model training method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0287] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be noted that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0288] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0289] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A model training method, characterized in that: The method comprises: Collect F groups of images, each group of images includes person images collected under the same soft light intensity but with different light source information or person posture information, different groups of images correspond to different soft light intensities, and F is an integer greater than 1; Based on the F groups of images, the first model is trained until a convergence condition is reached to obtain a light and shadow reconstruction model.

2. The method according to claim 1, characterized in that The step of training a first model based on the F groups of images comprises: Using M groups of images in the F groups of images as input data and N groups of images in the F groups of images as label data, training the first model until the convergence condition is reached, and obtaining a light and shadow reconstruction model; The soft light intensity corresponding to the M group of images is less than the soft light intensity corresponding to the N group of images, M+N=F, and F, M and N are positive integers.

3. The method according to claim 2, characterized in that Taking M groups of images in the F groups of images as input data and N groups of images in the F groups of images as label data, training a first model includes: Inputting the i-th image in the M groups of images into the first model, and performing N softening processes on the i-th image through the softening module in the first model based on the first light and shadow weight image corresponding to the i-th image and the soft light intensity corresponding to the N groups of images, to obtain N softened images of the i-th image, where the value range of i is [1, M*Q], Q is the number of images in each group of images, and i and Q are positive integers; Adjusting the parameters of the softening module based on the light and shadow parameters of the N softened images and the light and shadow parameters of the N first images; The N first images include images whose first information matches the i-th image in the N groups of images, the N first images correspond one-to-one to the N groups of images, and the first information includes portrait posture information and light source information.

4. The method according to claim 3, characterized in that Before performing N softening processes on the i-th image by the softening module in the first model based on the first light and shadow weight image corresponding to the i-th image and the soft light intensities corresponding to the N groups of images to obtain N softened images of the i-th image, the method further includes: Determining, by the light and shadow weight module in the first model, a second light and shadow weight image corresponding to the i-th image based on the second image, wherein the second image is an image corresponding to the maximum soft light intensity in the N groups of images, and the first information matches the i-th image; Based on the facial feature information of the portrait in the i-th image, weight adjustment is performed on the second light and shadow weight image to obtain the first light and shadow weight image; Based on the first light and shadow weight image and the second light and shadow weight image, parameters of the light and shadow weight module are adjusted.

5. The method according to any one of claims 2 to 4, characterized in that The step of training a first model based on the F groups of images comprises: Inputting a first group of images corresponding to the minimum soft light intensity in the F group of images into the first model, and calculating the light position transformation parameter of each image in the first group of images through the light position transformation module in the first model; Adjusting the parameters of the light position transformation module based on the light position transformation parameters and target light position transformation parameters corresponding to the light position transformation parameters; The light position transformation parameters include: portrait texture feature information, portrait normal feature information and light source information.

6. A model training device, characterized in that: The device comprises: a collection module and a training module; The acquisition module is used to acquire F groups of images, each group of images includes character images acquired under the same soft light intensity but with different light source information or character posture information, different groups of images correspond to different soft light intensities, and F is an integer greater than 1; The training module is used to train the first model based on the F groups of images acquired by the acquisition module until a convergence condition is reached to obtain a light and shadow reconstruction model.

7. The device according to claim 6, characterized in that The training module is specifically used to use M groups of images in the F groups of images as input data and N groups of images in the F groups of images as label data, train the first model until the convergence condition is reached, and obtain the light and shadow reconstruction model; The soft light intensity corresponding to the M group of images is less than the soft light intensity corresponding to the N group of images, M+N=F, and F, M and N are positive integers.

8. The device according to claim 7, characterized in that The training module is specifically used for: Inputting the i-th image in the M groups of images into the first model, and performing N softening processes on the i-th image through the softening module in the first model based on the first light and shadow weight image corresponding to the i-th image and the soft light intensity corresponding to the N groups of images, to obtain N softened images of the i-th image, where the value range of i is [1, M*Q], Q is the number of images in each group of images, and i and Q are positive integers; Adjusting the parameters of the softening module based on the light and shadow parameters of the N softened images and the light and shadow parameters of the N first images; The N first images include images whose first information matches the i-th image in the N groups of images, the N first images correspond one-to-one to the N groups of images, and the first information includes portrait posture information and light source information.

9. The device according to claim 8, characterized in that The device further comprises: a determination module; The determination module is used for, in the training module, performing N softening processes on the i-th image based on the first light and shadow weight image corresponding to the i-th image and the soft light intensities corresponding to the N groups of images through the softening module in the first model, and obtaining N softened images of the i-th image, to determine, based on the second image, a second light and shadow weight image corresponding to the i-th image through the light and shadow weight module in the first model, wherein the second image is an image corresponding to the maximum soft light intensity in the N groups of images, and the first information matches the i-th image; The training module is specifically configured to adjust the weight of the second light and shadow weight image based on the facial feature information of the portrait in the i-th image to obtain the first light and shadow weight image; The training module is specifically used to adjust the parameters of the light and shadow weight module based on the first light and shadow weight image and the second light and shadow weight image.

10. The device according to any one of claims 7 to 9, characterized in that The training module is specifically used for: Inputting a first group of images corresponding to the minimum soft light intensity in the F group of images into the first model, and calculating the light position transformation parameter of each image in the first group of images through the light position transformation module in the first model; Adjusting the parameters of the light position transformation module based on the light position transformation parameters and target light position transformation parameters corresponding to the light position transformation parameters; The light position transformation parameters include: portrait texture feature information, portrait normal feature information and light source information.

Images