Methods, apparatus, electronic equipment and storage media for determining the main material

Abstract

This disclosure provides a method, apparatus, electronic device, and storage medium for determining the material of a subject. The method includes: acquiring a first image to be processed and a second image to be processed, both containing a subject to be identified, from the same visual angle; wherein the first image to be processed is determined under the condition that a flash is on; determining the highlight center point of the first image to be processed, and determining material information to be fused based on the highlight center point and the first image to be processed; determining color information based on the second image to be processed; and determining the target material information of the subject to be identified based on the material information to be fused and the color information. The technical solution provided by this disclosure can conveniently determine the material information of a specific subject using only two images, improving the efficiency of determining material information, thereby resulting in better rendered image effects and facilitating the accurate presentation of the visual effects of the subject to be identified to the user.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and storage medium for determining the material of a main body. Background Technology

[0002] With the increasing demand for computer-based material rendering, material acquisition technology has become increasingly important. Based on material acquisition technology, a large amount of data related to a certain material can be processed and analyzed to obtain the material parameters of that material.

[0003] However, in practical applications, determining the material information of certain materials is extremely cumbersome. Furthermore, due to the complex colors or properties of some materials, such as cosmetics and paints used in decoration, existing methods cannot accurately obtain their material information. Therefore, rendering materials based on material information yields poor results and fails to accurately present the visual effects of the materials to the user. Summary of the Invention

[0004] This disclosure provides a method, apparatus, electronic device, and storage medium for determining the material of a subject, which can conveniently determine the material information of a specific subject using only two images, thereby improving the efficiency of determining material information.

[0005] In a first aspect, embodiments of this disclosure provide a method for determining the material of a main body, the method comprising:

[0006] Acquire a first image to be processed and a second image to be processed, both containing the subject to be identified, from the same visual angle; wherein the first image to be processed is determined based on the condition that the flash is on;

[0007] Determine the highlight center point of the first image to be processed, and determine the material information to be fused based on the highlight center point and the first image to be processed;

[0008] Based on the second image to be processed, determine the color information;

[0009] Based on the material information to be fused and the color information, the target material information of the subject to be identified is determined.

[0010] Secondly, this disclosure also provides a device for determining the main body material, the device comprising:

[0011] The image acquisition module is used to acquire a first image to be processed and a second image to be processed, which include the subject to be identified, from the same visual angle; wherein the first image to be processed is determined based on the condition that the flash is on.

[0012] The material information to be fused module is used to determine the highlight center point of the first image to be processed, and to determine the material information to be fused based on the highlight center point and the first image to be processed;

[0013] The color information determination module is used to determine color information based on the second image to be processed;

[0014] The target material information determination module is used to determine the target material information of the subject to be identified based on the material information to be fused and the color information.

[0015] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0016] One or more processors;

[0017] Storage device for storing one or more programs.

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the body material determination method as described in any embodiment of this disclosure.

[0019] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the body material determination method as described in any of the embodiments of this disclosure.

[0020] The technical solution of this disclosure involves acquiring a first image to be processed and a second image to be processed, including the subject to be identified, from the same visual angle. Specifically, it acquires images of the subject to be identified under both flash-on and flash-off conditions. The highlight center point of the first image to be processed is determined, and based on the highlight center point and the first image to be processed, the material information to be fused is determined. Color information is determined based on the second image to be processed. Furthermore, the target material information of the subject to be identified is determined based on the material information to be fused and the color information. This allows for convenient determination of the material information of a specific subject using only two images, improving the efficiency of material information determination. Simultaneously, by determining the material information to be fused and the color information in a decoupled manner, and then constructing the target material information based on these two types of information, the accuracy of the final material information is improved, resulting in a better rendered image and facilitating the accurate presentation of the visual effect of the subject to be identified to the user. Attached Figure Description

[0021] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0022] Figure 1 This is a flowchart illustrating a method for determining the main material provided in Embodiment 1 of this disclosure;

[0023] Figure 2 This is a structural block diagram of a main body material determination device provided in Embodiment 2 of this disclosure;

[0024] Figure 3 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of this disclosure. Detailed Implementation

[0025] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0026] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0027] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0028] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0029] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0030] Before introducing this technical solution, the application scenarios of the embodiments of this disclosure can be illustrated by example. For instance, in a pre-developed beauty application, based on the solution of this embodiment, users can be provided with the function of trying out certain cosmetics in a virtual scene. Specifically, when a user wants to try out a certain lipstick in a virtual scene, they can click on the trial control corresponding to the lipstick. At this time, the application can call the material information corresponding to the lipstick determined based on the solution of this embodiment, and then render the lipstick on the lips in the user's facial image according to the material information, thereby simulating the visual effect after applying the lipstick to the user. At the same time, it also allows the user to further understand the texture and color presented by the lipstick.

[0031] Alternatively, in home renovation applications, the solution of this embodiment can be used to provide users with a function to simulate the trial of certain paints. Specifically, when a user wants to know the effect of applying a certain paint to a wall, they can click the trial control corresponding to that paint. At this time, the application can call the material information corresponding to that paint, determined based on the solution of this embodiment, and then render the paint on the wall part of the house image according to the material information, thereby simulating the visual effect of the house after being painted with that paint.

[0032] Of course, in addition to various scenarios such as beauty product trials and home renovation simulations, the solutions of this disclosure can also be applied to scenarios such as 3D cloud rendering and virtual shopping. It can be understood as any scenario that requires determining the material information of a specific material and rendering the material based on the material information to accurately present its visual effects to the user.

[0033] Example 1

[0034] Figure 1 This is a flowchart illustrating a method for determining the material of a subject according to Embodiment 1 of this disclosure. This embodiment is applicable to situations where the material information corresponding to a certain subject can be determined in a simple manner. This method can be executed by a subject material determination device, which can be implemented in software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC, or a server. Special effects video displays are typically implemented through cooperation between a client and a server. The method provided in this embodiment can be executed by the server, the client, or through cooperation between the client and the server.

[0035] like Figure 1 The method in this embodiment includes:

[0036] S110. Acquire the first image to be processed and the second image to be processed, which include the subject to be identified, from the same visual angle.

[0037] The subject to be identified can be a material with specific physical properties and capable of exhibiting a certain color, such as cosmetics or chemical mixtures like paints used for decoration. Unlike more common materials, the computer does not store material information for the subject to be identified, therefore it cannot be directly rendered. This can be understood as follows: for an ordinary steel plate, its corresponding material information can be directly accessed in relevant image processing software to draw and render the steel plate. However, for a lipstick composed of multiple components and exhibiting a unique color, the relevant application software does not store its corresponding material information, thus it cannot be reproduced on the display interface. It should be noted that there can be one or more subjects to be identified. In this embodiment, only one subject to be identified is used as an example. Those skilled in the art should understand that when there are multiple subjects to be identified, the material information of each subject can be determined based on the scheme of this embodiment.

[0038] Therefore, in this embodiment, to obtain the material information of the subject to be identified, it is first necessary to acquire two images of the subject from the same visual angle, namely, a first image to be processed and a second image to be processed. The first image to be processed is determined based on the condition that the flash is on. This can be understood as the brightness of the environment in which the subject is located differing between the two images. In acquiring the two images, the camera device can be deployed at a fixed position, the lens pointed at the subject to be identified, and then the flash can be turned on to illuminate and photograph the subject, thus obtaining the first image to be processed. Further, the flash can be turned off and the subject can be photographed again, thus obtaining the second image to be processed.

[0039] In this embodiment, when the subject to be identified is a chemical mixture such as cosmetics or paint, due to its special physical properties, it is necessary to coat the subject to be identified onto a specific object during the process of capturing two images to make the final material information more accurate. In practical application, when the material determination control is triggered, the flash is activated to capture the first image to be processed, in which the subject to be identified is coated onto the target sphere; when the flash is turned off, the second image to be processed, in which the subject to be identified is coated onto the target sphere, is captured.

[0040] The target sphere can be a solid sphere model. It can be understood that when the subject to be identified, such as cosmetics or paint, is coated onto the target sphere, a strong, continuous solid film with a certain strength is formed on the surface of the target sphere; that is, a coating is formed on the surface of the sphere. The material determination control is a control pre-developed in the relevant application software to determine the material information of the subject to be identified. For example, when the user clicks the material information determination button in the application software, the computer can send an on command to the flash device and take a picture of the target sphere coated with the subject to be identified, obtaining the first image to be processed. After obtaining the first image, the computer sends an off command to the flash device and takes another picture of the target sphere, obtaining the second image to be processed. Based on this, it can be understood that the first and second images to be processed are taken from the same visual angle.

[0041] S120. Determine the highlight center point of the first image to be processed, and determine the material information to be fused based on the highlight center point and the first image to be processed.

[0042] In this embodiment, the first image to be processed is used to determine the material information of the subject to be identified. For example, when the subject to be identified is a lipstick, and its corresponding first image to be processed is obtained, the image can be processed to obtain the material information of the lipstick to be fused. For the subject to be identified, the material information to be fused is the material parameters to be fused, and the specific parameters include, but are not limited to, various visual attributes such as texture, smoothness, transparency, reflectivity, refractive index, and luminosity.

[0043] Meanwhile, since the first image to be processed was captured with the flash on, before determining the material information to be fused for the subject to be identified, it is first necessary to determine the highlight portion of the first image to be processed. When the subject to be identified is coated on the target sphere, the highlight portion is the part of the target sphere that directly reflects the light source, and it is also the brightest part in the presented visual effect. Furthermore, the brightest point in the highlight portion is the highlight center point.

[0044] Optionally, the first image to be processed is divided into at least one sub-region to be processed based on a preset region size; the brightness value corresponding to each sub-region to be processed is determined, and the sub-region to be processed with the highest brightness value is taken as the target sub-region; the center point of the target sub-region is taken as the highlight center point.

[0045] The sub-region to be processed is a portion of the image that has been divided out, and the preset region size represents the size of the sub-region to be processed. Specifically, in order to determine the highlight portion in the first image to be processed, the region can be divided into at least one sub-region to be processed according to the preset region adjustment size, with a step size of one pixel.

[0046] For example, after obtaining a first image to be processed at a specific resolution, a fixed-size frame can be generated for dividing the image. Based on this frame, the first image to be processed can be divided from the leftmost side, and the resulting area is the first sub-region to be processed. Further, when the region adjustment step is one pixel, the frame is moved one pixel to the right, and the image is divided based on the current location of the frame to obtain the second sub-region to be processed. This process continues, meaning that when the frame moves to the rightmost side of the first image to be processed, all sub-regions corresponding to that image can be divided. Those skilled in the art should understand that in practical applications, the preset region size and region adjustment step can be set according to requirements, and this embodiment does not impose specific limitations.

[0047] Furthermore, after obtaining the sub-regions corresponding to the first image to be processed, in order to determine the highlights of the image, it is also necessary to identify the region with the highest brightness in each region. Specifically, the weight value corresponding to each pixel in each sub-region to be processed is determined based on the Poisson distribution; based on the weight value corresponding to each pixel and the corresponding pixel brightness value, the region brightness value of each sub-region to be processed is determined; the sub-region to be processed corresponding to the region with the highest brightness value is taken as the target sub-region.

[0048] The Poisson distribution, as a discrete probability distribution, is suitable for describing the number of random events occurring per unit time. In this embodiment, the Poisson distribution can be used to determine the weight value of each pixel in each sub-region to be processed. Simultaneously, since the first image to be processed has a brightness value, each pixel in each sub-region also has a corresponding brightness value. After determining the weight value and brightness value of each pixel in each sub-region to be processed, the regional brightness value of the area to which these pixels belong can be calculated. It can be understood that a higher regional brightness value indicates the presence of a brighter portion within the sub-region to be processed, while a lower regional brightness value indicates the absence of a brighter portion within the sub-region to be processed. Finally, the region with the highest regional brightness value is the target sub-region containing the highlight center point.

[0049] For example, after obtaining the first image of the target sphere coated with lipstick and dividing the image into ten sub-regions, the weight value and brightness value of each pixel in each region can be determined based on the Poisson distribution, thereby calculating the regional brightness value of each region. Furthermore, after comparing the brightness values ​​of the ten regions, the region with the highest brightness value can be selected as the target sub-region. It can be understood that the highlight center of the target sphere coated with lipstick exists in the target sub-region.

[0050] In this embodiment, after determining the target sub-region, the center point of the region can be further taken as the highlight center point, which is the pixel with the highest brightness value in the first image to be processed. For example, when the determined target sub-region is a square, the highlight center point is the pixel corresponding to the intersection of the diagonals of the square; when the determined target sub-region is a circle, the highlight center point is the pixel corresponding to the center of the circle. Those skilled in the art should understand that the target sub-region can be various regular or irregular shapes, and the center point of the region can be determined based on corresponding calculation formulas, which will not be elaborated further in this embodiment.

[0051] In this embodiment, after identifying the target highlight point in the first image to be processed, the material parameters to be fused corresponding to the object to be identified can be further determined. Optionally, the target normal map of the first image to be processed is determined; the target normal map and highlight point information are processed based on the pre-trained parameter generation model to obtain the material parameters to be fused.

[0052] The target normal map can be understood as a normal map, that is, a normal is drawn at every point on the uneven surface of the original object, and the direction of the normal is marked by the RGB color channel. It can be understood as another different surface that is not parallel to the original uneven surface. In practical applications, by determining the target normal map of the first image to be processed, surfaces with low detail can be made to present accurate lighting directions and reflection effects with high detail. For example, after baking a normal map of a high-detail model, it can be applied to the normal map channel of a low-detail model in time, so that its surface can also have a rendering effect of light and shadow distribution. At the same time, using normal maps reduces the area and computation required in the rendering process of representing objects, thereby optimizing the rendering effect.

[0053] In this embodiment, the parameter generation model is a deep learning algorithm-based model used to determine the material parameters to be fused for the subject to be identified. It can be understood that its inputs are the target normal map and highlight information, and its output is the corresponding material parameters to be fused. For example, after obtaining a first image of a target sphere coated with a certain lipstick, and obtaining the target normal map and highlight information of the image, the parameter generation model can process these two types of data to obtain specific values ​​for various visual attributes of the lipstick, such as texture, smoothness, transparency, reflectivity, refractive index, and luminosity. It should be noted that after obtaining the material parameters to be fused for the subject to be identified, this information can be stored in a specific database, and then called upon in subsequent determinations of target material information and image rendering processes, avoiding the waste of computational resources caused by repeatedly determining the material parameters to be fused.

[0054] S130. Determine color information based on the second image to be processed.

[0055] In this embodiment, the second image to be processed is obtained by taking a picture of the object to be identified without a flash. Since there are no highlights in the image, the color of the object to be identified can be accurately reflected. Therefore, it can be understood that the second image to be processed is at least used to determine the color information of the object to be identified, for example, to determine the actual color of the lipstick that is the subject to be identified.

[0056] In this embodiment, to further improve the accuracy of the color information of the identified object, the color information of the subject to be identified can be determined based on the RGB values ​​of the pixels of the subject in the second image to be processed. Specifically, the second image to be processed can first be loaded into relevant image processing software, and the objects in the second image to be processed can be identified based on the software's built-in functions or pre-written program code. Once the subject to be identified is determined, the RGB values ​​of each pixel corresponding to the subject can be further read to obtain the color information of the subject. It can be understood that the color information obtained in this way is a set of RGB values ​​of multiple pixels.

[0057] For example, after obtaining the second image of the target sphere coated with lipstick, the image can be imported into relevant image processing software. Once the software recognizes the target sphere based on its built-in functions, it can further determine the RGB value of each pixel corresponding to the target sphere, thereby obtaining the color information of the lipstick, that is, determining the color of the lipstick presented in the second image to be processed.

[0058] S140. Determine the target material information of the subject to be identified based on the material information and color information to be fused.

[0059] The target material information refers to the multiple visual attributes of the subject to be identified, including not only attribute values ​​such as texture, smoothness, transparency, reflectivity, refractive index, and luminosity, but also the color information corresponding to the subject. It can be understood that since the material information and color information of the subject to be identified are determined separately from the two images in a decoupled manner, it is necessary to fuse these two types of information to obtain the target material information of the subject.

[0060] For example, after obtaining the material information to be fused and the RGB value of the lipstick based on the first and second images to be processed of the target sphere coated with lipstick, the two types of information can be integrated to obtain the target material information of the lipstick. It can be understood that after setting or adjusting the parameters of the relevant image processing software according to the target material information, the lipstick can be rendered and displayed in the display interface.

[0061] In this embodiment, after determining the target material information of the subject to be identified, the object corresponding to the subject to be identified can be displayed on the target display interface, and the object corresponding to the subject to be identified can be used as the test object.

[0062] Among them, the object corresponding to the subject to be identified is the object carrying a certain material, such as a lipstick or a can of paint, and the target display interface is the interface associated with the relevant application, such as the interface associated with beauty-related augmented reality (AR) applications, AR shopping applications, AR home decoration applications, and 3D cloud rendering applications.

[0063] Furthermore, when displaying the object in the target display interface, a 3D model or corresponding image of the object can be shown, thus making the object a trial object in a computer-constructed virtual scene. It can be understood that when a user's trigger operation on the image or 3D model of the trial object is detected, the trial of the object can be realized. The following uses a beauty scene as an example to explain the processing after triggering the trial object.

[0064] In this embodiment, when a command to use a trial object is detected, the facial image corresponding to the target object is obtained, and the target material information corresponding to the triggered target trial object is retrieved; based on the facial image and the target material information, the target rendering material information is determined; based on the target rendering material information, the target effect is added to the facial image to obtain the target effect image.

[0065] For example, when the object to be tried is a lipstick, the corresponding 3D model of the lipstick will be displayed in the lipstick product list of the beauty AR application. When the user's click or touch operation on the lipstick is detected, the user receives the instruction to try the object, indicating that the user needs to try the lipstick in the virtual scene.

[0066] Furthermore, in order to display the visual effect of applying the lipstick in the application interface, a specific image or model needs to be selected as the basis for applying the lipstick. Therefore, it can be understood that after detecting the instruction to try the object, it is also necessary to obtain the user's facial image, and then use that image as the basis for applying the lipstick.

[0067] Specifically, when a command to try out an object is detected, the application can use the mobile device's camera to capture the user's facial image, or guide the user to actively upload a facial image in a specific way. Simultaneously, during the process of acquiring the user's facial image, it also needs to retrieve the target material information corresponding to the lipstick to determine the target rendering material information. This target rendering material information is the information obtained after processing the parts of the facial image corresponding to the object to be tried out; for example, based on the target material information corresponding to the lipstick, the image obtained after rendering the lips area in the user's facial image.

[0068] In this embodiment, after obtaining the target rendering material information based on the facial image and target material information, it can be superimposed on the facial image to obtain the target special effects image. It can be understood that when the object to be tried is the lipstick in the above example, the target special effects image can present the visual effect of applying lipstick to the user's lips.

[0069] Compared to the traditional method of repeatedly debugging between material creation and special effects construction, this embodiment directly calls the target material information to render the corresponding parts in the image when acquiring the user's facial image, realizing a visual preview of the beauty effects. This not only reduces the threshold and cost of generating special effects, but also enhances the convenience of applying special effects in beauty, cloud rendering, and AR shopping scenarios.

[0070] It should also be noted that, in the process of determining the target rendering material information, the illumination to be used can also be determined based on the facial image; the target material information can be adjusted according to the illumination to be used to obtain the target rendering material information.

[0071] The illumination to be used can be the brightness corresponding to the facial image. In this embodiment, in order to make the final target effect image more natural, it is also necessary to adjust the target material information according to the illumination to be used. This can be understood as adapting the brightness value of the subject to be used to the brightness value of the facial image, so that the target effect image as a whole presents a more natural and realistic visual effect after rendering.

[0072] Furthermore, by adjusting the target material information based on the illumination level to be used, and then processing specific areas of the facial image, the target rendering material information can be obtained. For example, if the user-uploaded facial image is dark, after determining the image's brightness value, the target material information for the lipstick can be adjusted based on this brightness value. Then, the lips in the facial image can be processed based on the adjusted target material information to obtain the corresponding target rendering material information. It can be understood that in the final image constructed based on the target rendering material information, the lipstick will also appear darker, thus making the simulated facial image after applying the lipstick more natural and realistic.

[0073] Those skilled in the art should understand that, in practical applications, the parameters in the target material information can also be adjusted manually, thereby realizing the visual parameter adjustment of the subject to be identified, further enhancing the flexibility of applying the solution of this embodiment in the above-mentioned various scenarios.

[0074] The technical solution of this embodiment acquires a first image to be processed and a second image to be processed, including the subject to be identified, from the same visual angle. That is, it acquires images of the subject to be identified under conditions of flash on and flash off. It determines the highlight center point of the first image to be processed, and determines the material information to be fused based on the highlight center point and the first image to be processed. It also determines the color information based on the second image to be processed. Furthermore, it determines the target material information of the subject to be identified based on the material information to be fused and the color information. The material information of a specific subject can be conveniently determined using only two images, which improves the efficiency of determining the material information. At the same time, the material information to be fused and the color information are determined separately in a decoupled manner, and the target material information is constructed based on the two types of information, which improves the accuracy of the final material information. This results in a better rendered image effect, making it easier to accurately display the visual effect of the subject to be identified to the user.

[0075] Example 2

[0076] Figure 2 This is a structural block diagram of a main body material determination device provided in Embodiment 2 of this disclosure. It can execute the main body material determination method provided in any embodiment of this disclosure, and possesses the corresponding functional modules and beneficial effects of the method. For example... Figure 2 As shown, the device specifically includes: an image acquisition module 210, a material information determination module 220, a color information determination module 230, and a target material information determination module 240.

[0077] The image acquisition module 210 is used to acquire a first image to be processed and a second image to be processed that includes the subject to be identified from the same visual angle; wherein the first image to be processed is determined based on the condition that the flash is turned on.

[0078] The material information determination module 220 is used to determine the highlight center point of the first image to be processed, and to determine the material information to be fused based on the highlight center point and the first image to be processed.

[0079] The color information determination module 230 is used to determine color information based on the second image to be processed.

[0080] The target material information determination module 240 is used to determine the target material information of the subject to be identified based on the material information to be fused and the color information.

[0081] Based on the above technical solutions, the image acquisition module 210 includes a first image determination unit and a second image determination unit.

[0082] The first image determination unit is used to activate the flash to capture a first image of the subject to be identified coated onto the target sphere when the trigger material determination control is detected.

[0083] The second image determination unit is used to capture a second image of the subject to be identified coated onto the target sphere under the condition that the flash is turned off.

[0084] Based on the above technical solutions, the first image to be processed and the second image to be processed are taken from the same visual angle.

[0085] Based on the above technical solutions, the material information determination module 220 includes a sub-region division unit, a target sub-region determination unit, and a highlight center point determination unit.

[0086] The sub-region division unit is used to divide the first image to be processed into at least one sub-region based on a preset region size.

[0087] The target sub-region determination unit is used to determine the brightness value corresponding to each sub-region to be processed, and to take the sub-region to be processed with the highest brightness value as the target sub-region.

[0088] The highlight center point determination unit is used to determine the center point of the target sub-region as the highlight center point.

[0089] Optionally, the sub-region division unit is further configured to divide the first region to be processed into at least one sub-region to be processed according to the preset region adjustment size, with a region adjustment step of one pixel.

[0090] Optionally, the target sub-region determination unit is further configured to determine the weight value corresponding to each pixel in each sub-region to be processed based on the Poisson distribution; determine the region brightness value of each sub-region to be processed based on the weight value corresponding to each pixel and the corresponding pixel brightness value; and take the sub-region to be processed corresponding to the highest region brightness value as the target sub-region.

[0091] Based on the above technical solutions, the material information determination module 220 also includes a target normal map determination unit and a material parameter determination unit.

[0092] The target normal map determination unit is used to determine the target normal map of the first image to be processed.

[0093] The material parameter determination unit is used to process the target normal map and the specular information based on the pre-trained parameter generation model to obtain the material parameters to be fused.

[0094] Optionally, the color information determination module 230 is further configured to determine the color information of the subject to be identified based on the RGB values ​​of the subject pixels in the second image to be processed.

[0095] Based on the above technical solutions, the main material determination device also includes a display module and a target special effects image determination module.

[0096] The display module is used to display the object corresponding to the subject to be identified on the target display interface, and to use the object corresponding to the subject to be identified as the object to be tested.

[0097] The target effect image determination module is used to obtain the facial image corresponding to the target object and retrieve the target material information corresponding to the triggered target trial object when a trial object instruction is detected; determine the target rendering material information based on the facial image and the target material information; and add target effects to the facial image based on the target rendering material information to obtain the target effect image.

[0098] Optionally, the target effect image determination module is further configured to determine the illumination to be used based on the facial image; and adjust the target material information according to the illumination to be used to obtain the target rendering material information.

[0099] The technical solution provided in this embodiment acquires a first image to be processed and a second image to be processed, including the subject to be identified, from the same visual angle. That is, it acquires images of the subject to be identified under conditions where the flash is on and off. It determines the highlight center point of the first image to be processed, and determines the material information to be fused based on the highlight center point and the first image to be processed. It also determines the color information based on the second image to be processed. Furthermore, it determines the target material information of the subject to be identified based on the material information to be fused and the color information. The material information of a specific subject can be conveniently determined using only two images, which improves the efficiency of determining material information. At the same time, the material information to be fused and the color information are determined separately in a decoupled manner, and the target material information is constructed based on the two types of information, which improves the accuracy of the final material information. This results in a better rendered image effect, making it easier to accurately display the visual effect of the subject to be identified to the user.

[0100] The main material determination device provided in this disclosure can execute the main material determination method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of executing the method.

[0101] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0102] Example 3

[0103] Figure 3 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of this disclosure. Refer to the following... Figure 3 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 3 The diagram below shows the structure of the terminal device or server 300. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0104] like Figure 3 As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, pattern processor, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 306 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An edit / output (I / O) interface 305 is also connected to the bus 304.

[0105] Typically, the following devices can be connected to I / O interface 305: editing devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0106] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 306, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined in the methods of embodiments of this disclosure.

[0107] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0108] The electronic device provided in this embodiment and the method for determining the main material provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0109] Example 4

[0110] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the main material determination method provided in the above embodiments.

[0111] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0112] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0113] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0114] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to:

[0115] Acquire a first image to be processed and a second image to be processed, both containing the subject to be identified, from the same visual angle; wherein the first image to be processed is determined based on the condition that the flash is on;

[0116] Determine the highlight center point of the first image to be processed, and determine the material information to be fused based on the highlight center point and the first image to be processed;

[0117] Based on the second image to be processed, determine the color information;

[0118] Based on the material information to be fused and the color information, the target material information of the subject to be identified is determined.

[0119] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0120] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0121] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0122] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0123] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0124] According to one or more embodiments of this disclosure, [Example 1] provides a method for determining the material of a main body, the method comprising:

[0125] Acquire a first image to be processed and a second image to be processed, both containing the subject to be identified, from the same visual angle; wherein the first image to be processed is determined based on the condition that the flash is on;

[0126] Determine the highlight center point of the first image to be processed, and determine the material information to be fused based on the highlight center point and the first image to be processed;

[0127] Based on the second image to be processed, determine the color information;

[0128] Based on the material information to be fused and the color information, the target material information of the subject to be identified is determined.

[0129] According to one or more embodiments of this disclosure, [Example 2] provides a method for determining the material of a main body, further comprising:

[0130] Optionally, upon detecting the trigger material determination control, the flash is activated to capture the first image to be processed, in which the subject to be identified is coated onto the target sphere;

[0131] With the flash off, capture a second image of the subject to be identified coated onto the target sphere;

[0132] The first image to be processed and the second image to be processed were taken from the same visual angle.

[0133] According to one or more embodiments of this disclosure, [Example 3] provides a method for determining the material of a main body, which further includes:

[0134] Optionally, the first image to be processed is divided into at least one sub-region to be processed based on a preset region size;

[0135] Determine the brightness value corresponding to each sub-region to be processed, and take the sub-region with the highest brightness value as the target sub-region;

[0136] The center point of the target sub-region is taken as the highlight center point.

[0137] According to one or more embodiments of this disclosure, [Example 4] provides a method for determining the material of a main body, which further includes:

[0138] Optionally, the first region to be processed can be divided into at least one sub-region to be processed according to the preset region adjustment size by using a step size of one pixel.

[0139] According to one or more embodiments of this disclosure, [Example 5] provides a method for determining the material of a main body, further comprising:

[0140] Optionally, the weight value corresponding to each pixel in each sub-region to be processed can be determined based on the Poisson distribution;

[0141] Based on the weight value and corresponding pixel brightness value of each pixel, the region brightness value of each sub-region to be processed is determined.

[0142] The sub-region to be processed corresponding to the region with the highest brightness value is taken as the target sub-region.

[0143] According to one or more embodiments of this disclosure, [Example Six] provides a method for determining the material of a main body, further comprising:

[0144] Optionally, the target normal map of the first image to be processed is determined;

[0145] The target normal map and the specular information are processed based on the pre-trained parameter generation model to obtain the material parameters to be fused.

[0146] According to one or more embodiments of this disclosure, [Example Seven] provides a method for determining the material of a main body, further comprising:

[0147] Optionally, the color information of the subject to be identified is determined based on the RGB values ​​of the subject pixels in the second image to be processed.

[0148] According to one or more embodiments of this disclosure, [Example Eight] provides a method for determining the material of a main body, further comprising:

[0149] Optionally, the object corresponding to the subject to be identified is displayed on the target display interface, and the object corresponding to the subject to be identified is used as the test object.

[0150] According to one or more embodiments of this disclosure, [Example Nine] provides a method for determining the material of a main body, further comprising:

[0151] Optionally, when a command to use a trial object is detected, the facial image corresponding to the target object is obtained, and the target material information corresponding to the target trial object that was triggered is retrieved.

[0152] Based on the facial image and the target material information, determine the target rendering material information;

[0153] Based on the target rendering material information, target effects are added to the facial image to obtain a target effect image.

[0154] According to one or more embodiments of this disclosure, [Example 10] provides a method for determining the material of a main body, further comprising:

[0155] Optionally, the illumination level to be used can be determined based on the facial image;

[0156] The target material information is obtained by adjusting the illumination of the light to be used.

[0157] According to one or more embodiments of this disclosure, [Example 11] provides a body material determining device, comprising:

[0158] The image acquisition module is used to acquire a first image to be processed and a second image to be processed, which include the subject to be identified, from the same visual angle; wherein the first image to be processed is determined based on the condition that the flash is on.

[0159] The material information to be fused module is used to determine the highlight center point of the first image to be processed, and to determine the material information to be fused based on the highlight center point and the first image to be processed;

[0160] The color information determination module is used to determine color information based on the second image to be processed;

[0161] The target material information determination module is used to determine the target material information of the subject to be identified based on the material information to be fused and the color information.

[0162] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0163] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0164] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for determining the main material, characterized in that, include: Acquire a first image to be processed and a second image to be processed, both containing the subject to be identified, from the same visual angle; wherein the first image to be processed is determined based on the condition that the flash is on; Determine the highlight center point of the first image to be processed, and determine the material information to be fused based on the highlight center point and the first image to be processed. The highlight center point is the pixel with the highest brightness value in the first image to be processed. Based on the second image to be processed, determine the color information; Based on the material information to be fused and the color information, the target material information of the subject to be identified is determined.

2. The method according to claim 1, characterized in that, The acquisition of a first image to be processed and a second image to be processed, including the subject to be identified, from the same visual angle includes: When the trigger material determination control is detected, the flash is activated to capture the first image to be processed, showing the subject to be identified coated onto the target sphere. With the flash off, capture a second image of the subject to be identified coated onto the target sphere; The first image to be processed and the second image to be processed were taken from the same visual angle.

3. The method according to claim 1, characterized in that, Determining the highlight center point of the first image to be processed includes: The first image to be processed is divided into at least one sub-region to be processed based on a preset region size; Determine the brightness value corresponding to each sub-region to be processed, and take the sub-region with the highest brightness value as the target sub-region; The center point of the target sub-region is taken as the highlight center point.

4. The method according to claim 3, characterized in that The step of dividing the first image to be processed into at least one sub-region to be processed based on a preset region size includes: The first region to be processed is divided into at least one sub-region to be processed according to the preset region adjustment size by adjusting the step size using a pixel as the region.

5. The method according to claim 3, characterized in that, The step of determining the brightness value corresponding to each sub-region to be processed, and taking the sub-region with the highest brightness value as the target sub-region, includes: The weight value corresponding to each pixel in each sub-region to be processed is determined based on the Poisson distribution. Based on the weight value and corresponding pixel brightness value of each pixel, the region brightness value of each sub-region to be processed is determined. The sub-region to be processed corresponding to the region with the highest brightness value is taken as the target sub-region.

6. The method according to claim 1, characterized in that, The step of determining the material information to be fused based on the highlight center point and the first image to be processed includes: Determine the target normal map of the first image to be processed; The target normal map and the specular information are processed based on the pre-trained parameter generation model to obtain the material parameters to be fused.

7. The method according to claim 1, characterized in that, The step of determining color information based on the second image to be processed includes: The color information of the subject to be identified is determined based on the RGB values ​​of the subject pixels in the second image to be processed.

8. The method according to claim 1, characterized in that, Also includes: The object corresponding to the subject to be identified is displayed on the target display interface, and the object corresponding to the subject to be identified is used as the test object.

9. The method according to claim 8, characterized in that, Also includes: When a command to use a trial object is detected, the facial image corresponding to the target object is obtained, and the target material information corresponding to the target trial object that was triggered is retrieved. Based on the facial image and the target material information, determine the target rendering material information; Based on the target rendering material information, target effects are added to the facial image to obtain a target effect image.

10. The method according to claim 9, characterized in that, The step of determining the target rendering material information based on the facial image and the target material information includes: The illumination level to be used is determined based on the facial image; The target material information is obtained by adjusting the illumination of the light to be used.

11. A device for determining the main material, characterized in that, include: The image acquisition module is used to acquire a first image to be processed and a second image to be processed, which include the subject to be identified, from the same visual angle; wherein the first image to be processed is determined based on the condition that the flash is on. The material information to be fused module is used to determine the highlight center point of the first image to be processed, and to determine the material information to be fused based on the highlight center point and the first image to be processed, wherein the highlight center point is the pixel with the highest brightness value in the first image to be processed. The color information determination module is used to determine color information based on the second image to be processed; The target material information determination module is used to determine the target material information of the subject to be identified based on the material information to be fused and the color information.

12. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the body material determination method as described in any one of claims 1-10.

13. A storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the body material determination method as described in any one of claims 1-10.

Images