Virtual fitting method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides a virtual fitting method and device, equipment, a storage medium and a program product, and relates to the technical field of image processing. The method comprises the following steps: in response to an input operation of a user, determining a base part and at least one customized part of a target garment; splicing the base part and each customized part to generate a clothing model of the target clothing; acquiring a user video of a user acquired by image acquisition equipment; obtaining a mask model; and according to each user image, the mask model and the clothing model, generating and displaying a fitting video of the user. According to the method provided by the invention, the fineness of the clothes in the user video is improved, so that the authenticity of virtual fitting of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a virtual fitting method, device, equipment, storage medium and program product. Background Art

[0002] Based on the e-commerce platform, in order to reduce the user's fitting costs, virtual fitting services are provided to users to show them the fitting effects.

[0003] In the prior art, an image synthesis algorithm is used to process images and clothing models uploaded by users to generate clothing pictures in a user-trimmed state.

[0004] However, in the above method, the clothing details in the generated clothing pictures are not well expressed, and the clothing folds are not realistic, resulting in an unrealistic fitting effect for users, thereby reducing the user experience. Summary of the invention

[0005] The embodiments of the present application provide a virtual fitting method, apparatus, device, storage medium and program product to solve the technical problem that the clothing details in the generated clothing pictures are not well presented, resulting in an unrealistic fitting effect for the user.

[0006] In a first aspect, an embodiment of the present application provides a virtual fitting method, comprising:

[0007] In response to the user's input operation, a base component and at least one customized component of the target garment are determined; wherein the base component is a three-dimensional model of a non-customized garment component in the target garment; and the customized component is a three-dimensional model of a customized garment component in the target garment;

[0008] The base component and each of the customized components are spliced ​​together to generate a clothing model of the target clothing; wherein the clothing model represents a three-dimensional model of the target clothing;

[0009] Acquire a user video of the user acquired by an image acquisition device; and acquire a mask model; wherein the user video includes a plurality of user images, and the user images represent the portrait posture of the user at each moment; and the mask model is a three-dimensional model of a preset transparent human body model;

[0010] A fitting video of the user is generated and displayed according to each of the user images, the mask model and the clothing model; wherein the fitting video represents a virtual fitting effect of the user trying on the target clothing.

[0011] In a possible implementation, the base component and each of the customized components are spliced ​​together to generate a clothing model of the target clothing, including:

[0012] Performing correction processing on the base component to obtain a corrected base component;

[0013] The modified base component and each of the customized components are spliced ​​to generate a clothing model of the target clothing.

[0014] In a possible implementation manner, performing a correction process on the base component to obtain a corrected base component includes:

[0015] Determine a projection line segment corresponding to each grid point in the base component; wherein the grid point is a grid vertex in a grid constituting the base component; the grid also includes a plurality of grid faces and a plurality of grid edges; the projection line segment is a projection line segment of a preset length in a projection ray of the grid point in a normal direction;

[0016] If it is determined that the projection line segment intersects with any mesh surface in the customized component, the mesh point where the projection line segment is located, and the mesh edges and mesh surfaces associated with the mesh point where the projection line segment is located are deleted to obtain the modified base component.

[0017] In a possible implementation, the modified base component and each of the customized components are spliced ​​to generate a clothing model of the target clothing, including:

[0018] Determine a plurality of sewing edges from the custom component; wherein the sewing edge is a first free edge that needs to be sewn and connected with other garment components; and the first free edge is a mesh edge that is connected to only one mesh face in the mesh constituting the custom component;

[0019] Determine a plurality of second free edges from the modified base component; wherein the second free edges are mesh edges with a connection number of 1 in the mesh constituting the modified base component;

[0020] The modified base component and each customized component are spliced ​​according to each stitching edge and each second free edge to obtain the clothing model.

[0021] In a possible implementation, the modified base component and each customized component are spliced ​​together according to each stitched edge and each second free edge to obtain the clothing model, including:

[0022] Determine the grid point associated with the stitching edge as the stitching point; and determine the neighbor point of the stitching point from each of the second free edges; wherein the neighbor point is the grid point closest to the stitching point among all the second free edges;

[0023] Generate a stitching connection edge according to the stitching point and neighboring points of the stitching point;

[0024] Generate a plurality of stitched mesh surfaces according to the stitched connecting edges and the stitched edges;

[0025] According to each of the stitched mesh surfaces, the modified base component and each of the customized components are spliced ​​to obtain the clothing model.

[0026] In a possible implementation, generating and displaying a fitting video of the user according to each of the user images, the mask model, and the clothing model includes:

[0027] According to the user image, the mask model is subjected to skeleton rotation optimization processing to obtain an optimized mask model corresponding to the user image; wherein the optimized mask model represents the posture of the transparent human body model at each moment;

[0028] Performing cloth simulation calculation processing on the clothing model to obtain a simulated clothing model;

[0029] Performing rigid body collision simulation calculation processing on the optimized mask model and the clothing model to obtain a first optimized mask model; and performing rigid body collision simulation calculation processing on the clothing model to obtain a second simulated clothing model;

[0030] According to the video data of the user, the first simulated clothing model, the second simulated clothing model and each of the optimized mask models are rendered to obtain and display the fitting video.

[0031] In a possible implementation, performing skeleton rotation optimization processing on the mask model according to the user image to obtain an optimized mask model corresponding to the user image includes:

[0032] Determine a plurality of portrait key points in the user image; and determine a plurality of mask key points in the mask model; wherein the portrait key points are body key points in the user portrait; the mask key points are body key points in a preset human body model; the mask key points correspond one-to-one to the portrait key points;

[0033] Determine the corresponding bone rotation parameters between the user image and the mask model according to each portrait key point in the user image and each mask key point;

[0034] According to the skeleton rotation parameters, the mask model is subjected to skeleton rotation optimization processing to obtain an optimized mask model corresponding to the user image.

[0035] In a possible implementation, determining the corresponding bone rotation parameters between the user image and the mask model according to each portrait key point in the user image and each mask key point includes:

[0036] Calculating and processing the two-dimensional coordinates and three-dimensional coordinates of the key points of the portrait to obtain the posture information of the image acquisition device; wherein the posture information represents the position and orientation of the image acquisition device in space;

[0037] According to the posture information, the three-dimensional coordinates of the portrait key points are transformed to obtain the transformed three-dimensional coordinates of the portrait key points;

[0038] Determine a distance set corresponding to the user image according to the transformed three-dimensional coordinates of each portrait key point in the user image and the three-dimensional coordinates of each mask key point; wherein the distance set includes the Euclidean distance between each portrait key point in the user image and the mask key point corresponding to the portrait key point;

[0039] According to the distance set, the corresponding bone rotation parameters between the user image and the mask model are determined.

[0040] In a second aspect, an embodiment of the present application provides a virtual fitting device, comprising:

[0041] A determination module, configured to determine, in response to an input operation of a user, a base component and at least one customized component of a target garment; wherein the base component is a three-dimensional model of a non-customized garment component in the target garment; and the customized component is a three-dimensional model of a customized garment component in the target garment;

[0042] A splicing module, used for splicing the base component and each of the customized components to generate a clothing model of the target clothing; wherein the clothing model represents a three-dimensional model of the target clothing;

[0043] An acquisition module, used to acquire a user video of the user acquired by an image acquisition device; and acquire a mask model; wherein the user video includes a plurality of user images, and the user images represent the portrait posture of the user at each moment; and the mask model is a three-dimensional model of a preset transparent human body model;

[0044] A generation module is used to generate and display a fitting video of the user according to each of the user images, the mask model and the clothing model; wherein the fitting video represents a virtual fitting effect of the user trying on the target clothing.

[0045] In a possible implementation manner, the splicing module is specifically used for:

[0046] Performing correction processing on the base component to obtain a corrected base component;

[0047] The modified base component and each of the customized components are spliced ​​to generate a clothing model of the target clothing.

[0048] In a possible implementation manner, the splicing module is specifically used for:

[0049] Determine a projection line segment corresponding to each grid point in the base component; wherein the grid point is a grid vertex in a grid constituting the base component; the grid also includes a plurality of grid faces and a plurality of grid edges; the projection line segment is a projection line segment of a preset length in a projection ray of the grid point in a normal direction;

[0050] If it is determined that the projection line segment intersects with any mesh surface in the customized component, the mesh point where the projection line segment is located, and the mesh edges and mesh surfaces associated with the mesh point where the projection line segment is located are deleted to obtain the modified base component.

[0051] In a possible implementation manner, the splicing module is further specifically used for:

[0052] Determine a plurality of sewing edges from the custom component; wherein the sewing edge is a first free edge that needs to be sewn and connected with other garment components; and the first free edge is a mesh edge that is connected to only one mesh face in the mesh constituting the custom component;

[0053] Determine a plurality of second free edges from the modified base component; wherein the second free edges are mesh edges with a connection number of 1 in the mesh constituting the modified base component;

[0054] The modified base component and each customized component are spliced ​​according to each stitching edge and each second free edge to obtain the clothing model.

[0055] In a possible implementation manner, the splicing module is further specifically used for:

[0056] Determine the grid point associated with the stitching edge as the stitching point; and determine the neighbor point of the stitching point from each of the second free edges; wherein the neighbor point is the grid point closest to the stitching point among all the second free edges;

[0057] Generate a stitching connection edge according to the stitching point and neighboring points of the stitching point;

[0058] Generate a plurality of stitched mesh surfaces according to the stitched connecting edges and the stitched edges;

[0059] According to each of the stitched mesh surfaces, the modified base component and each of the customized components are spliced ​​to obtain the clothing model.

[0060] In a possible implementation manner, the generating module is specifically used to:

[0061] According to the user image, the mask model is subjected to skeleton rotation optimization processing to obtain an optimized mask model corresponding to the user image; wherein the optimized mask model represents the posture of the transparent human body model at each moment;

[0062] Performing cloth simulation calculation processing on the clothing model to obtain a first simulated clothing model; and performing rigid body collision simulation calculation processing on the clothing model to obtain a second simulated clothing model;

[0063] According to the video data of the user, the first simulated clothing model, the second simulated clothing model and each of the optimized mask models are rendered to obtain and display the fitting video.

[0064] In a possible implementation manner, the generating module is specifically used to:

[0065] Determine a plurality of portrait key points in the user image; and determine a plurality of mask key points in the mask model; wherein the portrait key points are body key points in the user portrait; the mask key points are body key points in a preset human body model; the mask key points correspond one-to-one to the portrait key points;

[0066] Determine the corresponding bone rotation parameters between the user image and the mask model according to each portrait key point in the user image and each mask key point;

[0067] According to the skeleton rotation parameters, the mask model is subjected to skeleton rotation optimization processing to obtain an optimized mask model corresponding to the user image.

[0068] In a possible implementation manner, the generating module is specifically used to:

[0069] Calculating and processing the two-dimensional coordinates and three-dimensional coordinates of the key points of the portrait to obtain the posture information of the image acquisition device; wherein the posture information represents the position and orientation of the image acquisition device in space;

[0070] According to the posture information, the three-dimensional coordinates of the portrait key points are transformed to obtain the transformed three-dimensional coordinates of the portrait key points;

[0071] Determine a distance set corresponding to the user image according to the transformed three-dimensional coordinates of each portrait key point in the user image and the three-dimensional coordinates of each mask key point; wherein the distance set includes the Euclidean distance between each portrait key point in the user image and the mask key point corresponding to the portrait key point;

[0072] According to the distance set, the corresponding bone rotation parameters between the user image and the mask model are determined.

[0073] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0074] The memory stores computer-executable instructions;

[0075] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0076] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0077] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0078] The virtual fitting method, apparatus, device, storage medium and program product provided in the embodiments of the present application dynamically create a clothing model of the target clothing by splicing the base components and customized components selected by the user, and perform image processing in combination with the user video uploaded by the user and the mask model corresponding to the human body model to obtain a virtual fitting effect in the fitting video; furthermore, more details of the clothing can be seen through the generated fitting video, thereby increasing the realism of the virtual fitting effect and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0080] Figure 1 A schematic diagram of an application scenario provided for this application;

[0081] Figure 2 A schematic diagram of a two-dimensional virtual fitting process provided in an embodiment of the present application;

[0082] Figure 3 A flowchart of a virtual fitting method provided in an embodiment of the present application;

[0083] Figure 4 A schematic diagram of a shirt model provided in an embodiment of the present application;

[0084] Figure 5 A flowchart of another virtual fitting method provided in an embodiment of the present application;

[0085] Figure 6 A schematic diagram of a three-dimensional virtual fitting process provided in an embodiment of the present application;

[0086] Figure 7 A schematic diagram of the structure of a virtual fitting device provided in an embodiment of the present application;

[0087] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0088] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0089] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application.

[0090] First, the terms involved in this application are explained:

[0091] Mesh refers to a data structure used to represent three-dimensional geometric shapes. A mesh is usually composed of vertices, edges, and faces, forming a polygonal network, usually triangles or quadrilaterals.

[0092] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0093] It should be noted that the virtual fitting method, device, equipment, storage medium and program product of the present application can be used in the field of image processing technology, and can also be used in any field other than image processing technology. The application field of the virtual fitting method, device, equipment, storage medium and program product of the present application is not limited.

[0094] Figure 1 A schematic diagram of an application scenario provided by this application. Figure 1 As shown, the specific application scenario of the present application includes an electronic device 101 , a user 102 , and an image acquisition device 103 .

[0095] In one example, Figure 2 A schematic diagram of a two-dimensional virtual fitting process provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, user 102 can take and upload user images through image acquisition device 103, and based on electronic device 101, use image synthesis algorithm to process the preset clothing model and the image collected by image acquisition device 103 to generate clothing pictures of user 102 in the trying-on state.

[0096] However, in the above method, the clothing details in the generated clothing pictures are not well expressed, and the clothing folds are not realistic, resulting in an unrealistic fitting effect for users, thereby reducing the user experience.

[0097] In response to the above technical problems, the present application proposes the following technical concept: by splicing the base components selected by the user and the customized components, a clothing model of the target clothing is dynamically produced, and image processing is performed in combination with the user video uploaded by the user and the mask model corresponding to the human body model to obtain a virtual fitting effect in the fitting video. In this way, the problem that the clothing details in the generated clothing pictures are not well expressed, resulting in an unrealistic fitting effect for the user, which exists in the prior art, can be solved.

[0098] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0099] Figure 3 A flowchart of a virtual fitting method provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the method includes:

[0100] 201. In response to a user input operation, determine a base component and at least one customized component of a target garment; wherein the base component is a three-dimensional model of a non-customized garment component in the target garment; and the customized component is a three-dimensional model of a customized garment component in the target garment.

[0101] For example, the execution subject of this embodiment may be an electronic device, hereinafter referred to as a device, which may be a physical device or a virtual device that executes the virtual fitting method. For custom clothing, the selection of components is a key step. Figure 4 A schematic diagram of a shirt model provided in an embodiment of the present application, such as Figure 4 As shown, taking a shirt as an example, there are different styles of collars, plackets, cuffs and other customized parts that users can choose, while the front piece, shoulder, back piece and other parts are generally not customized. All non-customized parts are collectively referred to as base parts, and the model of a garment can be divided into a base part and multiple customized parts. When the user selects the target garment to be tried on, the device displays the 3D model of the non-customized garment parts in the target garment, namely the base part, and the 3D models of multiple customized garment parts, namely the customized parts; the user selects the part at each displayed part position through the device input operation, and when the user completes the selection, the base part and multiple customized parts in the target garment that the user wants to try on can be determined for processing.

[0102] 202. Perform splicing processing on the base component and each customized component to generate a clothing model of the target clothing; wherein the clothing model represents a three-dimensional model of the target clothing.

[0103] Exemplarily, the device uses image processing technology to splice the base components and customized components selected by the user to complete the dynamic production of the clothing model, thereby obtaining a three-dimensional model of the target clothing, i.e., the clothing model, and displays the clothing model to the user.

[0104] 203. Obtain a user video of the user captured by an image capture device; and obtain a mask model; wherein the user video includes a plurality of user images, and the user images represent the portrait posture of the user at each moment; and the mask model is a three-dimensional model of a preset transparent human body model.

[0105] Exemplarily, after the user completes the selection of trying on clothes, the user uses an image acquisition device, such as recording and uploading a user video including multiple user images. For example, the user uses a monocular color camera device (such as a camera attached to a mobile phone, tablet computer, computer, etc., or a video camera) to record a video containing a single portrait; the device will obtain the user video captured by the image acquisition device, wherein the user image represents the user's portrait posture at each moment; and the device will pre-make a three-dimensional model of a transparent blocking human body model that is close to the user's body contour, i.e., a mask model, to synchronize the user's movements.

[0106] 204. Generate and display a user's fitting video based on each user image, mask model, and clothing model; wherein the fitting video represents a virtual fitting effect of the user trying on the target clothing.

[0107] Exemplarily, the device uses image processing technology to perform image processing on the clothing model of the obtained target clothing, the mask model of the transparent human body model, and each user image in the user video, generates a user fitting video, and displays the fitting video to form a real-life virtual fitting effect of the user trying on the target clothing.

[0108] In the present embodiment, a virtual fitting method is provided, in which a clothing model of the target clothing is dynamically produced by splicing the base component selected by the user and the various customized components, and image processing is performed in combination with the user video uploaded by the user and the mask model corresponding to the human body model to obtain a virtual fitting effect in the fitting video; furthermore, more details of the clothing can be seen through the generated fitting video, thereby increasing the realism of the virtual fitting effect and improving the user experience.

[0109] Figure 5 A flowchart of another virtual fitting method provided in an embodiment of the present application is shown as follows: Figure 5 As shown, the method includes:

[0110] 301. In response to a user input operation, determine a base component and at least one customized component of a target garment; wherein the base component is a three-dimensional model of a non-customized garment component in the target garment; and the customized component is a three-dimensional model of a customized garment component in the target garment.

[0111] Exemplarily, this step may refer to step 201 and will not be described in detail here.

[0112] 302. Perform correction processing on the base component to obtain a corrected base component.

[0113] For example, Figure 6 A schematic diagram of a three-dimensional virtual fitting process provided in an embodiment of the present application, such as Figure 6 As shown, the user selects components at each component position. After the user completes the selection, the component splicing module of the device will correct the base component selected by the user based on image processing technology to obtain a corrected base component.

[0114] In one example, step 302 includes the following steps:

[0115] The first step of step 302 is to determine the projection line segment corresponding to each grid point in the base component; wherein the grid point is a grid vertex in the grid constituting the base component; the grid also includes multiple grid faces and multiple grid edges; the projection line segment is a projection line segment of a preset length in the projection ray of the grid point along the normal direction.

[0116] The second step of step 302 is to delete the grid point where the projection line segment is located, and the grid edges and grid faces associated with the grid point where the projection line segment is located, if it is determined that the projection line segment intersects with the grid face in any customized component, so as to obtain a modified base component.

[0117] Exemplarily, for the base component or each customized component, each component model is constructed by a grid, and each grid includes multiple grid vertices, i.e., grid points, multiple grid faces, and multiple grid edges. The component splicing module of the device is based on image processing technology. When correcting the base component, the projection ray of each grid point in the base component in the normal direction is first calculated, and a preset distance is taken from each projection ray as a projection line segment, that is, the projection line segment corresponding to each grid point in the base component is obtained; then, for each grid point in the base component, if it is determined that the projection line segment corresponding to the grid point intersects with the grid face in any customized component, the grid point and the grid edges and grid faces related to the grid vertex are deleted to complete the correction of the base component model and obtain the corrected base component.

[0118] 303. The corrected base component and each customized component are spliced ​​together to generate a clothing model of the target clothing.

[0119] Exemplarily, the component splicing module splices the corrected base component and all the customized components selected by the user based on the model splicing processing technology to complete the dynamic production of the clothing model of the target clothing, obtain the clothing model, and display the clothing model in the clothing model to the user. Furthermore, by correcting the base component, the model in the corrected base component can be guaranteed not to overlap with the model in the customized component, eliminating the model overlap problem generated during the model splicing process.

[0120] In one example, step 303 includes the following steps:

[0121] The first step of step 303 is to determine multiple stitching edges from the customized component; wherein the stitching edge is the first free edge that needs to be stitched and connected with other clothing components; the first free edge is a mesh edge that only connects one mesh face in the mesh constituting the customized component.

[0122] The second step of step 303 is to determine a plurality of second free edges from the modified base component; wherein the second free edges are mesh edges with a connection number of 1 in the mesh constituting the modified base component.

[0123] In the third step of step 303, the corrected base component and each customized component are spliced ​​according to each stitching edge and each second free edge to obtain a clothing model.

[0124] Exemplarily, based on the component splicing module, each mesh edge in each customized component is classified, and the mesh edges that are only connected to one mesh surface in the mesh constituting the customized component can be determined as free edges in each customized component, i.e., first free edges. For all first free edges, according to clothing production rules, the first free edges that need to be sewn and connected with other clothing components can be determined as sewn edges, and the remaining first free edges are non-sewn edges. Moreover, each mesh edge in the corrected base component is classified, and the mesh edges that are only connected to one mesh surface in the mesh constituting the corrected base component can be determined as free edges in the corrected base component, i.e., second free edges. Then, through the model splicing technology, according to the sewn edges and each second free edge in each customized component, the corrected base component and each customized component can be spliced ​​to complete the clothing model production and obtain the clothing model, thereby improving the fineness of the clothing in the user video, and thereby improving the authenticity of the user's virtual fitting.

[0125] In one example, the third step of step 303 includes:

[0126] Step 1: determine the grid point associated with the stitching edge as the stitching point; and determine the neighbor point of the stitching point from each second free edge; wherein the neighbor point is the grid point closest to the stitching point in all the second free edges.

[0127] Step 2: Generate stitching edges based on stitching points and their neighboring points.

[0128] Step 3: Generate multiple stitched mesh surfaces based on the stitched connection edges and the stitched edges.

[0129] Step 4: According to each stitched mesh surface, the corrected base component and each customized component are spliced ​​to obtain a clothing model.

[0130] Specifically, for the stitching edges in each customized component, the component splicing module determines the grid points associated with the stitching edges in each customized component as stitching points, and calculates the distance between each stitching point and each grid point in all second free edges, and determines the grid points in all second free edges that are closest to the stitching points as neighbor points of the stitching points; then, each stitching point and each stitching point's neighbor point form corresponding stitching grid point pairs, and for each stitching grid point pair, a corresponding new grid edge, i.e., a stitching connection edge, is generated; based on the image stitching processing technology, all stitching connection edges and all stitching edges are spliced ​​to obtain corresponding multiple stitching grid surfaces, and then the corrected base component and each customized component can be spliced ​​according to all the generated stitching grid surfaces to obtain a clothing model to achieve the component splicing effect.

[0131] 304. Obtain a user video of the user captured by an image capture device; and obtain a mask model; wherein the user video includes a plurality of user images, and the user images represent the portrait posture of the user at each moment; and the mask model is a three-dimensional image of a preset transparent human body model.

[0132] Exemplarily, this step may refer to step 203 and will not be described in detail here.

[0133] 305. Perform skeleton rotation optimization processing on the mask model according to the user image to obtain an optimized mask model corresponding to the user image; wherein the optimized mask model represents the posture of the transparent human body model at each moment.

[0134] For example, in combination Figure 6 , based on the motion capture module of the device, through the preset three-dimensional model processing technology, the motion posture of each user image is identified, and the motion is applied to the mask model, and then the mask model is optimized for skeletal rotation to obtain the posture of the transparent human body model corresponding to each user image at each moment, that is, the optimized mask model, that is, all user images in the user video, that is, the interface obtains the corresponding animation including multiple optimized mask models to synchronize the user's movements.

[0135] In one example, step 305 includes the following steps:

[0136] The first step of step 305 is to determine multiple portrait key points in the user image; and determine multiple mask key points in the mask model; wherein the portrait key points are body key points in the user portrait; the mask key points are body key points in the preset human body model; the mask key points correspond one-to-one to the portrait key points.

[0137] The second step of step 305 is to determine the corresponding bone rotation parameters between the user image and the mask model according to each portrait key point and each mask key point in the user image.

[0138] The third step of step 305 is to perform skeleton rotation optimization processing on the mask model according to the skeleton rotation parameters to obtain an optimized mask model corresponding to the user image.

[0139] For example, in combination Figure 6 In order to obtain the mask posture of the most matching user action, for each user image in the acquired user video, the motion capture module can identify all the body key points of the portrait in each user image, that is, the portrait key points, such as shoulders, elbows, wrists, hips, knees, ankles, etc. through a preset image recognition program; at the same time, the body key points corresponding to each portrait key point in the human body model in the mask model, that is, multiple mask key points, can be identified. Based on the preset algorithm in computer vision, the motion capture module calculates and processes each portrait key point in each user image and the corresponding multiple mask key points, and can determine the position and posture of the three-dimensional object, that is, the user, and then calculates the corresponding bone rotation parameters between each user image and the mask model; then, according to each bone rotation parameter, the human body model in the mask model can be optimized for bone rotation, and the optimized mask model corresponding to each user image can be obtained.

[0140] In one example, the second step of step 305 includes:

[0141] Step 1: Calculate and process the two-dimensional coordinates and three-dimensional coordinates of the key points of the portrait to obtain the posture information of the image acquisition device; wherein the posture information represents the position and orientation of the image acquisition device in space.

[0142] Step 2: According to the posture information, the three-dimensional coordinates of the portrait key points are transformed to obtain the transformed three-dimensional coordinates of the portrait key points.

[0143] Step 3: Determine a distance set corresponding to the user image based on the transformed three-dimensional coordinates of each portrait key point in the user image and the three-dimensional coordinates of each mask key point; wherein the distance set includes the Euclidean distance between each portrait key point in the user image and the mask key point corresponding to the portrait key point.

[0144] Step 4: Determine the corresponding bone rotation parameters between the user image and the mask model based on the distance set.

[0145] Specifically, for each identified portrait key point, each portrait key point has a two-dimensional coordinate in a preset image coordinate system and a three-dimensional coordinate in a preset human body coordinate system. The motion capture module can calculate and process each portrait key point and the corresponding multiple mask key points based on the preset computer vision algorithm, so as to determine the position and posture of the three-dimensional object, that is, the user, and then, the position and direction of the image acquisition device in space, that is, the posture information, such as the posture in the preset human body coordinate system, can be calculated; then, according to the posture information of the image acquisition device, the three-dimensional coordinates of each portrait key point can be converted, that is, the three-dimensional human body key point is converted from the human body coordinate system to the three-dimensional coordinates in the coordinate system of the image acquisition device, and the converted three-dimensional coordinates of each portrait key point are obtained. For each user image, the motion capture module can use a visual computing algorithm to perform Euclidean distance calculation on the converted three-dimensional coordinates of each portrait key point in each user image and the three-dimensional coordinates of each corresponding mask key point, so as to obtain the Euclidean distance between each portrait key point in each user image and the mask key point corresponding to the portrait key point, that is, to obtain the corresponding distance set in each user image; then, based on the corresponding distance set in each user image, with the goal of minimizing each Euclidean distance, the optimal bone rotation parameters corresponding to each user image and the mask model are obtained; further, under the limit constraints of the human joint angle, the rotation of each bone of the mask can be optimized to obtain the posture of the human body model that best matches the user's action.

[0146] 306. Perform cloth simulation calculation processing on the clothing model to obtain a clothing model after a first simulation; at the same time, perform rigid body collision simulation calculation processing on the clothing model to obtain a clothing model after a second simulation.

[0147] For example, in combination Figure 6 ,In order to achieve the effect that clothes follow the user’s movement, the use of physical ,simulation calculation can achieve the strongest sense of reality. ,Based on the rendering module of the device, the cloth physics simulation method of the ,mass point-spring model is used to perform cloth physics simulation ,computation on the clothing model to obtain the simulated clothing model, that is, the first simulated clothing model, to ,show the clothing details of the target clothing.

[0148] At the same time, based on the rendering module, the clothing model is subjected to rigid body collision simulation calculation processing through the collision physics simulation operation method to obtain a simulated clothing model, that is, a second simulated clothing model, so as to achieve a dynamic human body dressing effect close to the real one.

[0149] 307. Render the first simulated clothing model, the second simulated clothing model, and each optimized mask model according to the user's video data to obtain and display a fitting video.

[0150] For example, in combination Figure 6 Based on the rendering module, the three-dimensional animation composed of the first simulated clothing model, the second simulated clothing model and all the optimized mask models is rendered, and each corresponding user image in the user video is added as the background image during rendering, and a three-dimensional to two-dimensional conversion process is performed to obtain a fitting video and display it; because the model in the mask model is transparent, the effect of the user actually trying on clothes can be achieved.

[0151] In this embodiment, based on the above embodiment, physical simulation calculation is used and each user image in the user video is added as a background image during rendering to obtain the user's fitting video, which can improve the authenticity of the real-life virtual fitting effect and thereby improve the user experience.

[0152] Figure 7 A schematic diagram of the structure of a virtual fitting device provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the device comprises:

[0153] The determination module 401 is used to determine the base component and at least one customized component of the target garment in response to the user's input operation; wherein the base component is a three-dimensional model of a non-customized garment component in the target garment; and the customized component is a three-dimensional model of a customized garment component in the target garment;

[0154] A splicing module 402 is used to splice the base component and each customized component to generate a clothing model of the target clothing; wherein the clothing model represents a three-dimensional model of the target clothing;

[0155] The acquisition module 403 is used to acquire a user video of a user acquired by an image acquisition device; and acquire a mask model; wherein the user video includes a plurality of user images, and the user images represent the portrait posture of the user at each moment; and the mask model is a three-dimensional model of a preset transparent human body model;

[0156] The generation module 404 is used to generate and display a user's fitting video based on each user image, mask model and clothing model; wherein the fitting video represents a virtual fitting effect of the user trying on the target clothing.

[0157] In a possible implementation, the splicing module 402 is specifically used to: perform correction processing on the base component to obtain a corrected base component; and perform splicing processing on the corrected base component and each customized component to generate a clothing model of the target clothing.

[0158] In one possible implementation, the stitching module 402 is specifically used to: determine the projection line segment corresponding to each grid point in the base component; wherein the grid point is a grid vertex in the grid constituting the base component; the grid also includes multiple grid faces and multiple grid edges; the projection line segment is a projection line segment of a preset length in the projection ray of the grid point along the normal direction; if it is determined that the projection line segment intersects with a grid face in any custom component, then the grid point where the projection line segment is located, and the grid edges and grid faces associated with the grid point where the projection line segment is located are deleted to obtain a corrected base component.

[0159] In a possible implementation, the stitching module 402 is further specifically used to: determine a plurality of stitching edges from a customized component; wherein the stitching edge is a first free edge that needs to be stitched and connected with other clothing components; the first free edge is a mesh edge that is connected to only one mesh face in the mesh constituting the customized component; determine a plurality of second free edges from a corrected base component; wherein the second free edge is a mesh edge that is connected to 1 in the mesh constituting the corrected base component; and stitch the corrected base component and the customized components according to each stitching edge and each second free edge to obtain a clothing model.

[0160] In a possible implementation, the stitching module 402 is further specifically used to: determine the grid points associated with the stitching edges as stitching points; and determine the neighbor points of the stitching points from each second free edge; wherein the neighbor points are the grid points closest to the stitching points in all the second free edges; generate stitching connection edges according to the stitching points and the neighbor points of the stitching points; generate multiple stitching mesh surfaces according to each stitching connection edge and each stitching edge; and stitch the corrected base component and each customized component according to each stitching mesh surface to obtain a clothing model.

[0161] In one possible implementation, the generation module 404 is specifically used to: perform skeletal rotation optimization processing on the mask model according to the user image to obtain an optimized mask model corresponding to the user image; wherein the optimized mask model represents the posture of the transparent human body model at each moment; perform cloth simulation calculation processing on the clothing model to obtain a first simulated clothing model; at the same time, perform rigid body collision simulation calculation processing on the clothing model to obtain a second simulated clothing model; according to the user's video data, render the first simulated clothing model, the second simulated clothing model and each optimized mask model to obtain and display the fitting video.

[0162] In a possible implementation, the generation module 404 is specifically used to: determine multiple portrait key points in the user image; and determine multiple mask key points in the mask model; wherein the portrait key points are the body key points in the user portrait; the mask key points are the body key points in the preset human body model; the mask key points correspond one-to-one to the portrait key points; according to each portrait key point and each mask key point in the user image, determine the corresponding bone rotation parameters between the user image and the mask model; according to the bone rotation parameters, perform bone rotation optimization processing on the mask model to obtain an optimized mask model corresponding to the user image.

[0163] In one possible implementation, the generation module 404 is specifically used to: calculate and process the two-dimensional coordinates and three-dimensional coordinates of the portrait key points to obtain the posture information of the image acquisition device; wherein the posture information represents the position and orientation of the image acquisition device in space; according to the posture information, transform the three-dimensional coordinates of the portrait key points to obtain the transformed three-dimensional coordinates of the portrait key points; determine the distance set corresponding to the user image according to the transformed three-dimensional coordinates of each portrait key point in the user image and the three-dimensional coordinates of each mask key point; wherein the distance set includes the Euclidean distance between each portrait key point in the user image and the mask key point corresponding to the portrait key point; according to the distance set, determine the corresponding bone rotation parameters between the user image and the mask model.

[0164] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0165] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. Each module can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The functions of each module. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0166] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 8As shown, the electronic device includes: a memory 501, a processor 502; the memory 501; and a memory for storing instructions executable by the processor 502.

[0167] The processor 502 is configured to execute the method provided in the above embodiment.

[0168] The electronic device further includes a receiver 503 and a transmitter 504. The receiver 503 is used to receive instructions and data sent by other devices, and the transmitter 504 is used to send instructions and data to external devices.

[0169] The specific implementation process of the processor can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0170] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0171] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0172] An embodiment of the present application also provides a chip for executing instructions, which is used to execute the technical solution of the processing method in the above embodiment.

[0173] The embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed on a computer, the computer executes the technical solution of the processing method of the above embodiment.

[0174] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0175] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0176] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the processing method in the above embodiment can be implemented.

[0177] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0178] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0180] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0181] Those skilled in the art will appreciate that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed.

[0182] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A virtual fitting method, characterized in that: include: In response to the user's input operation, a base component and at least one customized component of the target garment are determined; wherein the base component is a three-dimensional model of a non-customized garment component in the target garment; and the customized component is a three-dimensional model of a customized garment component in the target garment; The base component and each of the customized components are spliced ​​together to generate a clothing model of the target clothing; wherein the clothing model represents a three-dimensional model of the target clothing; Acquire a user video of the user acquired by an image acquisition device; and acquire a mask model; wherein the user video includes a plurality of user images, and the user images represent the portrait posture of the user at each moment; and the mask model is a three-dimensional model of a preset transparent human body model; A fitting video of the user is generated and displayed according to each of the user images, the mask model and the clothing model; wherein the fitting video represents a virtual fitting effect of the user trying on the target clothing.

2. The method according to claim 1, characterized in that The base component and each of the customized components are spliced ​​together to generate a clothing model of the target clothing, including: Performing correction processing on the base component to obtain a corrected base component; The modified base component and each of the customized components are spliced ​​to generate a clothing model of the target clothing.

3. The method according to claim 2, characterized in that Performing correction processing on the base component to obtain a corrected base component includes: Determine a projection line segment corresponding to each grid point in the base component; wherein the grid point is a grid vertex in a grid constituting the base component; the grid also includes a plurality of grid faces and a plurality of grid edges; the projection line segment is a projection line segment of a preset length in a projection ray of the grid point in a normal direction; If it is determined that the projection line segment intersects with any mesh surface in the customized component, the mesh point where the projection line segment is located, and the mesh edges and mesh surfaces associated with the mesh point where the projection line segment is located are deleted to obtain the modified base component.

4. The method according to claim 2, characterized in that: The modified base component and each of the customized components are spliced ​​together to generate a clothing model of the target clothing, including: Determine a plurality of sewing edges from the custom component; wherein the sewing edge is a first free edge that needs to be sewn and connected with other garment components; and the first free edge is a mesh edge that is connected to only one mesh face in the mesh constituting the custom component; Determine a plurality of second free edges from the modified base component; wherein the second free edges are mesh edges with a connection number of 1 in the mesh constituting the modified base component; The modified base component and each customized component are spliced ​​according to each stitching edge and each second free edge to obtain the clothing model.

5. The method according to claim 4, characterized in that According to each of the stitched edges and each of the second free edges, the modified base component and each of the customized components are spliced ​​to obtain the clothing model, including: Determine the grid point associated with the stitching edge as the stitching point; and determine the neighbor point of the stitching point from each of the second free edges; wherein the neighbor point is the grid point closest to the stitching point among all the second free edges; Generate a stitching connection edge according to the stitching point and neighboring points of the stitching point; Generate a plurality of stitched mesh surfaces according to the stitched connecting edges and the stitched edges; The modified base component and each customized component are spliced ​​according to each stitched mesh surface to obtain the clothing model.

6. The method according to any one of claims 1 to 5, characterized in that Generating and displaying a fitting video of the user according to each of the user images, the mask model, and the clothing model, including: According to the user image, the mask model is subjected to skeleton rotation optimization processing to obtain an optimized mask model corresponding to the user image; wherein the optimized mask model represents the posture of the transparent human body model at each moment; Performing cloth simulation calculation processing on the clothing model to obtain a first simulated clothing model; and performing rigid body collision simulation calculation processing on the clothing model to obtain a second simulated clothing model; According to the video data of the user, the first simulated clothing model, the second simulated clothing model and each of the optimized mask models are rendered to obtain and display the fitting video.

7. The method according to claim 6, characterized in that According to the user image, the mask model is subjected to skeleton rotation optimization processing to obtain an optimized mask model corresponding to the user image, including: Determine a plurality of portrait key points in the user image; and determine a plurality of mask key points in the mask model; wherein the portrait key points are body key points in the user portrait; the mask key points are body key points in a preset human body model; the mask key points correspond one-to-one to the portrait key points; Determine the corresponding bone rotation parameters between the user image and the mask model according to each portrait key point in the user image and each mask key point; According to the skeleton rotation parameters, the mask model is subjected to skeleton rotation optimization processing to obtain an optimized mask model corresponding to the user image.

8. The method according to claim 7, characterized in that Determining the corresponding bone rotation parameters between the user image and the mask model according to each portrait key point in the user image and each mask key point, including: Calculating and processing the two-dimensional coordinates and three-dimensional coordinates of the key points of the portrait to obtain the posture information of the image acquisition device; wherein the posture information represents the position and orientation of the image acquisition device in space; According to the posture information, the three-dimensional coordinates of the portrait key points are transformed to obtain the transformed three-dimensional coordinates of the portrait key points; Determine a distance set corresponding to the user image according to the transformed three-dimensional coordinates of each portrait key point in the user image and the three-dimensional coordinates of each mask key point; wherein the distance set includes the Euclidean distance between each portrait key point in the user image and the mask key point corresponding to the portrait key point; According to the distance set, the corresponding bone rotation parameters between the user image and the mask model are determined.

9. A virtual fitting device, characterized in that: include: A determination module, configured to determine, in response to an input operation of a user, a base component and at least one customized component of a target garment; wherein the base component is a three-dimensional model of a non-customized garment component in the target garment; and the customized component is a three-dimensional model of a customized garment component in the target garment; A splicing module, used for splicing the base component and each of the customized components to generate a clothing model of the target clothing; wherein the clothing model represents a three-dimensional model of the target clothing; An acquisition module is used to acquire a user video of the user acquired by an image acquisition device; and acquire a mask model; wherein the user video includes a plurality of user images, and the user images represent the portrait posture of the user at each moment; and the mask model is a three-dimensional model of a preset transparent human body model; A generation module is used to generate and display a fitting video of the user according to each of the user images, the mask model and the clothing model; wherein the fitting video represents a virtual fitting effect of the user trying on the target clothing.

10. The device according to claim 9, characterized in that The splicing module is specifically used for: Performing correction processing on the base component to obtain a corrected base component; The modified base component and each of the customized components are spliced ​​to generate a clothing model of the target clothing.

11. The device according to claim 10, characterized in that The splicing module is specifically used for: Determine a projection line segment corresponding to each grid point in the base component; wherein the grid point is a grid vertex in a grid constituting the base component; the grid also includes a plurality of grid faces and a plurality of grid edges; the projection line segment is a projection line segment of a preset length in a projection ray of the grid point in a normal direction; If it is determined that the projection line segment intersects with any mesh surface in the customized component, the mesh point where the projection line segment is located, and the mesh edges and mesh surfaces associated with the mesh point where the projection line segment is located are deleted to obtain the modified base component.

12. The device according to claim 10, characterized in that The splicing module is also specifically used for: Determine a plurality of sewing edges from the custom component; wherein the sewing edge is a first free edge that needs to be sewn and connected with other garment components; and the first free edge is a mesh edge that is connected to only one mesh face in the mesh constituting the custom component; Determine a plurality of second free edges from the modified base component; wherein the second free edges are mesh edges with a connection number of 1 in the mesh constituting the modified base component; The modified base component and each customized component are spliced ​​according to each stitching edge and each second free edge to obtain the clothing model.

13. The device according to claim 12, characterized in that The splicing module is also specifically used for: Determine the grid point associated with the stitching edge as the stitching point; and determine the neighbor point of the stitching point from each of the second free edges; wherein the neighbor point is the grid point closest to the stitching point among all the second free edges; Generate a stitching connection edge according to the stitching point and neighboring points of the stitching point; Generate a plurality of stitched mesh surfaces according to the stitched connecting edges and the stitched edges; The modified base component and each customized component are spliced ​​according to each stitched mesh surface to obtain the clothing model.

14. The device according to any one of claims 9 to 13, characterized in that The generation module is specifically used for: According to the user image, the mask model is subjected to skeleton rotation optimization processing to obtain an optimized mask model corresponding to the user image; wherein the optimized mask model represents the posture of the transparent human body model at each moment; Performing cloth simulation calculation processing on the clothing model to obtain a first simulated clothing model; and performing rigid body collision simulation calculation processing on the clothing model to obtain a second simulated clothing model; According to the video data of the user, the first simulated clothing model, the second simulated clothing model and each of the optimized mask models are rendered to obtain and display the fitting video.

15. The device according to claim 14, characterized in that The generation module is specifically used for: Determine a plurality of portrait key points in the user image; and determine a plurality of mask key points in the mask model; wherein the portrait key points are body key points in the user portrait; the mask key points are body key points in a preset human body model; the mask key points correspond one-to-one to the portrait key points; Determine the corresponding bone rotation parameters between the user image and the mask model according to each portrait key point in the user image and each mask key point; According to the skeleton rotation parameters, the mask model is subjected to skeleton rotation optimization processing to obtain an optimized mask model corresponding to the user image.

16. The device according to claim 15, characterized in that The generation module is specifically used for: Calculating and processing the two-dimensional coordinates and three-dimensional coordinates of the key points of the portrait to obtain the posture information of the image acquisition device; wherein the posture information represents the position and orientation of the image acquisition device in space; According to the posture information, the three-dimensional coordinates of the portrait key points are transformed to obtain the transformed three-dimensional coordinates of the portrait key points; Determine a distance set corresponding to the user image according to the transformed three-dimensional coordinates of each portrait key point in the user image and the three-dimensional coordinates of each mask key point; wherein the distance set includes the Euclidean distance between each portrait key point in the user image and the mask key point corresponding to the portrait key point; According to the distance set, the corresponding bone rotation parameters between the user image and the mask model are determined.

17. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by a processor.

19. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images