Image processing method, device, electronic device and storage medium

Abstract

This disclosure relates to an image processing method, apparatus, electronic device, and storage medium. The method includes: acquiring a source model and a target model for the same object; transforming the geometry of the source model according to the geometry of the target model to obtain a transformed model; baking the transformed model to obtain rendering information; and rendering an image containing the source model based on the rendering information to obtain a rendered image. This disclosure solves the problem in related technologies that cannot render an aesthetically pleasing 3D face model onto another face model.

Description

Technical Field

[0001] The present disclosure relates to the field of computers, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] With the popularity of mobile Internet and smart terminals (such as mobile phones), people are increasingly keen on using their mobile phones to take selfies to create and share beauty. Beauty makeup technology is a form of pursuit of beauty through mobile phones. It is essentially a computer graphics technology that first detects the face and calculates the face mesh, then renders the face mesh to obtain the beauty makeup effect, and finally uses an image blending scheme to superimpose the beauty makeup effect on the face image.

[0003] Generally speaking, an aesthetic makeup look mainly depends on the rendering effect of the 3D face model, and the rendering effect of the 3D face model is closely related to the 3D face model. It can be said that the 3D face model determines the final rendering effect. Therefore, to obtain an aesthetic rendering effect, you need to have an aesthetic 3D face model. After having an aesthetic 3D face model, there is currently no good solution for how to render the 3D face model onto the user's face model. Summary of the invention

[0004] The present disclosure provides an image processing method, device, electronic device and storage medium to at least solve the problem that the related art cannot render an aesthetically pleasing three-dimensional face model onto another face model.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, comprising: obtaining a source model and a target model for the same object; transforming the geometric structure of the source model according to the geometric structure of the target model to obtain a transformed model; baking the transformed model to obtain rendering information; and rendering an image containing the source model based on the rendering information to obtain a rendered image.

[0006] Optionally, the geometric structure of the source model is transformed according to the geometric structure of the target model to obtain a transformed source model, including: obtaining a predetermined number of first key points on the source model, and obtaining second key points on the target model corresponding to the first key points; through the correspondence between the first key points and the second key points, the positions of all vertices on the source model are transformed according to the positions of all vertices on the target model to obtain a transformed model.

[0007] Optionally, through the correspondence between key points and the correspondence between the first key point and the second key point, the positions of all vertices on the source model are transformed according to the positions of all vertices on the target model to obtain a transformed model, including: based on the correspondence between the first key point and the second key point, obtaining a transformation matrix, wherein the transformation matrix is ​​used to transform the position of the first key point on the source model to the position of the second key point on the target model; based on the transformation matrix, transforming the positions of all vertices on the source model to the positions of all vertices on the target model to obtain a transformed model.

[0008] Optionally, based on the correspondence between the first key point and the second key point, obtaining the transformation matrix includes: placing the coordinates of a predetermined number of first key points on the source model row by row in a first matrix X, where the number of rows of the first matrix is ​​a predetermined number; placing the coordinates of a predetermined number of second key points on the target model row by row in a second matrix Y, where the number of rows of the second matrix is ​​a predetermined number; and obtaining the transformation matrix M by minimizing the following formula:

[0009] ||MX-Y|| 2

[0010] Optionally, baking the transformation model to obtain rendering information includes: subdividing the transformation model to obtain a subdivided model; and baking the subdivided model to obtain rendering information.

[0011] Optionally, the source model is a quadrilateral mesh model.

[0012] Optionally, the rendering information includes a normal map, wherein information of each color channel of the normal map is stored in a 16-bit manner.

[0013] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, comprising: a model acquisition unit, configured to acquire a source model and a target model for the same object; a transformation unit, configured to transform the geometric structure of the source model according to the geometric structure of the target model to obtain a transformed model; a baking unit, configured to bake the transformed model to obtain rendering information; and a rendering unit, configured to render an image containing the source model based on the rendering information to obtain a rendered image.

[0014] Optionally, the transformation unit is also configured to obtain a predetermined number of first key points on the source model, and obtain second key points on the target model corresponding to the first key points; through the correspondence between the first key points and the second key points, the positions of all vertices on the source model are transformed according to the positions of all vertices on the target model to obtain a transformed model.

[0015] Optionally, the transformation unit is also configured to obtain a transformation matrix based on the correspondence between the first key point and the second key point, wherein the transformation matrix is ​​used to transform the position of the first key point on the source model to the position of the second key point on the target model; based on the transformation matrix, the positions of all vertices on the source model are transformed to the positions of all vertices on the target model to obtain a transformed model.

[0016] Optionally, the transformation unit is further configured to place the coordinates of a predetermined number of first key points on the source model row by row in a first matrix X, where the number of rows of the first matrix is ​​a predetermined number; place the coordinates of a predetermined number of second key points on the target model row by row in a second matrix Y, where the number of rows of the second matrix is ​​a predetermined number; and obtain the transformation matrix M by minimizing the following formula:

[0017] ||MX-Y|| 2

[0018] Optionally, the baking unit is further configured to perform subdivision processing on the transformed model to obtain a subdivision model; and perform baking processing on the subdivision model to obtain rendering information.

[0019] Optionally, the source model is a quadrilateral mesh model.

[0020] Optionally, the rendering information includes a normal map, wherein information of each color channel of the normal map is stored in a 16-bit manner.

[0021] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the image processing method according to the present disclosure.

[0022] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the image processing method according to the present disclosure as described above.

[0023] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer instructions, which implement the image processing method according to the present disclosure when executed by a processor.

[0024] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0025] According to the image processing method, device, electronic device and storage medium disclosed in the present invention, rendering information can be obtained by transforming the source model according to the target model (i.e., wrapping processing) and baking processing. Applying the rendering information on the source model can effectively transfer the rendering effect of the target model to the source model, so that the source model has the rendering effect on the target model, and the beauty of the source model is improved. Therefore, the present invention solves the problem that the related art cannot render a three-dimensional face model with beauty onto another face model.

[0026] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0028] Figure 1 is a schematic diagram showing an implementation scenario of an image processing method according to an exemplary embodiment of the present disclosure;

[0029] Figure 2 is a flowchart of an image processing method according to an exemplary embodiment;

[0030] Figure 3 is a comparison diagram of rendering effects of a package model and a target model according to an exemplary embodiment;

[0031] Figure 4 is a schematic diagram showing selected key points according to an exemplary embodiment;

[0032] Figure 5 is a schematic diagram of a geometric structure of a model according to an exemplary embodiment;

[0033] Figure 6 is a schematic diagram of a normal map according to an exemplary embodiment;

[0034] Figure 7 is a comparison diagram of rendering effects of an 8-bit image and a 16-bit image according to an exemplary embodiment;

[0035] Figure 8 is a comparison diagram of rendering effects after and without subdivision processing according to an exemplary embodiment;

[0036] Fig. 9 is a comparison diagram of rendering effects of models after different subdivision processing times according to an exemplary embodiment;

[0037] Fig.10is a comparison diagram showing rendering using a triangular mesh model and a quadrilateral mesh model as source models respectively according to an exemplary embodiment;

[0038] Fig.11 is a rendering effect diagram after a normal map is applied to a source model according to an exemplary embodiment;

[0039] Fig.12 is a flowchart showing a migration rendering effect according to an exemplary embodiment;

[0040] Fig.13 is a block diagram of an image processing device according to an exemplary embodiment;

[0041] Fig.14 is a block diagram of an electronic device 1400 according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0043] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0044] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0045] In view of the above problems, the present disclosure provides an image processing method, which can transfer the rendering effect of a target model to a source model. The following is an illustration taking the rendering of a face model as an example.

[0046] Figure 1is a schematic diagram showing an implementation scenario of an image processing method according to an exemplary embodiment of the present disclosure, such as Figure 1 The implementation scenario includes a server 100, a user terminal 110 and a user terminal 120, wherein the user terminals are not limited to two, and include but are not limited to mobile phones, personal computers and other devices. The user terminal can be equipped with a camera for acquiring facial images. The server can be a single server, or a server cluster composed of several servers, or a cloud computing platform or a virtualization center.

[0047] The server 100 first obtains a source model of the same category as the face model and a three-dimensional face model that conforms to aesthetics, and uses the three-dimensional face model that conforms to aesthetics as the target model, transforms the geometric structure of the source model according to the geometric structure of the target model to obtain a transformed model, and then performs subdivision processing on the transformed model to obtain a subdivision model, and then bakes the subdivision model to obtain rendering information. After obtaining the rendering information, the rendering information is sent to the user terminal 110 or the user terminal 120. When the user terminal 110 or the user terminal 120 obtains a face video through a camera, the face model in the face video is rendered using the rendering information to obtain a face model that is consistent with the rendering effect of the target model, so that the rendered face model has a certain aesthetic feeling.

[0048] Hereinafter, an image processing method, an apparatus, an electronic device, and a storage medium according to exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0049] Figure 2 is a flowchart of an image processing method according to an exemplary embodiment. Figure 2 As shown, the image processing method comprises the following steps:

[0050] In step S201, a source model and a target model for the same object are obtained. The object may be a human face or any other object, the target model may be an aesthetically pleasing three-dimensional human face model or any other model, and the source model may be any type of model to be rendered, as long as it is a model of the same object as the target model.

[0051] return Figure 2 In step S202, the geometric structure of the source model is transformed according to the geometric structure of the target model to obtain a transformed model. It should be noted that the transformation operation can also be regarded as a wrapping operation, that is, the wrapping operation is performed on the given source model and target model, and the source model is wrapped on the target model to obtain a model with a geometric structure similar to the target model, but the original topological structure remains unchanged, which can be called a wrapped model. Figure 3 is a comparison diagram of the rendering effects of a package model and a target model according to an exemplary embodiment. Figure 3As shown, the left side shows the rendering effect of the source model, the middle side shows the rendering effect of the wrapped model, and the right side shows the rendering effect of the target model. It can be seen that the geometric structures of the wrapped model and the target model are relatively close, and the rendering effect of the wrapped model is close to that of the target model.

[0052] According to an exemplary embodiment of the present disclosure, the geometric structure of the source model is transformed according to the geometric structure of the target model to obtain a transformed source model, which may include: obtaining a predetermined number of first key points on the source model, and obtaining a second key point on the target model corresponding to the first key point; through the correspondence between the first key point and the second key point, the positions of all vertices on the source model are transformed according to the positions of all vertices on the target model to obtain a transformed model. According to this embodiment, based on the correspondence between the predetermined number of key points on the source model and the target model, all vertices can be transformed without obtaining the correspondence for each vertex, thereby reducing the amount of calculation. The above correspondence is a position correspondence of key points, because the position can be transformed quickly, but it can also be any other relationship that can achieve vertex position transformation, and the present disclosure does not limit this.

[0053] For example, you can first select a series of key points on the source model and the target model. The key points are used to establish a correspondence between the two models. The purpose is to make the geometric structure of the source model as consistent as possible with the target model without modifying the topological structure of the source model. Therefore, it is necessary to select a series of meaningful key points, such as eyes, nose, forehead, lips and other areas, to establish an effective correspondence. Figure 4 is a schematic diagram of selected key points according to an exemplary embodiment, such as Figure 4 As shown, the left side is the source model, the right side is the target model, and 50 key points are selected for each model to establish the correspondence.

[0054] According to an exemplary embodiment of the present disclosure, through the correspondence between the first key point and the second key point, the positions of all vertices on the source model are transformed according to the positions of all vertices on the target model to obtain a transformed model, which may include: based on the correspondence between the first key point and the second key point, obtaining a transformation matrix, wherein the transformation matrix is ​​used to transform the position of the first key point on the source model to the position of the second key point on the target model; based on the transformation matrix, the positions of all vertices on the source model are transformed to the positions of all vertices on the target model to obtain the transformed model. According to this embodiment, the correspondence between the key points is converted into a matrix, and the vertices on the source model are transformed to the positions of the corresponding vertices in the target model through the matrix, so that the transformation can be realized quickly and conveniently.

[0055] According to an exemplary embodiment of the present disclosure, based on the correspondence between the first key point and the second key point, obtaining the transformation matrix may include: placing the coordinates of a predetermined number of first key points on the source model row by row in a first matrix X, where the number of rows of the first matrix is ​​a predetermined number; placing the coordinates of a predetermined number of second key points on the target model row by row in a second matrix Y, where the number of rows of the second matrix is ​​a predetermined number; and obtaining the transformation matrix M by minimizing the following formula:

[0056] ||MX-Y|| 2 (1)

[0057] According to this embodiment, the transformation matrix can be determined accurately and quickly.

[0058] For example, the positions of all key points of the source model {x i} are placed row by row into the first matrix X, and the positions of all key points of the target model {y i} are placed row by row into the second matrix Y. The matrix M is a transformation matrix used to transform the positions of the key points of the source model to the positions of the key points of the target model. In other words, the matrix M can be calculated by minimizing the above formula (1). This type of problem belongs to the least squares problem and has a standard solution. Finally, the positions of all vertices of the wrapped model can be obtained by applying the matrix M to all vertices of the source model. Figure 5 is a schematic diagram of the geometric structure of a model according to an exemplary embodiment, such as Figure 5 As shown, the left side represents the geometric structure of the source model, the middle side represents the geometric structure of the wrapped model, and the right side represents the geometric structure of the target model. It can be seen intuitively that the appearance of the wrapped model is close to that of the target model, and the topological structure (point-line connection method, number of vertices and triangles) is the same as the source model.

[0059] return Figure 2 In step S203, the transformed model is baked to obtain rendering information. The rendering information is used to simulate the rendering effect of the target model. Generally, the rendering information is a normal map, but the present disclosure is not limited to this. Here, the rendering information is taken as an example to illustrate that the baking operation is performed on the subdivision model in order to obtain the normal map. Since the subdivision model and the source model have the same topological structure, the normal map can be applied to the source model to transfer the rendering effect of the subdivision model to the source model. Figure 6 is a schematic diagram of a normal map according to an exemplary embodiment. Figure 6 Shown is a baked normal map.

[0060] According to an exemplary embodiment of the present disclosure, the rendering information includes a normal map, wherein information of each color channel of the normal map is stored in 16 bits. According to this embodiment, burrs can be avoided by storing in 16 bits.

[0061] Specifically, during the baking operation, the output normal map has high accuracy requirements, requiring each color channel to be stored in 16 bits, rather than the traditional 8 bits. This is because 8-bit images will cause rough edges in the rendering effect, while 16-bit images do not have such problems. Figure 7 is a comparison diagram of rendering effects of an 8-bit image and a 16-bit image according to an exemplary embodiment. Figure 7 As shown, the left side shows the rendering effect using an 8-bit normal map, with obvious burrs, and the right side shows the rendering effect using a 16-bit normal map, with no burrs.

[0062] According to an exemplary embodiment of the present disclosure, the transformation model is baked to obtain rendering information, including: subdividing the transformation model to obtain a subdivided model; baking the subdivided model to obtain rendering information. Generally, after the above transformation, that is, after the wrapping operation, the obtained wrapped model is not smooth enough. If you look closely at the wrapped model, you can see many sharp edges and corners, which will cause the rendering effect to have obvious jagged and mutation phenomena. Therefore, it is necessary to further perform a subdivision operation, and a smooth model can be obtained after the subdivision operation. For example, Figure 8 is a comparison diagram of rendering effects after and without subdivision processing according to an exemplary embodiment. Figure 8 As shown, the left 1 is the rendering effect of the source model, the left 2 is the rendering effect of the wrapped model, the left 3 is the rendering effect of the subdivided model after the subdivision operation is performed on the wrapped model, and the left 4 is the rendering effect of the target model. It can be seen that the rendering effect of the subdivided model is very delicate and smooth, without jagged or mutation problems, and is almost consistent with the rendering effect of the target model.

[0063] According to an exemplary embodiment of the present disclosure, performing subdivision processing on the transformation model to obtain the subdivision model may include: performing a first subdivision processing on the transformation model to obtain a first subdivision model; performing a second subdivision processing on the first subdivision model to obtain the subdivision model. According to this embodiment, the rendering effect of the model after two subdivisions has good fluidity and no roughness or discontinuity, so a model that meets the requirements can be obtained without multiple subdivisions, thereby reducing the workload.

[0064] It should be noted that, generally speaking, the number of subdivisions here is that two subdivisions are performed to obtain a face model that meets the requirements. Of course, other numbers of subdivisions can also be selected, and this disclosure does not limit this. Fig. 9is a comparison diagram of rendering effects of models after different subdivision times according to an exemplary embodiment. Fig. 9 As shown, the left side shows the rendering effect of the original wrapped model, the middle side shows the rendering effect of the wrapped model subdivided once, and the right side shows the rendering effect of the wrapped model subdivided twice. It can be seen that the rendering effect of the wrapped model subdivided twice has good fluidity, no roughness, and no discontinuity.

[0065] According to an exemplary embodiment of the present disclosure, the source model may be a quadrilateral mesh model. According to this embodiment, the selection of a quadrilateral mesh model can ensure the smoothness of the rendering result of the model that is subsequently subdivided. For example, in the process of performing the transformation of step S202, a quadrilateral mesh model may be selected as the source model. This is because if a subdivision process is performed later, the smoothness of the rendering effect after the triangular mesh model is subdivided is very poor, and obvious faults will appear, while the subdivision result of the quadrilateral mesh model meets the requirements. Fig.10 is a comparison diagram showing rendering using a triangular mesh model and a quadrilateral mesh model as source models, respectively, according to an exemplary embodiment. Fig.10 As shown, the left side represents the rendering result of the model subdivided from the triangular mesh model, and the right side represents the rendering result of the model subdivided from the quadrilateral mesh model.

[0066] In step S204, the image containing the source model is rendered based on the rendering information to obtain a rendered image. Generally, this step is to render the source model based on the rendering information to obtain a rendered model, that is, to obtain an image containing the rendered model, without modifying the geometric structure of the source model. Based on the normal map, the source model can obtain a rendering effect consistent with the target model.

[0067] For example, take the example of acting on the source model and rendering information as a normal map. Fig.11 FIG. 4 is a rendering effect diagram after a normal map is applied to a source model according to an exemplary embodiment. Fig.11 As shown, the left side is the rendering effect of the source model, the middle side is the rendering effect of the target model, and the right side is the rendering effect of the source model with the normal map applied. It can be seen from the rendering effect that the rendering effect consistent with the target model can be obtained without modifying the geometric structure of the source model, and the rendering result of the target model is effectively migrated to the source model.

[0068] In order to better understand the above embodiment, the following still takes the rendering information as a normal map as an example. Fig.12 Description of the system, Fig.12 is a flowchart showing a migration rendering effect according to an exemplary embodiment. Fig.12As shown, the target model is an aesthetically pleasing three-dimensional face model. The rendering effect of the target model can be transferred to the source model through three steps. First, the source model is wrapped according to the target model to obtain a wrapped model. Next, a subdivision operation is performed on the wrapped model to obtain a subdivision model. Then, a baking operation is performed on the subdivision model to obtain a normal map. Finally, the normal map is applied to the source model to obtain a rendering effect consistent with the target model.

[0069] In summary, this embodiment obtains a normal map after wrapping, subdividing, and baking operations. Applying the normal map on the source model can effectively migrate the rendering effect of the target model to the source model, which is extremely important for beauty applications. There are mainly two points: 1) Through the scheme disclosed in the present invention, the rendering effects of other aesthetically pleasing three-dimensional face models can be migrated to the current face model (source model), so that the current face model has the rendering effects on other face models, thereby improving the aesthetic feeling, and the rendering effect can be controlled manually. For example, if the rendering effect of certain face areas is unreasonable, the target model can be manually adjusted to obtain an aesthetically pleasing rendering effect, that is, the target model can be any model; 2) By applying the same normal map, different face models in the same posture can obtain the same rendering effect, thereby ensuring the stability of the makeup effect.

[0070] The solution disclosed in the present invention can migrate the rendering effect of the target face model so that the source model can produce the same rendering effect. It has strong universality. Any makeup effect can be migrated by modifying the target model without modifying the source model. This is because maintaining multiple sets of different face reconstruction algorithms is very laborious and not conducive to maintenance and management. The above problems can be avoided through this embodiment.

[0071] Fig.13 is a block diagram of an image processing device according to an exemplary embodiment. Fig.13 , the device comprises:

[0072] The model acquisition unit 130 is configured to acquire a source model and a target model for the same object; the transformation unit 132 is configured to transform the geometric structure of the source model according to the geometric structure of the target model to obtain a transformed model; the baking unit 134 is configured to bake the transformed model to obtain rendering information; the rendering unit 136 is configured to render an image containing the source model based on the rendering information to obtain a rendered image.

[0073] According to an exemplary embodiment of the present disclosure, the transformation unit 132 is also configured to obtain a predetermined number of first key points on the source model, and obtain second key points on the target model corresponding to the first key points; through the correspondence between the first key points and the second key points, the positions of all vertices on the source model are transformed according to the positions of all vertices on the target model to obtain a transformed model.

[0074] According to an exemplary embodiment of the present disclosure, the transformation unit 132 is also configured to obtain a transformation matrix based on the correspondence between the first key point and the second key point, wherein the transformation matrix is ​​used to transform the position of the first key point on the source model to the position of the second key point on the target model; based on the transformation matrix, the positions of all vertices on the source model are transformed to the positions of all vertices on the target model to obtain a transformed model.

[0075] According to an exemplary embodiment of the present disclosure, the transformation unit 132 is further configured to place the coordinates of a predetermined number of first key points on the source model row by row in a first matrix X, where the number of rows of the first matrix is ​​a predetermined number; place the coordinates of a predetermined number of second key points on the target model row by row in a second matrix Y, where the number of rows of the second matrix is ​​a predetermined number; and obtain the transformation matrix M by minimizing the following formula:

[0076] ||MX-Y|| 2

[0077] According to an exemplary embodiment of the present disclosure, the baking unit is further configured to perform subdivision processing on the transformation model to obtain a subdivision model; and perform baking processing on the subdivision model to obtain rendering information.

[0078] According to an exemplary embodiment of the present disclosure, the source model is a quadrilateral mesh model.

[0079] According to an exemplary embodiment of the present disclosure, the rendering information includes a normal map, wherein information of each color channel of the normal map is stored in a 16-bit manner.

[0080] According to an embodiment of the present disclosure, an electronic device may be provided. Fig.14 It is a block diagram of an electronic device 1400 according to an embodiment of the present disclosure, and the electronic device includes at least one memory 1401 and at least one processor 1402, wherein a set of computer executable instructions is stored in the at least one memory, and when the computer executable instruction set is executed by the at least one processor, an image processing method according to an embodiment of the present disclosure is executed.

[0081] As an example, the electronic device 1400 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above-mentioned instruction set. Here, the electronic device 1000 is not necessarily a single electronic device, but may also be any device or circuit collection capable of executing the above-mentioned instructions (or instruction sets) individually or in combination. The electronic device 1400 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0082] In the electronic device 1400, the processor 1402 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor 1402 may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0083] The processor 1402 may execute instructions or codes stored in the memory, wherein the memory 1401 may also store data. Instructions and data may also be sent and received over a network via a network interface device, wherein the network interface device may employ any known transmission protocol.

[0084] The memory 1401 may be integrated with the processor 1402, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 1401 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 1401 and the processor 1402 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 1402 can read files stored in the memory 1401.

[0085] In addition, the electronic device 1400 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device may be connected to each other via a bus and / or a network.

[0086] According to an embodiment of the present disclosure, a computer-readable storage medium may also be provided, wherein when the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the image processing method of the embodiment of the present disclosure. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0087] According to an embodiment of the present disclosure, a computer program product is provided, including computer instructions, and when the computer instructions are executed by a processor, the image processing method of the embodiment of the present disclosure is implemented.

[0088] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0089] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: include: Acquire a source model and a target model, wherein objects corresponding to the source model and the target model are different objects of the same type; Transforming the geometric structure of the source model according to the geometric structure of the target model to obtain a transformed model; Baking the transformation model to obtain rendering information; Rendering the image including the source model based on the rendering information to obtain a rendered image; The step of transforming the geometric structure of the source model according to the geometric structure of the target model to obtain a transformed source model includes: Acquire a predetermined number of first key points on the source model, and acquire second key points on the target model corresponding to the first key points; Through the correspondence between the first key point and the second key point, the positions of all vertices on the source model are transformed according to the positions of all vertices on the target model to obtain the transformed model.

2. The image processing method according to claim 1, characterized in that: The step of transforming the positions of all vertices on the source model according to the positions of all vertices on the target model through the correspondence between the first key point and the second key point to obtain the transformed model includes: Based on the correspondence between the first key point and the second key point, acquiring a transformation matrix, wherein the transformation matrix is ​​used to transform the position of the first key point on the source model to the position of the second key point on the target model; The positions of all vertices on the source model are transformed to the positions of all vertices on the target model based on the transformation matrix to obtain the transformed model.

3. The image processing method according to claim 2, characterized in that: The acquiring a transformation matrix based on the correspondence between the first key point and the second key point includes: Placing the coordinates of a predetermined number of first key points on the source model row by row in a first matrix X, where the number of rows of the first matrix is ​​the predetermined number; Placing the coordinates of a predetermined number of second key points on the target model row by row in a second matrix Y, where the number of rows of the second matrix is ​​the predetermined number; The transformation matrix M is obtained by minimizing the following formula: 。 4. The image processing method according to claim 1, wherein: The baking process of the transformation model to obtain rendering information includes: Performing subdivision processing on the transformation model to obtain a subdivision model; The subdivision model is baked to obtain rendering information.

5. The image processing method according to claim 4, characterized in that: The source model is a quadrilateral mesh model.

6. The image processing method according to claim 1, characterized in that: The rendering information includes a normal map, wherein information of each color channel of the normal map is stored in a 16-bit manner.

7. An image processing device, characterized in that: include: A model acquisition unit is configured to acquire a source model and a target model, wherein objects corresponding to the source model and the target model are different objects of the same type; a transformation unit, configured to transform the geometric structure of the source model according to the geometric structure of the target model to obtain a transformed model; A baking unit, configured to bake the transformation model to obtain rendering information; a rendering unit configured to render the image including the source model based on the rendering information to obtain a rendered image; Among them, the transformation unit is also configured to obtain a predetermined number of first key points on the source model, and obtain second key points on the target model corresponding to the first key points; through the correspondence between the first key points and the second key points, the positions of all vertices on the source model are transformed according to the positions of all vertices on the target model to obtain the transformed model.

8. The image processing device according to claim 7, wherein: The transformation unit is also configured to obtain a transformation matrix based on the correspondence between the first key point and the second key point, wherein the transformation matrix is ​​used to transform the position of the first key point on the source model to the position of the second key point on the target model; based on the transformation matrix, the positions of all vertices on the source model are transformed to the positions of all vertices on the target model to obtain the transformed model.

9. The image processing device according to claim 8, wherein: The transformation unit is further configured to place the coordinates of a predetermined number of first key points on the source model row by row in a first matrix X, where the number of rows of the first matrix is ​​the predetermined number; The coordinates of a predetermined number of second key points on the target model are placed row by row in a second matrix Y, where the number of rows of the second matrix is ​​the predetermined number; and the transformation matrix M is obtained by minimizing the following formula: 。 10. The image processing device according to claim 7, wherein: The baking unit is further configured to perform subdivision processing on the transformation model to obtain a subdivision model; and perform baking processing on the subdivision model to obtain rendering information.

11. The image processing device according to claim 10, wherein: The source model is a quadrilateral mesh model.

12. The image processing device according to claim 7, wherein: The rendering information includes a normal map, wherein information of each color channel of the normal map is stored in a 16-bit manner.

13. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image processing method according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the image processing method according to any one of claims 1 to 6.

15. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the image processing method according to any one of claims 1 to 6 is implemented.

Images