Image processing method and device, electronic equipment and computer readable storage medium

Abstract

An image processing method, an image processing device, an electronic device and a computer readable storage medium. The image processing method comprises: in response to detecting a detection object, obtaining current feature information of the detection object, the current feature information being used to indicate a current state of a target feature of the detection object; obtaining limit deformation information of the target feature, the limit deformation information being obtained by calculating a target virtual sub-image when the target feature is in at least one limit state; based on the limit deformation information and the current feature information, determining movement information of a feature point in an initial virtual image, the initial virtual image being obtained by superimposing a plurality of virtual sub-images, the plurality of virtual sub-images including a target virtual sub-image corresponding to at least part of the limit states in the at least one limit state; and driving the feature point in the initial virtual image to move according to the movement information to generate a current virtual image corresponding to the current state. The method can reduce the design difficulty and driving difficulty of the virtual image.

Description

Technical Field

[0001] Embodiments of this disclosure relate to an image processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the rapid development of the internet, virtual avatars are widely used in emerging fields such as live streaming, short videos, and games. The application of virtual avatars not only enhances the fun of human-computer interaction but also brings convenience to users. For example, on live streaming platforms, broadcasters can use virtual avatars to broadcast without showing their faces. Summary of the Invention

[0003] At least one embodiment of this disclosure provides an image processing method, comprising: in response to detecting a target object, acquiring current feature information of the target object, the current feature information being used to indicate the current state of a target feature of the target object; acquiring limit deformation information of the target feature, the limit deformation information being calculated from a target virtual sub-image when the target feature is in at least one limit state; determining movement information of feature points in an initial virtual image based on the limit deformation information and the current feature information, the initial virtual image being obtained by superimposing multiple virtual sub-images, the multiple virtual sub-images including a target virtual sub-image corresponding to at least some of the limit states; and driving the feature points in the initial virtual image to move according to the movement information to generate a current virtual image corresponding to the current state.

[0004] For example, in one embodiment of the image processing method provided in this disclosure, the method further includes: obtaining depth information of each of a plurality of virtual sub-images, and obtaining an initial virtual image based on the depth information of each of the plurality of virtual sub-images and the plurality of virtual sub-images.

[0005] For example, in an image processing method provided in an embodiment of this disclosure, obtaining the extreme deformation information of a target feature includes: determining a first extreme position and a second extreme position of the target feature based on a target virtual sub-image at at least one extreme state; sampling the first extreme position and the second extreme position to obtain sampling results of multiple sampling points; and calculating the sampling results to obtain the extreme deformation information of the target feature.

[0006] For example, in an image processing method provided in an embodiment of this disclosure, the target virtual sub-image includes a first layer in a first limit state and a second layer in a second limit state; determining the first limit position and the second limit position of the target feature based on the target virtual sub-image in at least one limit state includes: masking the alpha channels of the first layer and the second layer respectively to obtain two mask sub-images, and merging the two mask sub-images into one mask image; and determining the first limit position and the second limit position based on the mask image.

[0007] For example, in an image processing method provided in one embodiment of this disclosure, the sampling result includes the position coordinates of each sampling point in a plurality of sampling points at a first limit position and a second limit position respectively. The sampling result is calculated to obtain the limit deformation information of the target feature, including: calculating the height difference between the first limit position and the second limit position of each sampling point based on the position coordinates of each sampling point at the first limit position and the second limit position respectively; performing curve fitting on the plurality of sampling points based on the height difference to obtain the limit deformation value curve; substituting each target vertex in the target feature into the limit deformation value curve to obtain the limit deformation value of each target vertex in the target feature, wherein each target vertex corresponds to at least some feature points in the initial virtual sub-image.

[0008] For example, in an image processing method provided in one embodiment of this disclosure, curve fitting includes polynomial fitting, and the limiting deformation curve includes a polynomial curve.

[0009] For example, in an image processing method provided in an embodiment of this disclosure, the sampling result includes the position coordinates of each sampling point in a plurality of sampling points at a first limit position and a second limit position respectively. The sampling result is calculated to obtain the limit deformation information of the target feature, including: calculating the height difference between the first limit position and the second limit position of each sampling point based on the position coordinates of each sampling point at the first limit position and the second limit position respectively; and using the height difference between the first limit position and the second limit position of each sampling point as the limit deformation information.

[0010] For example, in an image processing method provided in an embodiment of this disclosure, determining the movement information of feature points in an initial virtual image based on limit deformation information and current feature information includes: determining the current state value of the target feature relative to the reference state based on the current feature information; and determining the movement information of feature points in the initial virtual image based on the current state value and limit deformation information.

[0011] For example, in an image processing method provided in one embodiment of this disclosure, determining the current state value of a target feature relative to a reference state based on current feature information includes: obtaining a mapping relationship between feature information and state value; and determining the current state value of the target feature relative to the reference state based on the mapping relationship and current feature information.

[0012] For example, in an image processing method provided in one embodiment of this disclosure, obtaining the mapping relationship between feature information and state values ​​includes: obtaining multiple samples, wherein each sample includes the correspondence between sample feature information of the target feature and sample state value; and constructing a mapping function based on the correspondence, wherein the mapping function represents the mapping relationship between feature information and state value.

[0013] For example, in an image processing method provided in an embodiment of this disclosure, the sample feature information includes first feature information and second feature information, and the sample state value includes a first value corresponding to the first feature information and a second value corresponding to the second feature information. Based on the correspondence, a mapping function is constructed, including: constructing a system of linear equations; substituting the first feature information and the first value, the second feature information and the second value into the system of linear equations respectively, and solving the system of linear equations to obtain the mapping function.

[0014] For example, in an image processing method provided in an embodiment of this disclosure, the motion information includes the motion distance. Determining the motion information of feature points in the initial virtual image based on the current state value and the limit deformation information includes: calculating the current state value and the limit deformation information to determine the motion distance of feature points in the initial virtual image.

[0015] For example, in an image processing method provided in one embodiment of this disclosure, the calculation of the current state value and the limit deformation information to determine the movement distance of feature points in the initial virtual image includes: performing a multiplication operation on the current state value and the limit deformation information to determine the movement distance of feature points in the initial virtual image.

[0016] For example, in an image processing method provided in an embodiment of this disclosure, the motion information includes a motion distance. Based on the motion information, driving the feature points in the initial virtual image to move includes: driving the feature points in the target virtual sub-image of the initial virtual image to move a motion distance from the position of the initial state to the position of at least one of the extreme states different from the initial state.

[0017] For example, in an image processing method provided in one embodiment of this disclosure, the motion information includes a target position, and the motion information is used to drive the feature points in the initial virtual image to move, including: driving the feature points in the initial virtual image to move to the target position.

[0018] For example, in an image processing method provided in one embodiment of this disclosure, the current feature information includes comparison information between target features and reference features of the detection object, and the comparison information remains unchanged when the distance between the detection object and the image acquisition device used to detect the detection object changes.

[0019] For example, in an image processing method provided in one embodiment of this disclosure, each virtual sub-image corresponds to one of a plurality of virtual features to be virtualized of a detection object. The plurality of virtual features to be virtualized include the target feature. The depth information of each of the plurality of virtual sub-images includes the depth value of each virtual sub-image and the depth value of the feature point in each virtual sub-image. In the direction perpendicular to the face of the detection object, the depth value of each virtual sub-image is proportional to a first distance, which is the distance between the virtual feature to be virtualized corresponding to the virtual sub-image and the eye of the detection object; and the depth value of the feature point in each virtual sub-image is proportional to the first distance.

[0020] For example, in an image processing method provided in an embodiment of this disclosure, the target feature includes at least one of eyelashes and mouth. When the target feature is eyelashes, the limit state of the target feature is the state of the target feature when the eyes of the detected object are open. When the target feature is mouth, the limit state of the target feature is the state of the mouth when the mouth is opened to the maximum extent.

[0021] At least one embodiment of this disclosure provides an image processing apparatus, comprising: a detection unit configured to, in response to detecting a detection object, acquire current feature information of the detection object, the current feature information indicating the current state of a target feature of the detection object; an acquisition unit configured to acquire limit deformation information of the target feature, the limit deformation information being calculated from a target virtual sub-image when the target feature is in a limit state; a determination unit configured to, based on the limit deformation information and the current feature information, determine movement information of feature points in an initial virtual image, the initial virtual image being obtained by superimposing multiple virtual sub-images, the multiple virtual sub-images including at least a portion of the target virtual sub-image; and a driving unit configured to, according to the movement information, drive the feature points in the initial virtual image to move, thereby generating a current virtual image corresponding to the current state.

[0022] At least one embodiment of this disclosure provides an electronic device including a processor; a memory including one or more computer program modules; the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include instructions for implementing the image processing method provided in any embodiment of this disclosure.

[0023] At least one embodiment of this disclosure provides a computer-readable storage medium for storing non-transitory computer-readable instructions that, when executed by a computer, can implement the image processing method provided in any embodiment of this disclosure. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0025] Figure 1A A flowchart of an image processing method provided by at least one embodiment of the present disclosure is shown;

[0026] Figure 1B The illustration shows a schematic diagram of multiple virtual sub-images provided in some embodiments of the present disclosure;

[0027] Figure 2A At least one embodiment of the present disclosure is shown. Figure 1A Flowchart of the method for step S20;

[0028] Figure 2B and Figure 2C A schematic diagram of a generated mask pattern provided in at least one embodiment of the present disclosure is shown;

[0029] Figure 2D At least one embodiment of the present disclosure is shown. Figure 2A Flowchart of the method for step S23;

[0030] Figure 3 At least one embodiment of the present disclosure is shown. Figure 1A Flowchart of the method for step S30;

[0031] Figure 4 A schematic diagram illustrating the effect of an image processing method provided in at least one embodiment of this disclosure is shown.

[0032] Figure 5 A schematic block diagram of an image processing apparatus provided in at least one embodiment of the present disclosure is shown;

[0033] Figure 6 A schematic block diagram of an electronic device provided in at least one embodiment of the present disclosure is shown;

[0034] Figure 7 A schematic block diagram of another electronic device provided in at least one embodiment of the present disclosure is shown; and

[0035] Figure 8 A schematic diagram of a computer-readable storage medium provided in at least one embodiment of the present disclosure is shown. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0037] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0038] Virtual avatars can be driven in real-time based on the actions and expressions of objects detected by electronic devices (e.g., users). The design and driving of virtual avatars are both highly complex. Creating a 3D virtual avatar requires character art design, model creation, and animation rigging. Further driving and presentation involve motion capture technology, holographic hardware technology, augmented reality (AR) technology, virtual reality (VR) technology, and driver development, resulting in a long production cycle, significant implementation and driving difficulties, and high costs. Creating a 2D virtual avatar requires professional concept art design according to the requirements of different concept art design platforms. During driving, each frame of animation is drawn to form actions and expressions, and material transformation is also required on specific rendering software (such as Live2D), making implementation and driving also highly complex.

[0039] At least one embodiment of this disclosure provides an image processing method, an image processing apparatus, an electronic device, and a computer-readable storage medium. The image processing method includes: in response to detecting a target object, acquiring current feature information of the target object, the current feature information indicating the current state of a target feature of the target object; acquiring limit deformation information of the target feature, the limit deformation information being calculated from a target virtual sub-image when the target feature is in a limit state; determining movement information of feature points in an initial virtual image based on the limit deformation information and the current feature information, the initial virtual image being obtained by superimposing multiple virtual sub-images, the multiple virtual sub-images including at least a portion of the target virtual sub-image; and driving the feature points in the initial virtual image to move according to the movement information to generate a current virtual image corresponding to the current state. This image processing method can reduce the design and driving difficulty of virtual avatars, making virtual avatars easier to implement and drive.

[0040] Figure 1A A flowchart of an image processing method provided by at least one embodiment of the present disclosure is shown.

[0041] like Figure 1A As shown, the method may include steps S10 to S40.

[0042] Step S10: In response to the detection of a target object, obtain the current feature information of the target object. The current feature information is used to indicate the current state of the target features of the target object.

[0043] Step S20: Obtain the limit deformation information of the target feature. The limit deformation information is calculated from the target virtual sub-image when the target feature is in at least one limit state.

[0044] Step S30: Based on the limit deformation information and the current feature information, determine the movement information of feature points in the initial virtual image. The initial virtual image is obtained by superimposing multiple virtual sub-images. The multiple virtual sub-images include at least a target virtual sub-image corresponding to at least some of the limit states in at least one limit state.

[0045] Step S40: Based on the movement information, drive the feature points in the initial virtual image to move, so as to generate the current virtual image corresponding to the current state.

[0046] This embodiment can determine the movement information of feature points in the initial virtual image based on the target virtual sub-image corresponding to the extreme state of the target feature and the current state of the target feature, thereby driving the feature points to move according to the movement information to generate the current virtual image corresponding to the current state.

[0047] In the embodiments of this disclosure, by acquiring the current state of the target features of the detected object in real time, and driving the movement of feature points in the initial virtual image according to the current feature information of the current state, the generation and driving of the virtual image can be realized. Therefore, the embodiments of this disclosure do not require the use of animation skeleton binding technology, holographic hardware technology, or specific drawing software (such as Live2D) to realize the generation and driving of the virtual image, reducing the difficulty of realizing and driving the virtual image.

[0048] For step S10, the object to be detected can be an object to be virtualized, such as a living being like a person or pet. For example, the object to be detected can be obtained through a detection device (camera, infrared device, etc.).

[0049] In some embodiments of this disclosure, the detection object includes a feature to be virtualized, which includes a target feature. For example, the feature to be virtualized is a feature in the detection object that needs to be virtualized, and the target feature is a key part of the detection object. The key part is the part that needs to be driven in the virtual image obtained after the feature to be virtualized. For example, the feature to be virtualized of the detection object may include, but is not limited to, cheeks, shoulders, hair, and various facial features. For example, among the features to be virtualized, cheeks and shoulders are parts that do not need to be driven in the virtual image, while eyebrows, upper eyelashes, lower eyelashes, mouth, upper eyelids, and lower eyelids need to be driven in the virtual image. Therefore, the target feature may include, but is not limited to, eyebrows, upper eyelashes, lower eyelashes, mouth, upper eyelids, and lower eyelids.

[0050] For example, in response to the camera detecting a target object, the current feature information of the target object is acquired. For example, multiple facial landmarks are used to locate and detect the target object, thereby acquiring the current feature information of the target object. For example, a 106-facial landmark detection algorithm or a 280-facial landmark detection algorithm can be used, or other applicable algorithms can be used; the embodiments disclosed herein are not limited in this regard.

[0051] In some embodiments of this disclosure, the current feature information may include facial movement information and body posture information of the detected object. For example, the current feature information may indicate the degree of eye opening, mouth opening, etc.

[0052] In some embodiments of this disclosure, the current feature information includes comparison information between target features and reference features of the detected object, and the comparison information remains unchanged when the distance of the detected object relative to the image acquisition device used to detect the detected object changes.

[0053] For example, when the distance between the detected object and the camera varies, as long as the expression of the detected object remains unchanged, these current feature information will not change. This can alleviate the shaking of the virtual image caused by the change in the distance between the detected object and the camera.

[0054] For example, reference features can be faces, eyes, etc. For example, the current feature information can include the ratio h1 / h0 of the height h1 from the eyebrows to the eyes to the height h0 of the face, the ratio h2 / h0 of the height h2 of the upper and lower eyelids to the height h0 of the face, the ratio h3 / k0 of the distance h3 from the pupil to the outer corner of the eye to the width k0 of the eye, and the ratio s1 / s0 of the area s1 of the mouth to the area s0 of the face. That is, the height of the eyebrows is represented by the ratio h1 / h0, the degree of eye opening is represented by the ratio h2 / h0, the position of the pupil is represented by the ratio h3 / k0, and the degree of mouth opening is represented by the ratio s1 / s0.

[0055] For step S20, in embodiments of this disclosure, the target feature has a limit state, which is, for example, the maximum deformation state of the target feature. For example, in the case of the target feature being an upper eyelash or upper eyelid, the limit state of the target feature includes the state of the target feature when the eyes of the detected object are opened to the maximum extent. For example, in the case of the target feature being an upper eyelash, the limit state of the upper eyelash includes the state of the upper eyelash when the eyes of the detected object are opened to the maximum extent. In the case of the target feature being a mouth, the limit state of the target feature includes the state of the mouth when the mouth is opened to the maximum extent.

[0056] In some embodiments of this disclosure, where the target feature is the upper eyelash or upper eyelid, the limiting state of the target feature may further include the state of the target feature when the eyes of the detected object are closed. For example, if the target feature is the upper eyelash, the limiting state of the upper eyelash includes not only the state of the upper eyelash when the eyes of the detected object are open to the maximum extent, but also the state of the upper eyelash when the eyes are closed. Similarly, if the target feature is the mouth, the limiting state of the target feature includes not only the state of the mouth when it is open to the maximum extent, but also the state of the mouth when it is closed.

[0057] In some embodiments of this disclosure, the virtual sub-image of the target when the target features of the detected object are in an extreme state can be pre-drawn by the designer. For example, the designer can pre-draw the virtual sub-image of the target corresponding to the upper eyelashes when the eyes are wide open, and the virtual sub-image of the target corresponding to the upper eyelashes when the eyes are closed.

[0058] For step S20, the limit deformation information can refer to the maximum deformation value of the target feature. For example, the limit deformation information can be calculated by taking the limit positions of multiple sampling points in the target feature when the target feature is in two different limit states. The following section will combine... Figure 2AOne implementation of step S20 has been described, and will not be repeated here.

[0059] Regarding step S30, in some embodiments of this disclosure, for example, the multiple virtual sub-images can be extracted and drawn by the designer in image editing software. These multiple virtual sub-images serve as multiple layers, and overlaying these layers yields an initial virtual image. For example, images are drawn in layers according to the layer names preset by the designer, allowing the layers to be retrieved and driven based on their names. Alternatively, the multiple virtual sub-images can be pre-drawn by the designer using graphic design tools such as Photoshop and stored in a storage unit. For example, the multiple virtual sub-images can be obtained by reading the PSD file from the Photoshop graphic design tool in the storage unit. Of course, the embodiments of this disclosure are not limited to these; the designer can use any software or tool to draw the virtual sub-images.

[0060] Figure 1B The illustration shows a schematic diagram of a plurality of virtual sub-images provided in some embodiments of the present disclosure.

[0061] like Figure 1B As shown, multiple virtual sub-images include a mouth image 110, an upper eyelash image 120, a pupil image 130, and a sclera image 140.

[0062] It is necessary to understand that Figure 1B Only a portion of the virtual sub-images are shown, not all of them. For example, multiple virtual sub-images also include images of hair, nose, and so on. Designers can draw the required virtual sub-images according to their actual needs.

[0063] The multiple virtual sub-images include target virtual sub-images corresponding to at least some of the limit states in at least one limit state. For example, the target feature has two limit states, and the multiple virtual sub-images include target virtual sub-images corresponding to one of the limit states of the target feature. For example, the multiple virtual sub-images include target virtual sub-images of eyelashes when the eyes are opened to their maximum extent, that is, the initial virtual image is the virtual image corresponding to when the eyes are opened to their maximum extent.

[0064] For example, in Figure 1B In the image, mouth image 110 is the image of the mouth when it is opened to its maximum extent, and upper eyelash image 120 is the image of the eyes when they are opened to their maximum extent.

[0065] For example, each virtual sub-image corresponds to one of multiple virtual features of the detected object, including the target feature.

[0066] For example, multiple virtual sub-images may include not only the target virtual sub-image when the target feature is in its extreme state, but also virtual sub-images of some parts of the feature to be virtualized that do not need to be driven, as well as a background image. For example, in addition to the individual target virtual sub-images of eyebrows, upper eyelashes, lower eyelashes, mouth, upper eyelids, and lower eyelids, multiple virtual sub-images may also include virtual sub-images of cheeks and shoulders. The background image may be, for example, an environmental image of the environment in which the object being detected is located.

[0067] In some embodiments of this disclosure, such as Figure 1A As shown, the image processing method may include step S50 in addition to steps S10 to S40. For example, step S50 may be performed before step S10.

[0068] Step S50: Obtain the depth information of each of the multiple virtual sub-images, and obtain an initial virtual image based on the depth information of each of the multiple virtual sub-images and the multiple virtual sub-images.

[0069] This embodiment sets depth information for each virtual sub-image, so that the initial virtual image obtained by superimposing multiple virtual sub-images has a three-dimensional effect, thereby driving the movement of feature points in the initial virtual image to obtain a virtual image that also has a three-dimensional effect.

[0070] For example, multiple virtual sub-images can be superimposed to obtain an initial virtual image.

[0071] For step S50, the depth information of each of the plurality of virtual sub-images includes the depth value of each virtual sub-image and the depth value of the feature points in each virtual sub-image. Each virtual sub-image corresponds to one of a plurality of virtual features to be virtualized of the detected object, the plurality of virtual features including the target feature.

[0072] In some embodiments of this disclosure, the depth information of each virtual sub-image may be preset by the designer.

[0073] In some embodiments of this disclosure, each virtual sub-image corresponds to one of a plurality of virtual features to be virtualized of a detection object. These virtual features include target features. In a direction perpendicular to the face of the detection object, the depth value of each virtual sub-image is proportional to a first distance, and the depth value of a feature point in each virtual sub-image is also proportional to the first distance, which is the distance between the virtual feature corresponding to the virtual sub-image and the eyes of the detection object. That is, in a direction perpendicular to the face of the detection object, the depth value of each virtual sub-image and the depth value of a feature point in each virtual sub-image are proportional to the distance between the virtual sub-image and the eyes of the detection object.

[0074] It should be understood that the distance in this disclosure can be a vector, meaning the distance can be either positive or negative.

[0075] For example, multiple virtual sub-images include a nose virtual sub-image corresponding to the nose of the detected object and a shoulder virtual sub-image corresponding to the shoulder of the detected object. In the direction perpendicular to the face of the detected object, the distance between the virtual feature (nose) and the eyes of the detected object is -f1, and the distance between the virtual feature (shoulder) and the eyes of the detected object is f2, where both f1 and f2 are greater than 0. Therefore, the depth value of the nose virtual sub-image is less than the depth value of the eye virtual sub-image, and the depth value of the eye virtual sub-image is less than the depth value of the shoulder virtual sub-image. This embodiment makes the nose of the virtual image appear in front of the eyes and the shoulders appear behind the eyes.

[0076] For example, for each virtual sub-image, the virtual sub-image is divided into multiple bounding boxes. The bounding box coordinates of each bounding box are extracted, and depth values ​​are set for the top-left and bottom-right vertices of each bounding box based on a first distance. Each vertex of the bounding box can represent a feature point. The depth value of a vertex of the bounding box is proportional to the distance between that vertex and the eye of the detected object.

[0077] For example, for the target feature eyebrow, the virtual sub-image of the eyebrow is divided into three equal rectangles. The top-left and bottom-right corner vertices of each of the three rectangles are extracted and their bounding box coordinates (X, Y) in the image coordinate system are obtained. X is the horizontal coordinate and Y is the vertical coordinate. A depth value Z is set for each vertex to obtain the coordinates (X, Y, Z) of each vertex. The W coordinate and four texture coordinates (s, t, r, q) are also added. Each vertex has 8 dimensions. The number of vertex groups is 2 rows and 4 columns (2, 4), for a total of 8. The first row is (X, Y, Z, W), and the second row is (s, t, r, q). The image coordinate system is, for example, with the width direction of the virtual sub-image as the X-axis, the height direction of the virtual sub-image as the Y-axis, and the bottom-left corner of the virtual sub-image as the origin.

[0078] Compared to the virtual sub-image of the eyebrows, the virtual sub-image of the posterior hair (the part of the hair furthest from the eyes of the detected object) is divided into more rectangles, each corresponding to a vertex group. In the virtual sub-image of the posterior hair, the depth value (i.e., z-coordinate) of the middle region is smaller than the depth values ​​(i.e., z-coordinate) of the sides, forming a curved shape of the posterior hair.

[0079] Figure 2A At least one embodiment of the present disclosure is shown. Figure 1A The flowchart of step S20.

[0080] like Figure 2A As shown, step S20 may include steps S21 to S23.

[0081] Step S21: Determine the first and second limit positions of the target features based on the target virtual sub-image at at least one limit state.

[0082] Step S22: Sample the first and second limit positions to obtain sampling results of multiple sampling points.

[0083] Step S23: Calculate the sampling results to obtain the limit deformation information of the target feature.

[0084] For step S21, the first limit position is the location of the target feature in the virtual image when the target feature is in the first limit state, and the second limit position is the location of the target feature in the virtual image when the target feature is in the second limit state. The location of the target feature can be represented, for example, by the position coordinates of multiple feature points in the target feature.

[0085] In some embodiments of this disclosure, the target virtual sub-image in at least one extreme state includes a first layer in a first extreme state and a second layer in a second extreme state. Step S21 includes: masking the alpha channels of the first layer and the second layer respectively to obtain two mask sub-images, and merging the two mask sub-images into one mask image; and determining the first extreme position and the second extreme position based on the mask image.

[0086] Figure 2B and Figure 2C A schematic diagram of a generated mask pattern provided in at least one embodiment of the present disclosure is shown.

[0087] exist Figure 2B and Figure 2C In the illustrated embodiment, the target feature is the upper eyelashes.

[0088] like Figure 2B As shown, the alpha channel of the first layer in the first extreme state of the upper eyelashes is masked to obtain the mask sub-image 210 of the first extreme state. The alpha channel of the second layer in the second extreme state of the upper eyelashes is masked to obtain the mask sub-image 220 of the second extreme state.

[0089] like Figure 2C As shown, mask sub-images 210 and 220 are merged into a single mask image 230. For example, mask sub-images 210 and 220 can be merged into a single mask image 230 based on the position coordinates of multiple feature points in mask sub-image 210 and mask sub-image 220.

[0090] like Figure 2CAs shown, the mask image 230 includes a target feature sub-image 231 and a target feature sub-image 232. The target feature sub-image 231 represents the first extreme position where the target feature is in the first extreme state, and the target feature sub-image 232 represents the second extreme position where the target feature is in the second extreme state.

[0091] For step S22, for example, uniform sampling is performed on the target feature sub-image 231 and the target feature sub-image of the mask image 230, respectively.

[0092] In some embodiments of this disclosure, the sampling result includes the position coordinates of each of the multiple sampling points at a first extreme position and a second extreme position, respectively.

[0093] For example, such as Figure 2C As shown, the abscissas x1, x2, ..., xn of multiple sampling points on the upper eyelash are determined, and the ordinates of these sampling points in target feature sub-maps 231 and 232 are obtained respectively. The ordinate of the sampling point in target feature sub-map 231 is the position coordinate of the sampling point at the first extreme position. The ordinate of the sampling point in target feature sub-map 232 is the position coordinate of the sampling point at the second extreme position.

[0094] like Figure 2C As shown, uniform sampling yields the coordinates (x1, y1), ..., (xn, yn) of multiple sampling points on the upper eyelash at the first extreme position and the coordinates (x1, b1), ..., (xn, bn) at the second extreme position.

[0095] For step S23, for example, based on the position coordinates of each sampling point at the first and second extreme positions, the height difference between the first and second extreme positions is calculated, and this height difference is used as the limit deformation information. The limit deformation information obtained in this embodiment is more accurate, thus enabling more precise driving of the initial virtual image.

[0096] For example, for each sampling point, the difference between the ordinate of the second extreme position and the ordinate of the first extreme position is calculated; this difference is the height difference. For example, in Figure 2C In the scenario shown, the height differences between multiple sampling points x1...xn are b1-y1...bn-yn, respectively. In this embodiment, the limiting deformation information includes the height differences between each sampling point x1,...,xn as b1-y1,...,bn-yn, respectively.

[0097] Figure 2D At least one embodiment of the present disclosure is shown. Figure 2A The flowchart of step S23.

[0098] like Figure 2D As shown, step S23 may include steps S231 to S233.

[0099] Step S231: Calculate the height difference between the first and second extreme positions of each sampling point based on the position coordinates of each sampling point at the first and second extreme positions respectively.

[0100] Step S232: Based on the height difference, perform curve fitting on multiple sampling points to obtain the limit deformation value curve.

[0101] Step S233: Substitute each target vertex in the target feature into the limit deformation value curve to obtain the limit deformation value of each target vertex in the target feature, wherein each target vertex corresponds to at least some feature points in the initial virtual sub-image.

[0102] In this embodiment, the limit deformation information includes the limit deformation value of each target vertex in the target feature. This embodiment fits a limit deformation value curve based on the height difference of multiple sampling points, and obtains the limit deformation value of each target vertex in the target feature based on the limit deformation value curve. This avoids some feature points on the edge parts of the target feature being missed and not driven, thus making the current virtual image more complete and realistic. For example, if the feature points corresponding to the edge parts of the upper and lower lips (e.g., the corners of the mouth) are not driven, the corners of the mouth in the current virtual image will not close when the mouth of the detected object is closed, causing the current virtual image to be incomplete and inconsistent with reality.

[0103] In step S231, for example, the height difference between the first and second extreme positions of each sampling point can be calculated according to the method described above, which will not be repeated here.

[0104] In step S232, curve fitting includes polynomial fitting, and the limiting deformation curve includes a polynomial curve. In some embodiments of this disclosure, the degree of the polynomial can be determined based on the complexity and approximate shape of the virtual sub-image (layer) to be driven.

[0105] For example, the abscissa and height difference of multiple sampling points form a fitting sample, that is, the fitting sample is (x1,b1-y1), ..., (xn,bn-yn). Substitute the fitting sample (x1,b1-y1), ..., (xn,bn-yn) into the following polynomial to perform polynomial fitting to obtain the limit deformation curve.

[0106] y = ax n +bx n-1 +cx n-2 +…+wx+z

[0107] In step S233, in some embodiments of this disclosure, the abscissa of each feature point in the virtual sub-image corresponding to the target feature is substituted into the limit deformation value curve to obtain the limit deformation value of each feature point. For example, the target vertex in the target feature is obtained by extracting the X coordinates of multiple feature points from the coordinates (X, Y, Z, W) described in step S50 above, and transforming the X coordinates into coordinates in the virtual sub-image corresponding to the target feature.

[0108] For step S30, in some embodiments of this disclosure, the movement information includes the movement distance. In other embodiments of this disclosure, the movement information includes the target location to which the feature points in the initial virtual image need to be moved.

[0109] Figure 3 At least one embodiment of the present disclosure is shown. Figure 1A The flowchart of step S30.

[0110] like Figure 3 As shown, step S30 may include steps S31 and S32.

[0111] Step S31: Determine the current state value of the target feature relative to the reference state based on the current feature information.

[0112] Step S32: Determine the movement information of feature points in the initial virtual image based on the current state value and the limit deformation information.

[0113] For step S31, the reference state can be preset by the designer. For example, the reference state is eyes closed and / or eyes open to the maximum extent.

[0114] The current state value can be a parameter used to reflect the relationship between the current feature information and the reference feature information of the target feature when the detected object is in the reference state.

[0115] For example, the reference state is eyes open to their maximum extent, the target feature is eyelashes, and the current state value can be a parameter that reflects the relationship between the current position of the eyelashes and the position of the eyelashes when the eyes are open to their maximum extent.

[0116] In some embodiments of this disclosure, step S31 may include obtaining the mapping relationship between feature information and state value, and determining the current state value of the target feature relative to the reference state based on the mapping relationship and the current feature information.

[0117] For example, multiple samples are acquired, each sample including the correspondence between sample feature information of the target feature and sample state value; based on the correspondence, a mapping function is constructed, which represents the mapping relationship between feature information and state value.

[0118] For example, sample feature information includes first feature information and second feature information, and sample state values ​​include a first value corresponding to the first feature information and a second value corresponding to the second feature information. Constructing the mapping function includes constructing a system of linear equations, and substituting the first feature information and the first value, the second feature information and the second value into the system of linear equations respectively, and solving the system of linear equations to obtain the mapping function.

[0119] For example, the linear equation system is a system of two linear equations in two variables. The first feature information is the position coordinate Y0 of the feature point on the eyelash in the direction of movement when the eye is fully open in the sample, with a first value of 0. The second feature information is the position coordinate Y1 of the feature point on the eyelash in the direction of movement when the eye is closed in the sample, with a second value of 1. The direction of movement of the eyelash is perpendicular to the width direction of the eye. The first and second feature information can be the result of statistical calculations on multiple samples. A system of two linear equations in two variables is constructed based on (Y0, 0) and (Y1, 1), and the mapping function u = av + b is obtained by solving the system of two linear equations in two variables, where a and b are the values ​​obtained by solving the system of two linear equations in two variables, v is the current feature information, and u is the current state value. For example, the current position coordinate v of each feature point on the eyelash (i.e., the coordinate of the feature point in the direction of movement) is substituted into the mapping function u = av + b to obtain the current state value u corresponding to the current feature information v.

[0120] In some other embodiments of this disclosure, the mapping relationship between feature information and state value can also be a mapping relationship table, and this disclosure does not limit the representation of the mapping relationship.

[0121] In some embodiments of this disclosure, the movement information includes the movement distance, and step S32 includes calculating the current state value and the limit deformation information to determine the movement distance of the feature points in the initial virtual image.

[0122] For example, multiplying the current state value and the limit deformation information can determine the movement distance of feature points in the initial virtual image.

[0123] For example, the limit deformation information is the limit deformation value described above. For feature point x1 in the upper eyelash, if the current state value is 0.5, then the moving distance of feature point x1 is the product of 0.5 and b1-y1.

[0124] Return to reference Figure 1A For step S40, for example, the camera of the electronic device detects the current state of the target feature of the detected object, and drives the feature points in the initial virtual image to move with the current state, so that the current virtual image displayed by the electronic device matches the current state of the target feature of the detected object.

[0125] In some embodiments of this disclosure, step S40 includes: moving a feature point in the target virtual sub-image a distance from its initial position to a position in at least one of at least one extreme state different from the initial state. For example, the initial state is the state displayed in the initial virtual image, and the position of the initial state is the position of the feature point in the initial virtual image.

[0126] For example, in the initial virtual image, the target virtual sub-image corresponding to the eyelash is the image when the eyes are fully open; that is, the initial position is the position of the eyelash in the target virtual sub-image when the eyes are fully open. After obtaining the movement information of the feature points in the target virtual sub-image corresponding to the eyelash, the feature points in the target virtual sub-image corresponding to the eyelash are driven to move from the position of the eyelash in the target virtual sub-image when the eyes are fully open to the position of the eyelash in the target virtual sub-image when the eyes are closed. That is, as shown... Figure 2C As shown, the feature points in the target virtual sub-image corresponding to the eyelashes are moved from the position shown in the target feature sub-image 231 to the position shown in the target feature sub-image 232.

[0127] In other embodiments of this disclosure, the movement information includes a target location, and step S40 includes: driving feature points in the initial virtual image to move to the target location.

[0128] For example, it drives the feature points in the target virtual sub-image corresponding to the eyelashes in the initial virtual image to move to the target position.

[0129] In some embodiments of this disclosure, the target position is calculated, for example, based on the initial position and the distance traveled.

[0130] In other embodiments of this disclosure, references above are made to... Figure 1A In addition to steps S10 to S50, the method shown may also include: calculating pitch angle, yaw angle and roll angle based on current feature information; calculating rotation matrix based on pitch angle, yaw angle and roll angle; driving feature points in the initial virtual image to move based on movement information; and controlling the rotation of the initial virtual image based on the rotation matrix to generate a current virtual image corresponding to the current state.

[0131] For example, if the detection target is a user, the pitch, yaw, and roll angles are calculated based on 280 key points of the user's face. The pitch angle is the angle of rotation of the face around a first axis, the yaw angle is the angle of rotation of the face around a second axis, and the roll angle is the angle of rotation of the face around a third axis. The first axis is perpendicular to the height direction of the face, the second axis is parallel to the height direction of the face, and the third axis is perpendicular to the first and second axes. The pitch, yaw, and roll angles are calculated based on the current feature information. The calculation of the rotation matrix based on the pitch, yaw, and roll angles can be referenced from relevant algorithms in related technologies, and will not be elaborated upon in this disclosure.

[0132] In this embodiment, while driving the feature points in the initial virtual image to move according to the movement information, the rotation of the initial virtual image is controlled according to the rotation matrix to generate the current virtual image corresponding to the current state.

[0133] For example, in response to the head rotation of the detected object, the current virtual image displayed by the electronic device changes from a virtual image of the detected object's front face to a virtual image of the detected object's side face after the head rotation.

[0134] In this embodiment, the rotation of the initial virtual image can be controlled according to the rotation matrix, making the current virtual image more realistic and vivid.

[0135] In embodiments of this disclosure, the current virtual image changes according to the current state of the target features of the detected object detected by the electronic device, thereby enabling the electronic device to display a virtual image that matches the current state of the target features of the detected object.

[0136] Figure 4 A schematic diagram illustrating the effect of an image processing method provided by at least one embodiment of the present disclosure is shown.

[0137] like Figure 4 As shown in the schematic diagram, the detection object is shown in state 401 at the first moment and state 402 at the second moment.

[0138] The effect diagram also includes the current virtual image 403 of the detected object displayed in the electronic device at the first moment, and the current virtual image 404 of the detected object displayed in the electronic device at the second moment.

[0139] like Figure 4 As shown, the eyes of the detected object are open and its mouth is closed at the first moment. Correspondingly, the eyes of the detected object in the current virtual image 403 displayed in the electronic device are also open and its mouth is closed.

[0140] like Figure 4As shown, at the second moment, the eyes of the object being detected are closed and the mouth is open. Correspondingly, at this time, the eyes of the object being detected in the current virtual image 404 displayed in the electronic device are also closed and the mouth is also open.

[0141] Figure 5 A schematic block diagram of an image processing apparatus 500 provided in at least one embodiment of the present disclosure is shown.

[0142] For example, such as Figure 5 As shown, the image processing apparatus 500 includes a detection unit 510, an acquisition unit 520, a determination unit 530, and a driving unit 540.

[0143] The detection unit 510 is configured to, in response to detecting the detection object, acquire current feature information of the detection object, wherein the current feature information is used to indicate the current state of the target features of the detection object. For example, the detection unit 510 may perform... Figure 1A Step S10 is described.

[0144] The acquisition unit 520 is configured to acquire the extreme deformation information of the target feature, which is calculated from the target virtual sub-image when the target feature is in its extreme state. For example, the acquisition unit 520 may execute... Figure 1A Step S20 is described.

[0145] The determining unit 530 is configured to determine the movement information of feature points in the initial virtual image based on the extreme deformation information and the current feature information. The initial virtual image is obtained by superimposing multiple virtual sub-images, and the multiple virtual sub-images include at least a portion of the target virtual sub-image. The determining unit 530 may, for example, perform... Figure 1A Step S30 is described.

[0146] The driving unit 540 is configured to drive feature points in the initial virtual image to move according to the movement information, so as to generate a current virtual image corresponding to the current state. The driving unit 540 may, for example, perform... Figure 1A Step S40 is described.

[0147] For example, the detection unit 510, acquisition unit 520, determination unit 530, and driving unit 540 can be hardware, software, firmware, or any feasible combination thereof. For example, the detection unit 510, acquisition unit 520, determination unit 530, and driving unit 540 can be dedicated or general-purpose circuits, chips, or devices, or a combination of a processor and memory. The embodiments of this disclosure do not limit the specific implementation of the above-mentioned units.

[0148] It should be noted that in the embodiments of this disclosure, each unit of the image processing device 500 corresponds to each step of the aforementioned image processing method. For the specific functions of the image processing device 500, please refer to the relevant description of the image processing method, which will not be repeated here. Figure 5 The components and structures of the image processing apparatus 500 shown are merely exemplary and not limiting. The image processing apparatus 500 may also include other components and structures as needed.

[0149] At least one embodiment of this disclosure also provides an electronic device including a processor and a memory, the memory including one or more computer program modules. The one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include instructions for implementing the image processing method described above. This electronic device can reduce the design and operation complexity of virtual avatars.

[0150] Figure 6 This is a schematic block diagram of an electronic device provided for some embodiments of this disclosure. For example... Figure 6 As shown, the electronic device 800 includes a processor 810 and a memory 820. The memory 820 stores non-transitory computer-readable instructions (e.g., one or more computer program modules). The processor 810 executes the non-transitory computer-readable instructions, which, when executed by the processor 810, can perform one or more steps in the image processing method described above. The memory 820 and the processor 810 can be interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0151] For example, processor 810 may be a central processing unit (CPU), a graphics processing unit (GPU), or other form of processing unit with data processing and / or program execution capabilities. For example, the central processing unit (CPU) may be an x86 or ARM architecture. Processor 810 may be a general-purpose processor or a special-purpose processor, capable of controlling other components in electronic device 800 to perform desired functions.

[0152] For example, memory 820 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and processor 810 may run one or more computer program modules to implement various functions of electronic device 800. Various application programs and various data, as well as various data used and / or generated by the application programs, may also be stored in the computer-readable storage medium.

[0153] It should be noted that, in the embodiments of this disclosure, the specific functions and technical effects of the electronic device 800 can be referred to the description of the image processing method above, and will not be repeated here.

[0154] Figure 7 This is a schematic block diagram of another electronic device provided in some embodiments of the present disclosure. The electronic device 900 is, for example, suitable for implementing the image processing method provided in the embodiments of the present disclosure. The electronic device 900 may be a terminal device, etc. It should be noted that... Figure 7 The illustrated electronic device 900 is merely an example and does not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0155] like Figure 7 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 910, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 920 or a program loaded from a storage device 980 into a random access memory (RAM) 930. The RAM 930 also stores various programs and data required for the operation of the electronic device 900. The processing device 910, the ROM 920, and the RAM 930 are interconnected via a bus 940. An input / output (I / O) interface 950 is also connected to the bus 940.

[0156] Typically, the following devices can be connected to I / O interface 950: input devices 960 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 970 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 980 including, for example, magnetic tapes, hard disks, etc.; and communication devices 990. Communication device 990 allows electronic device 900 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 7 An electronic device 900 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 900 may alternatively implement or have more or fewer devices.

[0157] For example, according to embodiments of this disclosure, the image processing method described above can be implemented as a computer software program. For instance, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program including program code for performing the image processing method described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 990, or installed from a storage device 980, or installed from a ROM 920. When the computer program is executed by the processing device 910, the functions defined in the image processing method provided by embodiments of this disclosure can be implemented.

[0158] At least one embodiment of this disclosure also provides a computer-readable storage medium for storing non-transitory computer-readable instructions that, when executed by a computer, can implement the image processing method described above. Using this computer-readable storage medium can reduce the design and operation complexity of virtual avatars.

[0159] Figure 8 This is a schematic diagram of a storage medium provided for some embodiments of this disclosure. For example... Figure 8 As shown, the storage medium 1000 is used to store non-transitory computer-readable instructions 1010. For example, when the non-transitory computer-readable instructions 1010 are executed by a computer, one or more steps in the image processing method described above can be performed.

[0160] For example, the storage medium 1000 can be used in the aforementioned electronic device 800. For example, the storage medium 1000 can be... Figure 6 The memory 820 in the illustrated electronic device 800. For example, a description of the storage medium 1000 can be found here. Figure 6 The corresponding description of the memory 820 in the illustrated electronic device 800 will not be repeated here.

[0161] The following points need to be explained:

[0162] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0163] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0164] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure should be determined by the scope of protection of the claims.

Claims

1. An image processing method, comprising: In response to the detection of a target object, the current feature information of the target object is obtained, wherein the current feature information is used to indicate the current state of the target feature of the target object; Obtain the extreme deformation information of the target feature, wherein the extreme deformation information is calculated from the target virtual sub-image when the target feature is in at least one extreme state; Based on the extreme deformation information and the current feature information, the movement information of feature points in the initial virtual image is determined, wherein the initial virtual image is obtained by superimposing multiple virtual sub-images, and the multiple virtual sub-images include target virtual sub-images corresponding to at least some of the extreme states in the at least one extreme state; Based on the movement information, the feature points in the initial virtual image are driven to move, so as to generate a current virtual image corresponding to the current state; The image processing method further includes: acquiring depth information of each of the plurality of virtual sub-images, and obtaining the initial virtual image based on the depth information of each of the plurality of virtual sub-images and the plurality of virtual sub-images. The process of obtaining the depth information of each of the plurality of virtual sub-images includes: for each virtual sub-image, dividing the virtual sub-image into a plurality of rectangular boxes, and setting a depth value for each vertex in the rectangular box.

2. The method according to claim 1, wherein, Obtaining the ultimate deformation information of the target feature includes: Based on the target virtual sub-image under the at least one extreme state, determine the first extreme position and the second extreme position of the target feature; Sampling is performed on the first and second extreme positions to obtain sampling results at multiple sampling points; and The sampling results are calculated to obtain the limit deformation information of the target feature.

3. The method according to claim 2, wherein, The target virtual sub-image in at least one extreme state includes a first layer in a first extreme state and a second layer in a second extreme state; Determining the first and second extreme positions of the target feature based on the target virtual sub-image under the at least one extreme state includes: The alpha channels of the first layer and the second layer are masked respectively to obtain two mask sub-images, and the two mask sub-images are merged into one mask image; as well as Based on the mask image, the first extreme position and the second extreme position are determined.

4. The method according to claim 3, wherein, The sampling result includes the position coordinates of each of the plurality of sampling points at the first extreme position and the second extreme position, respectively. The sampling results are calculated to obtain the limiting deformation information of the target feature, including: Based on the position coordinates of each sampling point at the first extreme position and the second extreme position, calculate the height difference between the first extreme position and the second extreme position for each sampling point. Based on the height difference, curve fitting is performed on the multiple sampling points to obtain the limit deformation value curve; Substitute each target vertex in the target feature into the limit deformation value curve to obtain the limit deformation value of each target vertex in the target feature, wherein each target vertex corresponds to at least some feature points in the initial virtual sub-image.

5. The method according to claim 4, wherein, The curve fitting includes polynomial fitting, and the limit deformation curve includes a polynomial curve.

6. The method according to claim 3, wherein, The sampling result includes the position coordinates of each of the plurality of sampling points at the first extreme position and the second extreme position, respectively. The sampling results are calculated to obtain the limiting deformation information of the target feature, including: Based on the position coordinates of each sampling point at the first extreme position and the second extreme position, calculate the height difference between the first extreme position and the second extreme position for each sampling point. The height difference between each sample at the first limit position and the second limit position is used as the limit deformation information.

7. The method according to any one of claims 1 to 6, wherein, Based on the extreme deformation information and the current feature information, the movement information of feature points in the initial virtual image is determined, including: Based on the current feature information, determine the current state value of the target feature relative to the reference state; and Based on the current state value and the extreme deformation information, the movement information of the feature points in the initial virtual image is determined.

8. The method according to claim 7, wherein, Determining the current state value of the target feature relative to the reference state based on the current feature information includes: Obtain the mapping relationship between feature information and state values; Based on the mapping relationship and the current feature information, the current state value of the target feature relative to the reference state is determined.

9. The method according to claim 8, wherein, Obtaining the mapping relationship between the feature information and the state value includes: Multiple samples are acquired, wherein each sample includes the correspondence between sample feature information of the target feature and sample state value; Based on the correspondence, a mapping function is constructed, wherein the mapping function represents the mapping relationship between the feature information and the state value.

10. The method according to claim 9, wherein, The sample feature information includes first feature information and second feature information, and the sample state value includes a first value corresponding to the first feature information and a second value corresponding to the second feature information. Based on the aforementioned correspondence, the mapping function is constructed, including: Construct a system of linear equations; The mapping function is obtained by substituting the first feature information and the first value, the second feature information and the second value into the system of linear equations and solving the system of linear equations.

11. The method according to claim 7, wherein, The movement information includes the distance traveled. Based on the current state value and the extreme deformation information, the movement information of the feature points in the initial virtual image is determined, including: The current state value and the limit deformation information are calculated to determine the movement distance of the feature points in the initial virtual image.

12. The method according to claim 11, wherein, Calculating the current state value and the limit deformation information to determine the movement distance of the feature points in the initial virtual image includes: The current state value and the limit deformation information are multiplied to determine the movement distance of the feature points in the initial virtual image.

13. The method according to claim 1, wherein, The movement information includes the distance traveled. Based on the movement information, driving the feature points in the initial virtual image to move includes: The feature point in the target virtual sub-image of the initial virtual image is driven to move by the distance from its initial position to at least one of the extreme states different from the initial state.

14. The method according to claim 1, wherein, The movement information includes the target location. Based on the movement information, driving the feature points in the initial virtual image to move includes: Drive the feature points in the initial virtual image to move to the target position.

15. The method according to claim 1, wherein, The current feature information includes comparison information between the target feature and the reference feature of the detected object, wherein the comparison information remains unchanged when the distance between the detected object and the image acquisition device used to detect the detected object changes.

16. The method according to claim 1, wherein, The depth information of each of the plurality of virtual sub-images includes the depth value of each virtual sub-image and the depth value of the feature points in each virtual sub-image. Each virtual sub-image corresponds to one of a plurality of virtual features to be virtualized of the detected object, the plurality of virtual features including the target feature. Wherein, in the direction perpendicular to the face of the detected object, the depth value of each virtual sub-image is proportional to a first distance, where the first distance is the distance between the virtual feature corresponding to the virtual sub-image and the eye of the detected object; and The depth value of the feature points in each virtual sub-image is proportional to the first distance.

17. The method according to claim 1, wherein, The target features include at least one of eyelashes and the mouth. In the case where the target feature is an eyelash, the limiting state of the target feature is the state of the target feature when the eyes of the detected object are open. In the case where the target feature is the mouth, the extreme state of the target feature is the state of the mouth when the mouth is opened to its maximum extent.

18. An image processing apparatus, comprising: The detection unit is configured to, in response to the detection of a detection object, acquire the current feature information of the detection object, wherein the current feature information is used to indicate the current state of the target features of the detection object; The acquisition unit is configured to acquire the extreme deformation information of the target feature, wherein the extreme deformation information is calculated from the target virtual sub-image when the target feature is in an extreme state; The determining unit is configured to determine the movement information of feature points in an initial virtual image based on the extreme deformation information and the current feature information, wherein the initial virtual image is obtained by superimposing multiple virtual sub-images, and the multiple virtual sub-images include at least a portion of the target virtual sub-image; The driving unit is configured to drive the feature points in the initial virtual image to move according to the movement information, so as to generate a current virtual image corresponding to the current state; The image processing device further includes a depth unit configured to: acquire depth information of each of the plurality of virtual sub-images, and obtain the initial virtual image based on the depth information of each of the plurality of virtual sub-images and the plurality of virtual sub-images. The depth unit is further configured to: for each virtual sub-image, divide the virtual sub-image into multiple rectangular boxes, and set a depth value for each vertex in the rectangular box.

19. An electronic device comprising: processor; Memory, which includes one or more computer program instructions; The one or more computer program instructions are stored in the memory and, when executed by the processor, implement the image processing method according to any one of claims 1-17.

20. A computer-readable storage medium that non-transitoryly stores computer-readable instructions, wherein, The image processing method according to any one of claims 1-17 is implemented when the computer-readable instructions are executed by a processor.

Images