Method, apparatus, readable medium, and program product for driving facial expressions

Abstract

The present application provides a method, device, readable medium and program product for driving facial expressions. The method according to the present application includes: in response to an expression driving request of a target object, determining at least one target bone corresponding to the facial expression to be driven; obtaining the bone deformation information of the at least one target bone generated by the face pinching operation; based on a preset moving distance threshold of the at least one target bone, determining the driving movement range of the at least one target bone during the expression driving process; performing expression driving processing based on the driving movement range to drive the virtual character corresponding to the target object to present the facial expression, and the expression driving processing makes the movement trajectory of the at least one target bone during the expression driving process not exceed the driving movement range. The present application avoids abnormal driving effects such as closed-eye penetration caused by the face pinching operation, thereby improving the driving effect.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, computer-readable medium, and computer program product for driving facial expressions. Background Art

[0002] With the widespread use of virtual characters in games, movies, and animation, the requirements for facial expressions are becoming increasingly demanding. Traditional facial rigging methods are mainly based on simple facial skeletons and deformation blending (Blendshape). When using Unreal Engine (UE) to drive facial expressions, skeleton face pinching and Blendshape are generally used.

[0003] However, because Blendshape creation relies on a neutral facial base, once the face is pinched, the base is changed. Large movements during the driving process can easily cause clipping or other issues, and the driving effect needs to be improved. For example, if you pinch the eyes to be larger and then use Blendshape to drive them when they are closed, the glasses may not close. Summary of the Invention

[0004] Various aspects of the present application provide a method, apparatus, computer-readable medium, and computer program product for driving facial expressions.

[0005] In one aspect of the present application, a method for driving facial expressions is provided, wherein the method comprises:

[0006] In response to an expression driving request of a target object, determining at least one target bone corresponding to a facial expression to be driven;

[0007] Obtaining bone deformation information of the at least one target bone generated by the face pinching operation;

[0008] Determining a driving movement range of the at least one target bone during expression driving based on a preset movement distance threshold of the at least one target bone;

[0009] Expression driving processing is performed based on the driving movement range to drive the virtual character corresponding to the target object to present the facial expression, and the expression driving processing ensures that the movement trajectory of at least one target bone during the expression driving process does not exceed the driving movement range.

[0010] In one aspect of the present application, a device for driving facial expressions is provided, wherein the device comprises:

[0011] means for determining, in response to an expression driving request of a target object, at least one target bone corresponding to a facial expression to be driven;

[0012] means for obtaining bone deformation information of the at least one target bone generated by the face pinching operation;

[0013] A device for determining a driving movement range of the at least one target bone during expression driving based on a preset movement distance threshold of the at least one target bone;

[0014] A device for performing expression driving processing based on the driving movement range to drive the virtual character corresponding to the target object to present the facial expression, wherein the expression driving processing ensures that the movement trajectory of at least one target bone during the expression driving process does not exceed the driving movement range.

[0015] Another aspect of the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of the embodiment of the application.

[0016] In another aspect of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored. The computer program instructions can be executed by a processor to implement the method of the embodiment of the application.

[0017] In another aspect of the present application, a computer program product is provided, including a computer program, which implements the method of the embodiment of the present application when executed by a processor.

[0018] The solution provided in the embodiment of the present application determines the movable range of the skeleton during the expression driving process through the preset movement distance threshold of the skeleton and the movement distance generated by the face pinching operation of the skeleton, and performs expression driving based on the movable range, so that the movement trajectory of the skeleton during the expression driving process does not exceed the range, thereby avoiding abnormal driving effects such as closed-eye penetration due to the face pinching operation, and improving the driving effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, a brief introduction will be given below to the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0021] Figure 1 A schematic flow chart of a method for driving facial expressions provided in an embodiment of the present application is shown;

[0022] Figure 2 A schematic structural diagram of a device for driving facial expressions provided in an embodiment of the present application is shown;

[0023] Figure 3 A structural diagram of a device suitable for implementing the solution in the embodiments of the present application is shown.

[0024] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0025] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] In a typical configuration of the present application, the terminal and the equipment of the service network each include one or more processors (CPUs), input / output interfaces, network interfaces and memories.

[0027] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0028] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for information storage. The information can be computer program instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc-read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0029] The following describes the terms involved in the embodiments of this application.

[0030] Blendshape: A digital animation technique used to create 3D character animations, Blendshape allows users to precisely control a character's facial expressions to create realistic facial animations. In a 3D model, a character's face is composed of a large number of vertices. By creating mesh data in a consistent number but inconsistent shapes, a simple linear transformation model is formed. For example, each facial expression can be composed of the same number of vertices, and the positions of these vertices remain the same from one facial expression to another. This makes interpolation and deformation between different facial expressions much easier and more precise.

[0031] Skeleton: A skeleton is a virtual three-dimensional skeletal structure used to control the movements of a 3D character. Skeletons can be composed of multiple joints and bones, each of which has positional and rotational information. During the creation process, users can control the pose and movement of a 3D model by moving, rotating, and scaling the skeleton. For example, by driving the skeleton, you can control the full-body movements of a character, such as walking, running, and jumping.

[0032] Binding: Binding refers to the process of binding points on the surface of a model (such as vertices, etc.) to bones, etc. Binding can connect the surface of a character model to its bones or other controllers to achieve the purpose of bones controlling the character's movements. During the binding process, each point can be assigned to the closest bone or controller, and the character's movements are controlled by moving the bones or controllers, and the points on the character's surface will also move with it. Binding is performed after the skeleton animation or other controller animation is completed, and can be done using various binding tools or manually. After binding is completed, the character model can be animated, and various character movements can be achieved by moving the bones or controllers. Binding can be regarded as the entire process of making bones and skinning.

[0033] Skinning: Skinning is the process of connecting the character model's surface to its skeleton. Skinning allows for more natural and fluid character movement, as well as realistic skin and muscle deformation during motion. During skinning, each point is assigned to the closest skeleton. Interpolation and other techniques are then used to calculate surface deformation during the character's motion, achieving smooth and natural movement.

[0034] Figure 1 A flow chart of a method for data processing and transmission provided in an embodiment of the present application is shown, wherein the method comprises at least step S101, step S102, step S103 and step S104.

[0035] The method of the embodiment of the present application can be applied to various scenarios that require driving a virtual human to present corresponding facial expressions, such as face pinching scenarios, virtual character editing scenarios, etc. More specifically, the method of the embodiment of the present application can be used to edit a virtual character in a game and drive the virtual character in the game to present the desired facial expressions.

[0036] In actual scenarios, the execution subject of this method can be a user device, or it can be an application running on the user device. The user device includes but is not limited to computers, mobile phones, tablets, smart watches, bracelets and other terminal devices.

[0037] Optionally, the method of the embodiment of the present application is performed by an electronic device that supports the Unreal Engine (UE), which may be a terminal device or a server. The terminal device includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, a virtual reality (VR) device, an augmented reality (AR) device, etc. The server may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server.

[0038] The information (including but not limited to object device information, object account information, object operation information, etc.), data (including but not limited to stored data, object feature data, etc.) and signals involved in the embodiments of this application are all authorized by the relevant objects or fully authorized by all parties, and the collection, use and processing of relevant data comply with the relevant laws, regulations and standards of relevant countries and regions.

[0039] The method further includes step S105 and step S106 before step S101.

[0040] In step S105, based on one or more face pinching operations, skeletal deformation information of at least one bone deformed by the face pinching operation is obtained, where the skeletal deformation information includes a bone movement direction and a bone movement distance.

[0041] The face pinching operation includes face pinching processing operations for multiple facial organs. Optionally, the face pinching processing operations include but are not limited to adjusting the face shape, adjusting the size or position of facial features, etc.

[0042] Specifically, the method obtains the skeleton deformation information of at least one skeleton deformed by the face pinching operation based on the initial face pinching skeleton and the position information of each skeleton in the target face pinching skeleton obtained by the face pinching operation.

[0043] Optionally, the face pinching parameters of the target face pinching skeleton are subsequently read, and face pinching slider information is generated based on the face pinching parameters, so that the production staff can understand the animation posture of the current target face pinching skeleton through the face pinching slider information.

[0044] In step S106, the bone deformation information of the at least one bone is stored.

[0045] Optionally, the method executes steps S105 and S106 to obtain and store corresponding skeletal deformation information after each face pinching operation is completed. Alternatively, the method executes steps S105 and S106 to obtain and store corresponding skeletal deformation information after confirming that all face pinching operations are completed.

[0046] Refer to the following Figure 1 To illustrate, in step S101 , in response to an expression driving request of a target object, at least one target bone corresponding to a facial expression to be driven is determined.

[0047] The target object is, but is not limited to, a user or the user's face photographed by a camera of a terminal device, a user's face contained in a picture captured by the camera of the terminal device, and the like.

[0048] The expression drive request is used to request driving the facial expression of the target object so that the facial expression is presented by the virtual character corresponding to the target object. The facial expression includes but is not limited to blinking, smiling, laughing, pouting, etc.

[0049] Optionally, the facial expression to be driven may be a facial expression of a target object collected in real time.

[0050] Optionally, the facial expression to be driven is a matching basic expression obtained by matching the facial expression collected in real time with a plurality of pre-stored basic expressions.

[0051] Among them, those skilled in the art should be familiar with that expression matching can be performed in a variety of ways, for example, performing face detection on the target object to obtain the position information of several key points on the face to match the key points corresponding to each pre-stored basic expression, and then obtaining a matching basic expression, which will not be repeated here.

[0052] In step S102, the bone deformation information of the at least one target bone generated by the face pinching operation is obtained.

[0053] Specifically, based on the pre-stored bone deformation information of each bone deformed by the face pinching operation, it is determined whether the target bone is deformed, and the bone movement direction and bone movement distance of the deformed target bone are obtained.

[0054] In step S103, based on a preset moving distance threshold of the at least one target bone, a driving moving range of the at least one target bone during the expression driving process is determined.

[0055] The movement distance threshold is used to indicate the maximum distance that each bone can move in a specific expression during the face pinching and driving process.

[0056] For example, suppose the two eyes correspond to two bones, one for the upper eyelid and one for the lower eyelid. Furthermore, in the face-pinching tool, the rigger uses facial expression standards to control the specific displacement of a certain action bone. For the "eyes closed" expression, the upper eyelid bone needs to be displaced downward, and the lower eyelid bone needs to be displaced upward, with the degree of displacement just enough to close the eyelids. The displacement of the upper and lower eyelid bones from the open eye state to the closed eyelid state is used as the preset movement distance threshold for the two bones.

[0057] The method further includes step S107 before step S103.

[0058] In step S107, a preset moving distance threshold of the at least one target skeleton corresponding to the facial expression to be driven is obtained.

[0059] Optionally, the preset movement distance threshold of the at least one target skeleton corresponding to the facial expression to be driven is obtained from a database for storing preset movement distance thresholds of various skeletons.

[0060] Optionally, the movement distance threshold corresponds to different pinching face shapes. In step S107, the method obtains the facial expression to be driven and the movement distance threshold of the pinching expression preset by the at least one target bone based on the pinching face shape obtained by the pinching operation.

[0061] According to one embodiment, the driving movement range includes information indicating one or more directions in which the bone can move and a maximum distance in which the bone can move in the one or more directions.

[0062] According to step S103 of this embodiment, the following steps are included: for each target bone, the preset moving distance threshold of the target bone is used as the maximum value of the sum of the displacement caused by the face pinching operation and the displacement caused by the expression driving, thereby determining the driving movement range of the target bone during the expression driving process based on the maximum value and the bone deformation information of the target bone deformed by the face pinching operation.

[0063] According to one embodiment, the method determines a standard distance based on a preset moving distance threshold of the target bone, and determines the moving distance of the target bone based on the standard distance.

[0064] Optionally, the method maps the length transformation of the target bone movement to a standard interval between 0 and 1. Specifically, the preset movement distance threshold of the target bone is set to 1, and the initial position of the target bone in the base skeleton corresponds to 0, so that the displacement caused by the face pinching operation and the displacement caused by the expression drive are limited to the standard interval between 0 and 1.

[0065] For example, using the standard range of 0 to 1 above to calculate the target bone's movement distance, assuming the target bone's movement distance generated by the face pinching operation is 0.7, then during the expression driving process, the target bone's movement distance in the same direction should be limited to 0.3, that is, the movable range in that direction is 0.3. Assuming that the target bone's movement distance generated by the face pinching operation has reached 1, that is, the target bone's movement range has reached the maximum value, then during the driving process, the target bone cannot move in that direction.

[0066] In step S104, expression driving processing is performed based on the driving movement range to drive the virtual character corresponding to the target object to present the facial expression.

[0067] The expression driving process ensures that the movement trajectory of the at least one target bone during the expression driving process does not exceed the driving movement range.

[0068] According to one embodiment, step S104 includes steps S1041 to S1043.

[0069] In step S1041, a target movement distance of the at least one target skeleton after the driving process is obtained.

[0070] In step S1042, it is determined whether the movement trajectory of the at least one target bone exceeds the driving movement range.

[0071] In step S1043, if the driving movement range is exceeded, the target movement distance of the corresponding target bone is corrected, and expression driving is performed based on the corrected movement distance. If the driving movement range is not exceeded, expression driving is performed based on the target movement distance.

[0072] For example, for the expression to be driven from the normal eye shape to the closed eye shape, the maximum bone displacement of the upper eyelid bone in the vertical direction pre-set by the binding artist is used as the standard distance. When calculating the movement distance of the upper eyelid bone later, the standard distance is used as 1 to calculate the distance.

[0073] For the case of pinching the eyes smaller during the previous face pinching operation, assuming the vertical deformation distance of the eyes is 0.3 of the standard distance, which is equivalent to the eyes being closed 0.2, the maximum vertical closing distance of the bone should be limited to 0.7 during the driving process to avoid excessive eye closing movements that may cause the model to be clipped.

[0074] As for the case of pinching the eyes larger, assuming that the deformation distance of pinching the eyes is 0.3 of the standard distance, since pinching and closing the eyes displace the bones in opposite directions, it is equivalent to a displacement of -0.3. Therefore, when driving, the maximum closing distance of the bone in the vertical direction is limited to 1.3 to avoid the situation where the eyes cannot be closed after pinching.

[0075] According to one embodiment, the method controls the movement distance of the face-pinching bones during the face-pinching process based on a preset movement distance threshold of the face-pinching bones. The face-pinching bones are one or more bones that are deformed by the face-pinching operation.

[0076] The method of this embodiment further includes step S108 and step S109.

[0077] In step S108, based on the deformation-related information of the face-pinching skeleton, it is determined whether the movement distance of the face-pinching skeleton exceeds a preset movement distance threshold.

[0078] In step S109, if the movement distance of a face-pinching bone exceeds its preset movement distance threshold, the preset movement distance threshold of the face-pinching bone is used as the movement distance generated by the face-pinching operation, so that the movement distance of the face-pinching bone during the face-pinching process does not exceed the movement distance threshold.

[0079] Among them, the process of controlling the movement distance of the face-pinching bone by pinching the face based on the preset movement distance threshold of the face-pinching bone in step S108 and step S109 is similar to the process of the above-mentioned steps S1041 to step S1043, and will not be repeated here.

[0080] According to one embodiment, during the face-pinching process, the method controls the movement distance of a bone due to rotation, displacement, scaling, and other operations based on a preset movement distance threshold for the bone by executing steps S108 and S109. Furthermore, during the expression-driven process, the method controls the movement distance of the bone during the expression-driven process by executing steps S101 to S104, so that the sum of the movement distances of a bone after the face-pinching operation and expression-driven operation does not exceed the preset movement distance threshold.

[0081] According to the method of the embodiment of the present application, the movable range of the skeleton during the expression driving process is determined by the preset movement distance threshold of the skeleton and the movement distance of the skeleton generated by the face pinching operation, and expression driving is performed based on the movable range, so that the movement trajectory of the skeleton during the expression driving process does not exceed the range, thereby avoiding abnormal driving effects such as closed-eye penetration due to the face pinching operation, and improving the driving effect.

[0082] In addition, the embodiment of the present application also provides a device for driving facial expressions, the structure of the device is as follows: Figure 2 shown.

[0083] The device may be included in an electronic device that supports the Unreal Engine (UE) and executed, and the electronic device may be a terminal device or a server.

[0084] The device includes: a device for determining at least one target bone corresponding to the facial expression to be driven in response to an expression drive request of a target object (hereinafter referred to as "bone determination device 101"); a device for obtaining bone deformation information generated by the at least one target bone through a face pinching operation (hereinafter referred to as "deformation acquisition device 102"); a device for determining the driving movement range of the at least one target bone during the expression driving process based on a preset movement distance threshold of the at least one target bone (hereinafter referred to as "movement range determination device 103"); and a device for performing expression drive processing based on the driving movement range to drive the virtual character corresponding to the target object to present the facial expression (hereinafter referred to as "drive processing device 104").

[0085] The device further includes a face-pinching deformation acquisition device and a deformation information storage device. Before describing the operation of the skeleton determination device 101, the operations of the face-pinching deformation acquisition device and the deformation information storage device are first described.

[0086] The face pinching deformation acquisition device acquires bone deformation information of at least one bone deformed by the face pinching operation based on one or more face pinching operations, and the bone deformation information includes the bone movement direction and the bone movement distance.

[0087] The face pinching operation includes face pinching processing operations for multiple facial organs. Optionally, the face pinching processing operations include but are not limited to adjusting the face shape, adjusting the size or position of facial features, etc.

[0088] Specifically, the face pinching deformation acquisition device obtains the skeleton deformation information of at least one skeleton deformed by the face pinching operation through the position information of each skeleton in the target face pinching skeleton obtained based on the initial face pinching skeleton and the face pinching operation.

[0089] Optionally, the face pinching deformation acquisition device subsequently reads the face pinching parameters of the target face pinching skeleton and generates face pinching slider information based on the face pinching parameters, so that the production staff can understand the animation posture of the current target face pinching skeleton through the face pinching slider information.

[0090] The deformation information storage device stores the bone deformation information of the at least one bone.

[0091] Optionally, after each face pinching operation is completed, the face pinching deformation acquisition device and the deformation information storage device are operated to acquire and store the corresponding skeletal deformation information. Alternatively, after confirming that all face pinching operations are completed, the face pinching deformation acquisition device and the deformation information storage device are operated to acquire and store the corresponding skeletal deformation information.

[0092] Refer to the following Figure 2 To illustrate, the skeleton determining device 101 determines at least one target skeleton corresponding to the facial expression to be driven in response to the expression driving request of the target object.

[0093] The target object is, but is not limited to, a user or the user's face photographed by a camera of a terminal device, a user's face contained in a picture captured by the camera of the terminal device, and the like.

[0094] The expression drive request is used to request driving the facial expression of the target object so that the facial expression is presented by the virtual character corresponding to the target object. The facial expression includes but is not limited to blinking, smiling, laughing, pouting, etc.

[0095] Optionally, the facial expression to be driven may be a facial expression of a target object collected in real time.

[0096] Optionally, the facial expression to be driven is a matching basic expression obtained by matching the facial expression collected in real time with a plurality of pre-stored basic expressions.

[0097] Among them, those skilled in the art should be familiar with that expression matching can be performed in a variety of ways, for example, performing face detection on the target object to obtain the position information of several key points on the face to match the key points corresponding to each pre-stored basic expression, and then obtaining a matching basic expression, which will not be repeated here.

[0098] The deformation acquisition device 102 acquires the bone deformation information of the at least one target bone generated by the face pinching operation.

[0099] Specifically, based on the pre-stored bone deformation information of each bone deformed by the face pinching operation, it is determined whether the target bone is deformed, and the bone movement direction and bone movement distance of the deformed target bone are obtained.

[0100] The movement range determining device 103 determines the driving movement range of the at least one target bone during the expression driving process based on a preset movement distance threshold of the at least one target bone.

[0101] The movement distance threshold is used to indicate the maximum distance that each bone can move in a specific expression during the face pinching and driving process.

[0102] For example, suppose the two eyes correspond to two bones, one for the upper eyelid and one for the lower eyelid. Furthermore, in the face-pinching tool, the rigger uses facial expression standards to control the specific displacement of a certain action bone. For the "eyes closed" expression, the upper eyelid bone needs to be displaced downward, and the lower eyelid bone needs to be displaced upward, with the degree of displacement just enough to close the eyelids. The displacement of the upper and lower eyelid bones from the open eye state to the closed eyelid state is used as the preset movement distance threshold for the two bones.

[0103] Wherein, the method further includes a threshold value acquisition device.

[0104] Before the operation of the movement range determination device 103, the threshold value acquisition device acquires the movement distance threshold value preset by the at least one target skeleton corresponding to the facial expression to be driven.

[0105] Optionally, the threshold value obtaining device obtains the preset movement distance threshold value of the at least one target skeleton corresponding to the facial expression to be driven from a database for storing preset movement distance threshold values ​​of various skeletons.

[0106] Optionally, the movement distance threshold corresponds to different pinching face shapes. The threshold acquisition device acquires the movement distance threshold corresponding to the facial expression to be driven and the pinching expression preset by the at least one target bone based on the pinching face shape obtained by the pinching operation.

[0107] According to one embodiment, the driving movement range includes information indicating one or more directions in which the bone can move and a maximum distance in which the bone can move in the one or more directions.

[0108] According to this embodiment, for each target bone, the movement range determination device 103 uses the preset movement distance threshold of the target bone as the maximum value of the sum of the displacement caused by the face pinching operation and the displacement caused by the expression driving, thereby determining the driving movement range of the target bone during the expression driving process based on the maximum value and the bone deformation information of the target bone deformed by the face pinching operation.

[0109] According to one embodiment, the movement range determining device 103 determines a standard distance based on a preset movement distance threshold of the target bone, and determines the movement distance of the target bone based on the standard distance.

[0110] Optionally, the device maps the length transformation of the target bone movement to a standard range of 0 to 1. Specifically, the preset movement distance threshold of the target bone is set to 1, and the initial position of the target bone in the base skeleton corresponds to 0, so that the displacement caused by the face pinching operation and the displacement caused by the expression drive are limited to the standard range of 0 to 1.

[0111] For example, using the standard range of 0 to 1 above to calculate the target bone's movement distance, assuming the target bone's movement distance generated by the face pinching operation is 0.7, then during the expression driving process, the target bone's movement distance in the same direction should be limited to 0.3, that is, the movable range in that direction is 0.3. Assuming that the target bone's movement distance generated by the face pinching operation has reached 1, that is, the target bone's movement range has reached the maximum value, then during the driving process, the target bone cannot move in that direction.

[0112] The driving processing device 104 performs expression driving processing based on the driving movement range to drive the virtual character corresponding to the target object to present the facial expression.

[0113] The expression driving process ensures that the movement trajectory of the at least one target bone during the expression driving process does not exceed the driving movement range.

[0114] According to one embodiment, the driving processing device 104 includes a driving distance acquisition device, an out-of-range determination device, and a correction processing device.

[0115] The driving distance acquiring device acquires the target moving distance that the at least one target bone will generate after the driving process.

[0116] The out-of-range determining device determines whether the movement trajectory of the at least one target bone exceeds the driving movement range.

[0117] If it exceeds the driving movement range, the correction processing device corrects the target movement distance of the corresponding target bone, thereby performing expression driving based on the corrected movement distance. If it does not exceed the driving movement range, expression driving is performed based on the target movement distance.

[0118] For example, for the expression to be driven from the normal eye shape to the closed eye shape, the maximum bone displacement of the upper eyelid bone in the vertical direction pre-set by the binding artist is used as the standard distance. When calculating the movement distance of the upper eyelid bone later, the standard distance is used as 1 to calculate the distance.

[0119] For the case of pinching the eyes smaller during the previous face pinching operation, assuming the vertical deformation distance of the eyes is 0.3 of the standard distance, which is equivalent to the eyes being closed 0.2, the maximum vertical closing distance of the bone should be limited to 0.7 during the driving process to avoid excessive eye closing movements that may cause the model to be clipped.

[0120] As for the case of pinching the eyes larger, assuming that the deformation distance of pinching the eyes is 0.3 of the standard distance, since pinching and closing the eyes displace the bones in opposite directions, it is equivalent to a displacement of -0.3. Therefore, when driving, the maximum closing distance of the bone in the vertical direction is limited to 1.3 to avoid the situation where the eyes cannot be closed after pinching.

[0121] According to one embodiment, the method controls the movement distance of the face-pinching bone during the face-pinching process based on a preset movement distance threshold of the face-pinching bone, wherein the face-pinching bone is one or more bones that are deformed by the face-pinching operation.

[0122] The method of this embodiment also includes a face pinching out-of-range determination device and a face pinching distance control device.

[0123] The face pinching out-of-range determining device determines whether the movement distance of the face pinching bone exceeds a preset movement distance threshold based on the deformation related information of the face pinching bone.

[0124] If the movement distance of a face-pinching bone exceeds its preset movement distance threshold, the face-pinching distance control device uses the preset movement distance threshold of the face-pinching bone as the movement distance generated by the face-pinching operation, so that the movement distance of the face-pinching bone during the face-pinching process does not exceed the movement distance threshold.

[0125] Among them, the process of the face pinching out-of-range determination device and the face pinching distance control device controlling the movement distance generated by the face pinching bone through face pinching based on the preset movement distance threshold of the face pinching bone is similar to the operation of the above-mentioned driving processing device 104, and will not be repeated here.

[0126] According to one embodiment, during the face pinching process, the device controls the movement distance of a bone due to rotation, displacement, scaling, and other operations based on a preset movement distance threshold for the bone by executing the operations of the face pinching out-of-range determination device and the face pinching distance control device. Furthermore, during the expression driving process, the device controls the movement distance of the bone during the expression driving process by executing the operations of the bone determination device 101, the deformation acquisition device 102, the movement range determination device 103, and the drive processing device 104, so that the sum of the movement distances of a bone after the face pinching operation and expression driving does not exceed the preset movement distance threshold.

[0127] According to the method of the embodiment of the present application, the movable range of the skeleton during the expression driving process is determined by the preset movement distance threshold of the skeleton and the movement distance of the skeleton generated by the face pinching operation, and expression driving is performed based on the movable range, so that the movement trajectory of the skeleton during the expression driving process does not exceed the range, thereby avoiding abnormal driving effects such as closed-eye penetration due to the face pinching operation, and improving the driving effect.

[0128] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present application. The method corresponding to the electronic device may be the method for driving facial expressions in the aforementioned embodiment, and its principle of solving the problem is similar to that of the method. The electronic device provided in an embodiment of the present application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of the aforementioned multiple embodiments of the present application.

[0129] The electronic device may be a user device, or a device formed by integrating a user device and a network device via a network, or an application running on the above device. The user device includes but is not limited to various terminal devices such as computers, mobile phones, tablets, smart watches, and bracelets. The network device includes but is not limited to network hosts, single network servers, multiple network server sets, or cloud computing-based computer collections, and can be used to implement some of the processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing (Cloud Computing), where cloud computing is a type of distributed computing, a virtual computer composed of a group of loosely coupled computers.

[0130] Figure 3 The structure of a device suitable for implementing the method and / or technical solution in the embodiment of the present application is shown. The device 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1202 or the program loaded from the storage part 1208 into the random access memory (RAM) 1203. Various programs and data required for system operation are also stored in RAM 1203. CPU 1201, ROM 1202 and RAM 1203 are connected to each other through a bus 1204. Input / output (I / O) interface 1205 is also connected to bus 1204.

[0131] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, a touch screen, a microphone, an infrared sensor, and the like; an output section 1207 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), an LED display, an OLED display, and a speaker; a storage section 1208 including one or more computer-readable media such as a hard disk, an optical disk, a magnetic disk, and a semiconductor memory; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication section 1209 performs communication processing via a network such as the Internet.

[0132] In particular, the methods and / or embodiments of the present application can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the method shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 1201, the above-mentioned functions defined in the method of the present application are performed.

[0133] Another embodiment of the present application further provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of the present application.

[0134] Specifically, the present embodiment can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0135] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0136] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0137] Computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0138] The flow chart or block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the equipment, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code include one or more executable instructions for realizing the logical function of the specification. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs the function or operation of the specification, or can be implemented with a combination of dedicated hardware and computer instructions.

[0139] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or page components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0141] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0142] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0143] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program code.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0145] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device through software or hardware. Terms such as "first" and "second" are used to indicate names and do not imply any particular order.

Claims

1. A method for driving facial expressions, wherein: The method comprises: In response to an expression driving request of a target object, determining at least one target bone corresponding to a facial expression to be driven; Obtaining bone deformation information of the at least one target bone generated by the face pinching operation; Determining a driving movement range of the at least one target bone during expression driving based on a preset movement distance threshold of the at least one target bone; Performing expression driving processing based on the driving movement range to drive the virtual character corresponding to the target object to present the facial expression, wherein the expression driving processing ensures that the movement trajectory of the at least one target bone during the expression driving process does not exceed the driving movement range; The step of determining the driving movement range of the at least one target bone during the expression driving process based on a preset movement distance threshold of the at least one target bone includes: For each target bone, the preset moving distance threshold of the target bone is used as the maximum value of the sum of the displacement caused by the face pinching operation and the displacement caused by the expression driving, so as to determine the driving movement range of the target bone during the expression driving process based on the maximum value and the bone deformation information of the target bone deformed by the face pinching operation.

2. The method according to claim 1, wherein The method further comprises: Based on one or more face pinching operations, obtaining bone deformation information of at least one bone deformed by the face pinching operation, the bone deformation information including a bone movement direction and a bone movement distance; The bone deformation information of the at least one bone is stored.

3. The method according to claim 1 or 2, wherein: The performing expression driving processing based on the driving movement range to drive the virtual character corresponding to the target object to present the facial expression includes: Obtaining a target movement distance of the at least one target bone after the driving process; Based on the target movement distance, determining whether the movement trajectory of the at least one target bone exceeds the driving movement range; If it exceeds the driving movement range, the target movement distance of the corresponding target bone is corrected, and expression driving is performed based on the corrected movement distance. If it does not exceed the driving movement range, expression driving is performed based on the target movement distance.

4. The method according to claim 1 or 2, wherein: The method controls the movement distance of the face-pinching bone generated by face-pinching based on a preset movement distance threshold of the target bone during the face-pinching process, and the method further includes: Determining whether the movement distance of the face-pinching bone exceeds a preset movement distance threshold based on the deformation-related information of the face-pinching bone; If the movement distance of a face-pinching bone exceeds its preset movement distance threshold, the preset movement distance threshold of the face-pinching bone will be used as the movement distance generated by the face-pinching operation, so that the movement distance of the face-pinching bone during the face-pinching process does not exceed the movement distance threshold.

5. The method according to claim 1 or 2, wherein: The method determines a standard distance based on a preset moving distance threshold of the target bone, and determines the moving distance of the target bone based on the standard distance.

6. The method according to claim 1 or 2, wherein: The method further comprises: Obtain a movement distance threshold preset for the at least one target bone corresponding to the facial expression to be driven.

7. The method according to claim 6, wherein: The moving distance threshold corresponds to different face shapes for pinching, and the obtaining of the moving distance threshold corresponding to the facial expression to be driven preset by the at least one target bone includes: Based on the pinched face shape obtained by the pinching operation, obtain the facial expression to be driven and the movement distance threshold of the pinching operation preset by the at least one target bone.

8. A device for driving facial expressions, wherein: The device comprises: means for determining, in response to an expression driving request of a target object, at least one target bone corresponding to a facial expression to be driven; means for obtaining bone deformation information of the at least one target bone generated by the face pinching operation; A device for determining a driving movement range of the at least one target bone during expression driving based on a preset movement distance threshold of the at least one target bone; A device for performing expression driving processing based on the driving movement range to drive the virtual character corresponding to the target object to present the facial expression, wherein the expression driving processing ensures that the movement trajectory of the at least one target bone during the expression driving process does not exceed the driving movement range; The device for determining the driving movement range of the at least one target bone during the expression driving process based on a preset movement distance threshold of the at least one target bone is used to: For each target bone, the preset moving distance threshold of the target bone is used as the maximum value of the sum of the displacement caused by the face pinching operation and the displacement caused by the expression driving, so as to determine the driving movement range of the target bone during the expression driving process based on the maximum value and the bone deformation information of the target bone deformed by the face pinching operation.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable medium having computer program instructions stored thereon, wherein the computer program instructions can be executed by a processor to implement the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images