A method, device and storage medium for determining virtual hand action

Abstract

The application provides a method for determining virtual human hand action, a device for determining virtual human hand action and a computer readable storage medium. The method for determining virtual human hand action comprises the following steps: obtaining a broadcast text of a virtual human; performing semantic analysis on the broadcast text to determine a semantic classification label thereof; and determining a hand movement vector of a corresponding semantic gesture according to the semantic classification label. Through the execution of the steps, the method can determine a hand action highly matched with semantics, so that the hand action and the semantics have higher correlation and fitting degree, and unnatural feeling of a user is avoided, thereby improving naturalness and accuracy of the virtual human action.

Description

Technical Field

[0001] This invention relates to the field of virtual human motion technology, and in particular to a method for determining virtual human hand movements, a device for determining virtual human hand movements, and a corresponding computer-readable storage medium. Background Technology

[0002] With technological innovation, increasingly diverse virtual digital humans are taking center stage. Among them, interactive service-oriented virtual humans, such as virtual tour guides, virtual anchors, virtual narrators, virtual teachers, and virtual employees, are appearing more and more frequently in daily life. The naturalness and intelligence of a virtual human's performance greatly influences the interaction experience. In particular, the virtual human's motion system—its gestures and movements—is a crucial aspect determining its expressiveness. If a virtual human's movements are as natural as a real person's and match their spoken content, it significantly increases the virtual human's perceived naturalness.

[0003] Currently, virtual human motion systems typically employ an animation playback method. This involves first creating a vast library of preset animations using motion capture and manual creation by animators. During compositing, a specific preset animation is retrieved and played back, rendered as a video, and then displayed in real-time or via streaming. While logically simple, this method suffers from several drawbacks in practice. Because it involves playing back animations from a library, the movements are often mechanical and repetitive, and the semantics don't always perfectly match the actions. This results in a stiff and unnatural experience for users, easily discouraging them from interacting with the virtual human. Furthermore, to cover a wider range of movements, a large-scale motion library needs to be built, leading to high labor costs, long production cycles, and difficulties in maintenance and updates.

[0004] In order to overcome the above-mentioned defects of the existing technology, there is an urgent need in the field for a technology to determine the hand movements of virtual humans, so as to determine the hand movements that are highly matched with the semantics, so that the hand movements and semantics have a higher degree of correlation and fit, and avoid making users feel unnatural, thereby improving the naturalness and accuracy of the virtual human's movements. Summary of the Invention

[0005] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed descriptions that follow.

[0006] To overcome the aforementioned deficiencies in the existing technology, the present invention provides a method for determining the hand movements of a virtual human, a system for determining the hand movements of a virtual human, and a corresponding computer-readable storage medium, which can determine hand movements that are highly semantically matched, so that the hand movements and semantics have a higher degree of correlation and fit, and avoid making users feel unnatural, thereby improving the naturalness and accuracy of the virtual human's movements.

[0007] Specifically, the method for determining the hand movements of a virtual human provided by the first aspect of the present invention includes the following steps: acquiring the broadcast text of the virtual human; performing semantic parsing on the broadcast text to determine its semantic classification label; and determining the hand motion vector of the corresponding semantic gesture based on the semantic classification label.

[0008] Furthermore, in some embodiments of the present invention, the step of performing semantic parsing on the broadcast text to determine its semantic classification label includes: performing phrase segmentation on the broadcast text based on a single semantic to obtain at least one phrase containing a single semantic; and processing each phrase respectively via a pre-trained text classification model to obtain at least one corresponding semantic classification label.

[0009] Furthermore, in some embodiments of the present invention, before processing each phrase separately via the text classification model, the method further includes the following steps: performing keyword matching on each phrase based on pre-set keywords; and in response to any phrase successfully matching any keyword, determining the corresponding symbol gesture's hand motion vector based on the successfully matched keyword.

[0010] Furthermore, in some embodiments of the present invention, each of the semantic classification labels corresponds to multiple semantic gestures involving the left and / or right hand of the virtual human, and the step of determining the hand motion vector of the corresponding semantic gesture according to the semantic classification label includes: randomly selecting one of the multiple corresponding semantic gestures according to the semantic classification label, and determining its hand motion vector.

[0011] Furthermore, in some embodiments of the present invention, the step of randomly selecting one of the corresponding multiple semantic gestures according to the semantic classification label includes: obtaining the historical record of the selection of multiple semantic gestures corresponding to the semantic classification label; randomly selecting a non-repeating semantic gesture from the multiple semantic gestures corresponding to the semantic classification label according to the historical record; and recording the selected semantic gesture.

[0012] Furthermore, in some embodiments of the present invention, each semantic gesture corresponds to a reference motion vector, and the step of determining the hand motion vector of the corresponding semantic gesture according to the semantic classification label includes: determining the reference motion vector of the corresponding semantic gesture according to the semantic classification label; and adding a random value to the starting point coordinates and / or ending point coordinates of the reference motion vector to determine the hand motion vector of the diversified processing.

[0013] Furthermore, in some embodiments of the present invention, the reference motion vector includes a first reference motion vector with the origin of the coordinate system as its starting point or ending point, and a second reference motion vector not with the origin of the coordinate system as its starting point or ending point. The step of adding a random value to the starting point coordinates and / or ending point coordinates of the reference motion vector to determine the hand motion vector of the diversified processing includes: determining whether the starting point coordinates and / or ending point coordinates of the reference motion vector are the origin of the coordinate system; and adding the random value to the starting point coordinates and / or ending point coordinates that are not the origin of the coordinate system to determine the hand motion vector of the diversified processing.

[0014] Furthermore, in some embodiments of the present invention, after determining the hand motion vector, the method further includes the following steps: determining a reference hand motion trajectory corresponding to the hand motion vector, wherein the reference hand motion trajectory includes at least the features of starting point coordinates, ending point coordinates and trajectory radii; and adding a random value to the trajectory radii to determine a diversified hand motion trajectory.

[0015] Furthermore, in some embodiments of the present invention, after determining the hand motion vector, the method further includes the following steps: obtaining the personality parameters and / or emotion parameters of the virtual human; and determining the movement speed of the virtual human's hand based on the personality parameters and / or the emotion parameters.

[0016] Furthermore, in some embodiments of the present invention, after determining the hand motion vector, the method further includes the following step: synthesizing an animation of the virtual human's hand movements based on the hand motion vector.

[0017] Furthermore, the apparatus for determining virtual human hand movements provided according to a second aspect of the present invention includes: a memory; and a processor, wherein the processor is connected to the memory and is configured to perform the above-described method for determining virtual human hand movements.

[0018] Furthermore, according to a third aspect of the present invention, a computer-readable storage medium is provided thereon storing computer instructions that, when executed by a processor, implement a method for determining virtual human hand movements. Attached Figure Description

[0019] The above-described features and advantages of the present invention will be better understood after reading the following detailed description of embodiments of the present disclosure in conjunction with the accompanying drawings. In the drawings, components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.

[0020] Figure 1 An architectural diagram of an apparatus for determining virtual human hand movements according to some embodiments of the present invention is shown.

[0021] Figure 2 A flowchart of a method for determining virtual human hand movements according to some embodiments of the present invention is shown.

[0022] Figure 3 A flowchart of a method for determining virtual human hand movements according to some embodiments of the present invention is shown.

[0023] Figure 4 A schematic diagram of a coordinate system for establishing virtual human hand movements is shown, according to some embodiments of the present invention. Detailed Implementation

[0024] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Although the description of the present invention is presented in conjunction with preferred embodiments, this does not mean that the features of the invention are limited to these embodiments. On the contrary, the purpose of describing the invention in conjunction with embodiments is to cover other options or modifications that may be derived based on the claims of the present invention. To provide a thorough understanding of the invention, many specific details will be included in the following description. The invention may also be implemented without using these details. Furthermore, to avoid confusion or obscuring the focus of the invention, some specific details will be omitted in the description.

[0025] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0026] Furthermore, the terms "upper," "lower," "left," "right," "top," "bottom," "horizontal," and "vertical" used in the following description should be understood as the orientations shown in the relevant paragraphs and accompanying drawings. These relative terms are for illustrative purposes only and do not imply that the described apparatus must be manufactured or operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0027] It is understood that although terms such as "first," "second," and "third" may be used herein to describe various components, regions, layers, and / or parts, these components, regions, layers, and / or parts should not be limited by these terms, and these terms are only used to distinguish different components, regions, layers, and / or parts. Therefore, the first components, regions, layers, and / or parts discussed below may be referred to as second components, regions, layers, and / or parts without departing from some embodiments of the present invention.

[0028] As mentioned above, current virtual human motion systems typically employ an animation playback method. This involves first creating a motion library containing a vast number of preset animations through motion capture and manual creation by animators. During compositing, a specific preset animation is retrieved and played back, rendered as a video, and then displayed in real-time or via streaming. While logically simple, this method suffers from several drawbacks in practice. Because it involves playing back animations from a motion library, the movements are often mechanical and repetitive, and the semantics are difficult to perfectly match. This results in a stiff and unnatural experience for users, easily discouraging them from interacting with the virtual human. Furthermore, to cover a wider range of movements, a large-scale motion library needs to be built, leading to high labor costs, long production cycles, and difficulties in maintenance and updates.

[0029] To overcome the aforementioned deficiencies in the existing technology, the present invention provides a method for determining the hand movements of a virtual human, a device for determining the hand movements of a virtual human, and a corresponding computer-readable storage medium, which can determine hand movements that are highly semantically matched, so that the hand movements and semantics have a higher degree of correlation and fit, and avoid making users feel unnatural, thereby improving the naturalness and accuracy of the virtual human's movements.

[0030] In some non-limiting embodiments, the method for determining the hand movements of a virtual human provided in the first aspect of the present invention can be implemented via the apparatus for determining the hand movements of a virtual human provided in the second aspect of the present invention. Specifically, the apparatus may be configured with a memory and a processor. The memory includes, but is not limited to, the computer-readable storage medium provided in the third aspect of the present invention, on which computer instructions are stored. The processor is connected to the memory and configured to execute the computer instructions stored in the memory to implement the method for determining the hand movements of a virtual human provided in the first aspect of the present invention.

[0031] Please refer to the following first. Figure 1 , Figure 1 A schematic diagram of an apparatus for determining the hand movements of a virtual human, according to some embodiments of the present invention, is shown.

[0032] like Figure 1 As shown, the apparatus for determining the hand movements of a virtual human provided in the second aspect of the present invention may include a communication module 105, a memory, and a processor 102. The memory may include an internal communication bus 101, a processor 102, a read-only memory (ROM) 103, a random access memory (RAM) 104, a communication port 105, and a hard disk 107. The internal communication bus 101 enables data communication between the components of the apparatus for determining the hand movements of the virtual human. The processor 102 can perform judgments and issue prompts. In some embodiments, the processor 102 may consist of one or more processors. The communication port 105 enables data transmission and communication between the apparatus for determining the hand movements of the virtual human and external input / output devices. In some embodiments, the apparatus for determining the hand movements of the virtual human can send and receive information and data from a network through the communication port 105. In some embodiments, the apparatus for determining the hand movements of the virtual human can transmit and communicate with external input / output devices in a wired manner through an input / output terminal 106. The device for determining the hand movements of a virtual human may also include different forms of program storage units and data storage units, such as hard disk 107, read-only memory (ROM) 103, and random access memory (RAM) 104, capable of storing various data files used for computer processing and / or communication, as well as possible program instructions executed by processor 102. Processor 102 executes these instructions to implement the main part of the method. The results processed by processor 102 are transmitted to an external output device via communication port 105 and displayed on the user interface of the output device.

[0033] The working principle of the device for determining virtual human hand movements will be described below with reference to some embodiments of methods for determining virtual human hand movements. Those skilled in the art will understand that these embodiments are merely non-limiting implementations provided by the present invention, intended to clearly demonstrate the main concept of the invention and provide specific solutions convenient for public implementation, rather than limiting all functions or operating methods of the device for determining virtual human hand movements. Similarly, the device for determining virtual human hand movements is also only a non-limiting implementation provided by the present invention and does not limit the entities performing the steps in these methods for determining virtual human hand movements.

[0034] Please refer to the reference. Figure 2 as well as Figure 3 . Figure 2A flowchart of a method for determining virtual human hand movements according to some embodiments of the present invention is shown. Figure 3 A flowchart of a method for determining virtual human hand movements according to some embodiments of the present invention is shown.

[0035] like Figure 2 As shown, in determining the virtual human's hand movements, this invention first acquires the virtual human's spoken text, then performs semantic analysis on the spoken text to determine its semantic classification label. Subsequently, this invention can determine the corresponding semantic gesture's hand motion vector based on the semantic classification label determined by the semantic analysis.

[0036] Furthermore, such as Figure 3 As shown, in the process of semantic parsing the broadcast text, the present invention can first perform phrase segmentation based on a single semantic to obtain at least one phrase containing a single semantic, and then perform semantic parsing based on each phrase containing a single semantic.

[0037] Specifically, in some embodiments of the present invention, the present invention can first collect the speech data of the virtual human and store it in a database as a virtual human speech corpus. Then, the present invention can use the virtual human speech data as corresponding annotations, mapping the virtual human speech data to semantic classification labels and storing this mapping in the database. The database can be stored with the speech data as the primary key and the corresponding semantic labels as the corresponding keys, storing these two as a single database record. Next, the present invention can model the determination of semantic labels from the speech data as a multi-classification problem, integrating various multi-classification models as sub-decision trees into the ensemble learning model to train the multi-classification problem. The trained ensemble learning model is then saved as a pre-trained text classification model. Thus, after pre-training the text classification model, the present invention can obtain the virtual human's broadcast text through a speech conversion module, and then use a BERT model, a SegNet model, and / or a U-Net model to perform semantic segmentation on the broadcast text to obtain at least one phrase containing a single semantic meaning.

[0038] After obtaining at least one phrase containing a single semantic meaning through a semantic segmentation model, the present invention can preferably perform keyword matching on the aforementioned phrases based on pre-set keywords. In response to any phrase successfully matching any pre-set keyword, the present invention can determine the corresponding hand motion vector of the symbol gesture based on the successfully matched keyword.

[0039] Specifically, the aforementioned symbolic gestures are gestures endowed with specific meanings, usually possessing clear semantic connotations and strongly correlated with specific keywords in the text. For example, a greeting gesture typically corresponds to the keywords "Hello everyone" or "Nice to meet you all." Similarly, a heart-shaped gesture usually corresponds to the keywords "heart" or "heart gesture." A cheering gesture typically corresponds to the keywords "Keep going" or "Let's cheer together." Therefore, this invention can pre-establish a database table of symbolic gestures and their corresponding semantics based on pre-set keywords such as "Hello everyone," "Nice to meet you all," "heart," "heart," and "cheer up." By matching keywords in this database table, the hand motion vectors of the symbolic gestures corresponding to the virtual human can be directly determined, and the animation of the symbolic gestures can be synthesized. Thus, by prioritizing the use of symbolic gestures for animation synthesis, this invention facilitates user understanding of gestures, has higher expressive efficiency, and avoids the prediction process of semantic parsing, thereby significantly improving the accuracy and efficiency of the method.

[0040] Furthermore, compared to the aforementioned symbolic gestures, semantic gestures are gestures made with natural hand movements in accordance with the semantic content the speaker intends to express. For example, when expressing the semantics of "starting a journey" or "opening a new chapter," a natural person usually accompanies the gesture with both hands opening to express the meaning of "start." Similarly, when expressing the semantics of "significantly reduced output" or "greatly reduced error rate," a natural person usually accompanies the gesture with a downward movement of the hands to express the meaning of "reduction." Therefore, in some embodiments, for single semantic phrases that do not match any keyword, this invention can input them into a pre-trained text classification model. Based on this text classification model, each single semantic phrase is processed separately to obtain its corresponding semantic classification label. Here, the text classification model can be implemented based on traditional statistical models such as Bayesian classification, linear discriminant analysis based on the traditional Fisher method, and / or traditional multivariate analysis models.

[0041] Furthermore, in some embodiments, to overcome the drawback of repetitive actions, each semantic classification label of the present invention can preferably correspond to multiple semantic gestures of the virtual human's left and / or right hands. The present invention can obtain semantic classification labels based on semantic parsing, randomly select one from its corresponding multiple semantic gestures, and use this to determine the virtual human's hand motion vector.

[0042] Please refer to the details. Figure 4 And Table 1. Figure 4 A schematic diagram of a coordinate system for establishing virtual human hand movements according to some embodiments of the present invention is shown. Table 1 shows the correspondence between each semantic tag and multiple hand movement vectors.

[0043] like Figure 4As shown, when establishing the coordinate system of the virtual human's hand movements, this invention can establish a three-dimensional coordinate system based on the service etiquette posture of a natural person, with the virtual human's hand naturally placed in front of the abdomen as the origin (0,0,0), and record each hand movement in the coordinate system in the form of three-dimensional coordinates to obtain the correspondence table between semantic tags and hand movement vectors shown in Table 1.

[0044] Table 1. Correspondence between gestures and semantic tags

[0045]

[0046]

[0047] In the embodiments shown in Table 1, one semantic label can correspond to the average vector of multiple hand movements. In the process of determining the virtual human's hand movement vector based on semantic classification labels obtained through semantic parsing, the present invention preferably acquires the historical record of multiple semantic gestures selected corresponding to the aforementioned semantic classification labels, and randomly selects a non-repeating semantic gesture from the multiple semantic gestures corresponding to the semantic classification labels based on this historical record, as the hand movement vector selected this time. Furthermore, the present invention can also record the hand movement vector of the selected semantic gesture this time as a historical record for subsequent selections of hand movement vectors.

[0048] Specifically, when determining the hand movement vector corresponding to the semantic tag "leave" in the virtual human's first sentence, this invention can randomly select any hand movement vector corresponding to that semantic tag based on a blank historical record, for example, (0,0,0)→(1,0,0), and record it as a record in the action record table of the database. Subsequently, when determining the hand movement vector corresponding to the semantic tag for the second time, this invention can, based on the current action record table, avoid using the already reused hand movement vector (0,0,0)→(1,0,0), and instead prioritize using other unused hand movement vectors under the same semantic tag, for example, (-1,0,0)→(1,0,0). Furthermore, the clearing frequency of this action record table can be set according to the virtual human's dynamic needs, thereby providing various dynamic change effects such as not repeating any historical records, not repeating the previous N times, and not repeating the last time.

[0049] Thus, by configuring multiple semantic gestures for each semantic category label and performing deduplication filtering, this invention can provide more dynamic and varied action effects while ensuring semantic matching, thereby avoiding mechanical and repetitive actions in virtual humans and improving the naturalness and aesthetics of virtual humans.

[0050] Furthermore, in some embodiments, in order to provide more dynamic and varied effects while ensuring semantic matching, the present invention may add a random value to the starting coordinates and / or ending coordinates of each semantic gesture based on the baseline motion vectors shown in Table 1, so as to further determine the hand motion vectors for diversified processing.

[0051] Specifically, the aforementioned reference motion vectors may include first reference motion vectors such as (0,0,0)→(1,0,0) and (0,1,0)→(0,0,0) with the origin of the coordinate system as the starting point or ending point, and second reference motion vectors such as (0,1,-1)→(0,1,1) and (0,1,1)→(0,1,-1) without the origin of the coordinate system as the starting point or ending point. In determining the hand motion vector corresponding to the semantic classification label, this invention can first determine whether the starting point coordinates and / or ending point coordinates of the selected reference motion vector are the origin of the coordinate system. If the starting point coordinates or ending point coordinates of the selected reference motion vector are the origin of the coordinate system, i.e., whether the semantic gesture starts from the origin position in front of the abdomen or ends at the origin position in front of the abdomen, this invention may not offset the starting and ending points involving that origin position, in order to prioritize making the virtual human's performance more natural.

[0052] Conversely, if the starting or ending coordinates of the selected reference motion vector are not the origin of the coordinate system, this invention can add a random value to the non-originating starting and ending coordinates of the reference motion vector, so that the starting and ending positions of each motion are slightly different. Here, the random value can be a random value that follows a Gaussian distribution ~N(0,0.15). Specifically, when adding random values, this invention can change the starting and ending positions of the reference motion vector respectively, that is, add a change amount (△x, △y, △z) to the starting and ending coordinates of the reference motion vector respectively, where △x, △y, and △z are all random values ​​that follow a Gaussian distribution ~N(0,0.15). In addition, to avoid a large shift in the overall direction of the motion vector, the value range of each change amount △x, △y, and △z can preferably be [-0.2, 0.2].

[0053] Furthermore, in some embodiments of the present invention, after determining the aforementioned hand motion vector, the present invention can also determine its corresponding reference hand motion trajectory based on the hand motion vector. Here, the reference hand motion trajectory may at least include the features of starting point coordinates, ending point coordinates, and trajectory radii. Here, when the radii are greater than 0 or less than 0, the motion trajectory is an arc pointing in the corresponding direction, and the curvature of the arc increases with the increase of the absolute value of the radii. Conversely, when the radii are 0, the motion trajectory is a straight line. When determining the motion trajectory of the semantic gesture corresponding to the semantic tag, the present invention may also preferably add a random value to the trajectory radii parameter to determine the hand motion trajectory for diversified processing.

[0054] Furthermore, in some embodiments of the present invention, after determining the aforementioned hand movement vector, the present invention may also preferably acquire the personality parameters and / or emotion parameters of the virtual human. Here, the personality parameters can be pre-set by the creator when constructing the virtual human, while the emotion parameters can be generated in real time based on the virtual human's current or preset time threshold voice interaction content. After acquiring the virtual human's personality parameters and / or emotion parameters, the present invention can also determine the movement speed of the virtual human's hands based on the aforementioned personality parameters and / or emotion parameters. For example, for a virtual human with a cheerful and lively personality, or an excited and happy emotion, the present invention can appropriately increase the movement speed of the virtual human's hands. As another example, for a virtual human with a gentle and dignified personality, or a calm and sad emotion, the present invention can appropriately slow down the movement speed of the virtual human's hands. Thus, the present invention can further provide diversified semantic actions to enhance the dynamic and natural feel of the virtual human's movements.

[0055] Furthermore, in some embodiments of the present invention, the virtual human's hand posture can always maintain a posture with the five fingers together and the palm straight, in order to conform to the service etiquette of service-oriented virtual human gesture guidance, and also to allow users to clearly see the gesture movements, thereby improving the user experience.

[0056] Furthermore, such as Figure 3 As shown, in some embodiments of the present invention, after determining the hand motion vectors of the virtual human's symbolic gestures and / or semantic gestures, the present invention can also use modules such as IK (Inverse Kinematics) to calculate the motion trajectories of the virtual human's forearm, elbow, and upper arm based on the aforementioned hand motion vectors, and fuse the animations of each symbolic gesture and each semantic gesture in text order to synthesize a complete hand motion animation of the virtual human. Specifically, the IK module can use a forward and backward iterative motion method to find a new position of a joint along a line to the next joint, thereby solving the problem of reverse motion in position space rather than direction space.

[0057] In summary, the present invention provides a method for determining the hand movements of a virtual human, a device for determining the hand movements of a virtual human, and a corresponding computer-readable storage medium, which can determine hand movements that are highly matched with semantics, so that the hand movements and semantics have a higher degree of correlation and fit, and avoid making users feel unnatural, thereby improving the naturalness and accuracy of the virtual human's movements.

[0058] Although the methods described above are illustrated and depicted as a series of actions for the sake of simplicity, it should be understood and appreciated that these methods are not limited by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but which may be understood by those skilled in the art, according to one or more embodiments.

[0059] Those skilled in the art will understand that information, signals, and data can be represented using any of a variety of different techniques and arts. For example, the data, instructions, commands, information, signals, bits, symbols, and chips described throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.

[0060] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0061] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read and write information to / from the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0062] In one or more exemplary embodiments, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Any connection is also legitimately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. As used in this article, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0063] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining virtual human hand movements, characterized in that, Includes the following steps: Obtain the broadcast text of the virtual human; The broadcast text is semantically parsed to determine its semantic classification label; as well as Based on the semantic classification label, the corresponding semantic gesture hand motion vector is determined, wherein each semantic gesture corresponds to a reference motion vector. The step of determining the corresponding semantic gesture hand motion vector based on the semantic classification label includes: determining the reference motion vector of the corresponding semantic gesture based on the semantic classification label; determining whether the starting point coordinates and / or ending point coordinates of the reference motion vector are the origin of the coordinate system; adding random values ​​to the starting point coordinates and / or ending point coordinates that are not the origin of the coordinate system to determine the diversified hand motion vector, wherein the reference motion vector includes a first reference motion vector with the origin of the coordinate system as the starting point or ending point, and a second reference motion vector that does not have the origin of the coordinate system as the starting point or ending point.

2. The method as described in claim 1, characterized in that, The step of performing semantic parsing on the broadcast text to determine its semantic classification label includes: The broadcast text is segmented based on a single semantic meaning to obtain at least one phrase containing a single semantic meaning; and Each phrase is processed by a pre-trained text classification model to obtain at least one corresponding semantic classification label.

3. The method as described in claim 2, characterized in that, Before processing each phrase individually via the text classification model, the method further includes the following steps: Based on pre-set keywords, keyword matching is performed on each of the phrases; and In response to a successful match between any of the phrases and any of the keywords, the hand motion vector of the corresponding symbol gesture is determined based on the successfully matched keyword.

4. The method as described in claim 1, characterized in that, Each semantic classification label corresponds to multiple semantic gestures involving the left and / or right hands of the virtual human. The step of determining the hand motion vector of the corresponding semantic gesture based on the semantic classification label includes: Based on the semantic classification label, one of the corresponding semantic gestures is randomly selected and its hand motion vector is determined.

5. The method as described in claim 4, characterized in that, The step of randomly selecting one of the corresponding semantic gestures based on the semantic classification label includes: Obtain the historical records of multiple semantic gestures selected corresponding to the semantic classification label; Based on the historical records, a non-repeating semantic gesture is randomly selected from multiple semantic gestures corresponding to the semantic classification label; and Record the semantic gestures selected this time.

6. The method as described in claim 1, characterized in that, After determining the hand motion vector, the method further includes the following steps: Based on the hand motion vector, a corresponding reference hand motion trajectory is determined, wherein the reference hand motion trajectory includes at least the features of a starting point coordinate, an ending point coordinate, and a trajectory radian; and A random value is added to the trajectory radian to determine the hand movement trajectory for diverse processing.

7. The method as described in claim 1, characterized in that, After determining the hand motion vector, the method further includes the following steps: Obtain the personality parameters and / or emotional parameters of the virtual human; and The movement speed of the virtual human's hand is determined based on the personality parameters and / or the emotion parameters.

8. The method according to any one of claims 1 to 7, characterized in that, After determining the hand motion vector, the method further includes the following steps: Animation of the virtual human's hand movements is synthesized based on the hand motion vectors.

9. A method for determining the hand movements of a virtual human, characterized in that, include: Memory; as well as A processor, wherein the processor is connected to the memory, and is configured to implement the method for determining virtual human hand movements as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, the method for determining the virtual human hand movements as described in any one of claims 1 to 8 is implemented.

Images