Apparatus and method for automatically generating character rigging
The automatic character rigging generation device and method automate the process of creating 3D characters by processing 3D data to generate new joints and apply skinning, allowing users to create their own characters without professional expertise.
Patent Information
- Application Number
- KR1020230183705
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-12-15
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2043-12-15
AI Technical Summary
General users lack the ability to create their own 3D characters through rigging, which is currently a manual process requiring professional expertise.
An automatic character rigging generation device and method that utilizes an algorithm to process 3D character data, generate new joints, and apply skinning using essential joint position data, including preprocessing, data generation, and application units to automate the rigging process.
Enables anyone to easily perform character rigging, automatically generating new joints and skinning, thereby democratizing the creation of 3D characters.
Smart Images

Figure R1020230183705_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to an automatic character rigging generation device and method, and more specifically, to an automatic character rigging generation device and method capable of automatically performing rigging, which was previously done manually, through an algorithm. Background Technology
[0002] Recently, with the growing interest in Augmented Reality (AR) and Virtual Reality (VR), research and development regarding 3D video technology are actively underway. In addition, a virtual world known as the Metaverse, where social, economic, and cultural activities can take place just like in the real world, has emerged, and with the appearance of Metaverse platforms, a wide variety of Metaverse content is being generated explosively.
[0003] In 3D virtual spaces, a character representing the user—namely, an avatar—acts on behalf of the user. Rigging is essential for creating such 3D characters. Rigging refers to a series of processes involving creating, implanting, or assigning a character's skeleton so that the character can move as the user desires. Currently, rigging is still performed directly by professionals, and there is a problem in that general users have no way to create their own 3D characters through rigging. The problem to be solved
[0004] The technical problem that the present invention aims to solve is to provide a character automatic rigging generation device and method capable of automatically generating new joints and skinning using basic 3D character data and essential joint position data. means of solving the problem
[0005] A character automatic rigging generation device according to one embodiment of the present invention includes: a preprocessing unit that receives character data and essential joints, digitizes character mesh data, and performs a preprocessing process to correct the essential joints; a data generation unit that uses the data generated through the preprocessing process to generate joints of each segment according to the joint ratio of the human body, matches the relationships and structures of the generated joints, and generates skinning data; and a data application unit that performs a data application process to apply the data generated through the data generation process to the character data.
[0006] The above preprocessing unit may include a character data input unit that renders in 2D based on the character mesh data included in the character data, which is a 3D model file.
[0007] The above preprocessing unit may further include an essential joint input unit that adds the position of the essential joint to the data rendered in 2D based on the character mesh data.
[0008] The above preprocessing unit may further include a character data preprocessing unit that quantifies the character mesh data of the mesh node in the character data, and an essential joint preprocessing unit that performs an essential joint preprocessing process that converts the 2D position of the essential joint into a position on 3D coordinates.
[0009] The above preprocessing unit may further include an essential joint validation unit that validates the position of the preprocessed essential joint to prevent cases where the position of the essential joint deviates from the character's vertex position or is entered at a different location after the essential joint preprocessing process.
[0010] The data generation unit may include a joint generation unit that receives information on 8 essential joints preprocessed through the preprocessing process and generates a total of 65 joints to match the character by adding 57 joints to the 8 essential joints.
[0011] The data generation unit may further include a joint data generation unit that generates the 3D coordinate positions of the 57 joints based on the 3D coordinate positions of the 8 preprocessed essential joints.
[0012] The above data generation unit may further include a joint data validation unit that performs a validation of the joint generated by the joint data generation unit.
[0013] The above data generation unit may further include a joint hierarchy data generation unit that matches the relationships between joints into a joint hierarchy structure according to the characteristics of the human body, using the joints generated by the joint data generation unit.
[0014] The above data generation unit may further include a skinning data generation unit that performs a weight painting process for the data matched to the joint hierarchy, which sets the sensitivity of mesh deformation and movement according to bone movement.
[0015] A method for generating automatic character rigging according to another embodiment of the present invention includes: a step of receiving character data and essential joints as input, performing a preprocessing step of quantifying character mesh data and correcting the essential joints; a step of performing a data generation step of generating joints of each segment according to the joint ratio of the human body using the data generated through the preprocessing step, matching the relationships and structures of the generated joints, and generating skinning data; and a step of performing a data application step of applying the data generated through the data generation step to the character data.
[0016] The step of performing the above preprocessing process includes the step of rendering in 2D based on the character mesh data included in the character data, which is a 3D model file.
[0017] The step of performing the above preprocessing process may further include the step of adding the positions of the essential joints to the data rendered in 2D based on the character mesh data.
[0018] The step of performing the above preprocessing process may further include the step of quantifying the character mesh data of the mesh node in the character data, and the step of performing an essential joint preprocessing process that converts the 2D position of the essential joint into a position on 3D coordinates.
[0019] The step of performing the above preprocessing process may further include a step of validating the position of the preprocessed essential joint to prevent the position of the essential joint from being outside the character's vertex position or being entered at a different location after the above essential joint preprocessing process.
[0020] The step of performing the above data generation process may include receiving information on 8 essential joints preprocessed through the above preprocessing process, and adding 57 joints to the 8 essential joints to generate a total of 65 joints to fit the character.
[0021] The step of performing the above data generation process may further include the step of performing a joint data generation process that generates the 3D coordinate positions of the 57 joints based on the 3D coordinate positions of the 8 preprocessed essential joints.
[0022] The step of performing the above data generation process may further include the step of performing a validation of the joint generated by the above joint data generation process.
[0023] The step of performing the above data generation process may further include the step of matching the relationships between joints into a joint hierarchy structure according to the characteristics of the human body, using the joints generated by the above joint data generation process.
[0024] The step of performing the above data generation process may further include the step of performing a weight painting process for the data matched to the joint hierarchy, which sets the sensitivity of mesh deformation and movement according to bone movement. Effects of the invention
[0025] The character automatic rigging generation device and method according to an embodiment of the present invention can automatically generate new joints and skinning using basic 3D character data and essential joint position data, and accordingly, anyone can easily process the rigging work. Brief explanation of the drawing
[0026] FIG. 1 is a block diagram showing a character automatic rigging generation device according to one embodiment of the present invention. Figure 2 shows an example of rendering 3D character data into 2D. Figure 3 shows an example of adding the positions of 8 essential joints to 2D rendered data. Figure 4 shows an example of character mesh data being converted into 3D vectors. Figure 5 shows an example of an Fbx SDK file structure. Figure 6 shows the extraction range of the vertices of the Lower Arm. Figure 7 shows the range of vertices to be extracted for extracting the Z value of Hand. Figure 8 shows the range of vertices to be extracted for extracting the Z value of the Foot. Figure 9 shows a set of vertices of the hand for estimating the end points of each finger. Fig. 10 shows the hand joint (J 10, J 34 From ) to the end point of the middle finger, the 3rd joint of the middle finger (J 22 , J 46 It indicates the distance to ). Fig. 11 shows the thumb joint (J 14 or J 38 It represents a temporary thumb joint for generating joint data of ). Figure 12 shows the positions of the index, ring, and little fingers centered on the middle finger. Figure 13 shows the joint (left) generated by the joint data generation unit and the original joint (right). Figure 14 shows an example of classifying vertices into 5 clusters by applying the K-Means algorithm as a classification method for 5 fingers. Figure 15 shows the character generated by the character creation process (left) and the original character data. Specific details for implementing the invention
[0027] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present invention. The present invention may be embodied in various different forms and is not limited to the embodiments described herein.
[0028] To clearly explain the present invention, parts unrelated to the explanation have been omitted, and the same reference numerals are used for identical or similar components throughout the specification.
[0029] Furthermore, throughout the specification, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.
[0030] Hereinafter, an apparatus and method for generating automatic character rigging according to an embodiment of the present invention will be described with reference to FIGS. 1 to 15.
[0031] FIG. 1 is a block diagram showing a character automatic rigging generation device according to one embodiment of the present invention.
[0032] Referring to FIG. 1, a character automatic rigging generation device according to one embodiment of the present invention may include a preprocessing unit (100), a data generation unit (200), and a data application unit (300). The preprocessing unit (100) may receive character data and essential joints, digitize character mesh data, and perform a preprocessing process to correct the essential joints. The data generation unit (200) may perform a data generation process using the data generated through the preprocessing process to generate joints of each segment according to the joint ratio of the human body, match the relationships and structures of the generated joints, and generate skinning data. The data application unit (300) may perform a data application process to apply the data generated through the data generation process to the character data initially input into the character automatic rigging generation device.
[0033] A character automatic rigging generation method according to one embodiment of the present invention may include a preprocessing process, a data generation process, and a data application process, and may include an algorithm operation method that automatically performs the preprocessing process, the data generation process, and the data application process.
[0034] For example, a character automatic rigging generation device and method according to one embodiment of the present invention may be configured (developed, written) in Python, a type of computer language, and may perform character automatic rigging using Autodesk's Fbx SDK.
[0035] The preprocessing unit (100) may include a character data input unit (110), an essential joint input unit (120), a character data preprocessing unit (130), an essential joint preprocessing unit (140), and an essential joint validation unit (150) to perform a preprocessing process.
[0036] The character data input unit (110) receives character data, which is a 3D model file (e.g., fbx, obj, glb, etc.). The character data may include character mesh data that defines the skin of a 3D character. The character mesh data includes vertices representing coordinate points (X, Y, Z) in 3D space, edges connecting each vertex, and faces formed by connecting three or more lines. The character data input unit (110) can perform a process of rendering 3D (three-dimensional) character data into 2D (two-dimensional) using only 2D data (X, Y) based on the character mesh data for essential joint input. For example, the character data input unit (110) can perform rendering using Three.js, a library that helps render 3D objects, and can perform rendering by determining the rendering ratio based on the scale of the input 3D model file. Figure 2 shows an example of rendering 3D character data into 2D.
[0037] The required joint input unit (120) can perform the process of adding the position (x, y) of the required joint to the 2D rendered data based on the character mesh data. The required joint may be pre-specified by the user. The required joint may include 8 joints as shown in Table 1.
[0038]
[0039] That is, the essential joint input section (120) can add the positions of eight essential joints, including the hip, neck, right hand, left hand, right elbow, left elbow, right ankle, and left foot, to the 2D rendered data. Since humanoid characters generally have a left-right symmetrical shape, the Hand, Lower Arm, and Foot values are left-right symmetrical values. FIG. 3 shows an example of adding the positions of eight essential joints to the 2D rendered data.
[0040] The character data preprocessing unit (130) can perform the process of converting character mesh data of mesh nodes in character data, which is a 3D model file, into numerical values. When loading character data, which is a 3D model file (e.g., character data in Fbx Sdx), the character data preprocessing unit (130) can read it in a tree structure format as exemplified in FIG. 5. FIG. 5 shows an example of the structure of an Fbx Sdk file. In the tree structure, a Scene node is located at the top, and a Root node and each data node are located below it. Each node contains a Mesh, Skeleton, Animation, etc. The character data preprocessing unit (130) can convert character mesh data of mesh nodes among the multiple nodes included in the character data. Character mesh data can be composed of vertices and polygons, which are sets of connections of vertices. Character mesh data includes the Global position 3D location values (X, Y, Z) of the vertices, and each vertex can be represented as a 3D vector. FIG. 4 illustrates an example of character mesh data being converted into 3D vectors. Here, it is based on the form of a character standing in a T pose. The character data preprocessing unit (130) can calculate the maximum, minimum, and average values of all vertices. Equation 1 represents all vertices of the character. In the following equations, 'V' represents a vortex and J represents a joint.
[0041]
[0042] The essential joint preprocessing unit (140) can perform an essential joint preprocessing process that converts the 2D position (x, y) of the essential joint into a position in 3D coordinates. The position of each joint in 3D coordinates is a 3D coordinate value J n It can be expressed as.
[0043] Among the essential joints, the Hip (J0) is located in the center, and the essential joint preprocessing unit (140) uses only the Y value among the input values as in Equation 2, and the central X value ( ) and median Z-value( You can calculate the 3D coordinate values of Hip(J0) using ).
[0044]
[0045]
[0046] The essential joint preprocessing unit (140) is the Lower Arm (J9, J) based on the X-axis. 33 The vertices of ) can be extracted. The essential joint preprocessing unit (140) can extract vertices between the X value of the Lower Arm and the X value of the Hand, as exemplified in FIG. 6. FIG. 6 shows the extraction range of the vertices of the Lower Arm. The essential joint preprocessing unit (140) extracts the central Z value of the extracted vertex (as per Equation 3) Using ), Lower Arm(J9, J 33 The 3D coordinate values of ) can be calculated.
[0047]
[0048]
[0049]
[0050]
[0051]
[0052] Among the essential joints, Neck (J4) is located in the center when viewed along the X-axis, just like Hip (J0). Therefore, the essential joint preprocessing unit (140) uses only the Y value among the input values and the center X value as the X value. Use ) and Lower Arm(J9, J with Z value 33 The median Z-value of ) The 3D coordinate values of Neck (J4) can be calculated as in Equation 4 using ).
[0053]
[0054] Hand(J 10 , J 34 Due to the physical structural characteristics, the Lower Arm (J9, J 33 It becomes thinner than ). The essential joint preprocessing unit (140) is Hand(J 10 , J 34 To extract the Z value of ), Hand(J as exemplified in Fig. 7 10 , J 34 ) and Lower Arm(J9, J 33 The value divided in half from the value between ) Lower Arm(J9, J 33 The value added to the X value of ) ) and Hand(J 10 , J 34 )'s X value( Vertices between ) can be extracted. FIG. 7 shows the range of vertex extraction for extracting the Z value of the Hand. The essential joint preprocessing unit (140) is Hand(J as in Equation 5. 10 , J 34 The center Z value of the extracted vertex of ) Using ) Hand(J 10 , J 34 The 3D coordinate values of ) can be calculated. Since the left hand and the right hand are symmetrical, the Z value is the same and the two hands are indistinguishable.
[0055]
[0056]
[0057]
[0058]
[0059] The essential joint preprocessing unit (140) is based on the Y-axis Foot(J 57 , J 62The vertices of ) can be extracted. The essential joint preprocessing unit (140) is Foot(J 57 , J 62 To extract the Z value of ), Hip(J0) and Foot(J 57 , J 62 Vertex extraction range between ) inside ) can be specified. FIG. 8 shows the range of vertex extraction for extracting the Z value of the Foot. The essential joint preprocessing unit (140) is Foot(J as in Equation 6. 57 , J 62 The center Z value of the extracted vertex of ) Using ) Foot(J 57 , J 62 The 3D coordinate values of ) can be calculated. Since the left and right ankles are symmetrical, the Z values are the same and the two feet are not distinguished.
[0060]
[0061]
[0062]
[0063]
[0064] The essential joint validation unit (150) can perform a validation process to validate the position of the preprocessed essential joint in order to prevent the position of the essential joint from being outside the vertex position of the character or being entered at a different position after the essential joint preprocessing process.
[0065] The essential joint validation unit (150) can perform validation to check whether the hip joint is located below the neck joint as in mathematical formula 7.
[0066]
[0067] The essential joint validation unit (150) can perform validation to check whether both ankle joints are located below the hip joints as in mathematical formula 8.
[0068]
[0069] The essential joint validation unit (150) can perform validation to check whether the neck joint is positioned above both elbow joints as in mathematical formula 9.
[0070]
[0071] The essential joint validation unit (150) can perform validation to determine whether the left elbow joint is located to the right of the neck joint and to the left of the left hand joint, as shown in mathematical formula 10.
[0072]
[0073] The essential joint validation unit (150) can perform validation to determine whether the right elbow joint is located to the left of the neck joint and to the right of the right hand joint, as shown in mathematical formula 11.
[0074]
[0075] The essential joint validation unit (150) can perform validation to check whether the position of the essential joint is included between the minimum and maximum values of the vertex position as in Equation 12.
[0076]
[0077] The data generation unit (200) may include a joint generation unit (210), a joint data generation unit (220), a joint data validation unit (230), a joint hierarchy data generation unit (240), and a skinning data generation unit (250) to perform a data generation process.
[0078] The joint generation unit (210) receives information on 8 essential joints preprocessed from the preprocessing unit (100) and can perform a joint generation process to generate a total of 65 joints to match the character by adding 57 joints to the 8 essential joints as shown in Table 2. The 65 joints can be classified into 5 categories, such as Body, Arms, Leg, Head, and Finger.
[0079]
[0080]
[0081]
[0082]
[0083]
[0084] The joint data generation unit (220) can perform a joint data generation process to generate the positions of the remaining 57 joints in 3D coordinates based on the positions of the essential joints (input joints in Table 2) in 3D coordinates.
[0085] 1) Generation of joint data for Body classification
[0086] Although the joint ratios differ from person to person, the joint data generation unit (220) can calculate the body ratio from the hip joint (J0) to the neck joint (J4) on average as J0: J1: J2: J3: J4 = 0 : 0.6 : 0.7 : 0.8 : 0.9. The joint data generation unit (220) can calculate the intermediate value (from the hip joint (J0) to the neck joint (J4) After calculating the difference in Y values, the position values of the remaining joints can be calculated using the intermediate value. Since the X values for joints J0 to J4 are all the same, the X values are not modified and all follow the X value of the hip joint (J0).
[0087] The joint data generation unit (220) can calculate joint data for the hip joint (J0), spine joint (J1), lower chest joint (J2), and upper chest joint (J3) as shown in Equation 13.
[0088]
[0089]
[0090]
[0091]
[0092] 2) Generation of joint data for arm classification
[0093] Although the ratio varies from person to person, the joint data generation unit (220) on average from the lower chest joint (J2) to the hand (J 10 , J 34 The ratio from ) to J2 : J7 : J8 : J9 : J10 can be calculated as J2 : J7 : J8 : J9 : J10 = 0 : 1 : 2 : 5 : 6. The joint data generation unit (220) generates the joint data from the lower chest joint (J2) to the hand (J 10 , J 34 The value between ) and up ( After calculating the difference in X values, the position values of the remaining joints can be calculated according to the ratio. Since both arms are symmetrical and have the same length ratio, the joint data generation unit (220) can calculate the joints of the left arm in the same way.
[0094] The joint data generation unit (220) includes the right shoulder joint (J7) and the left shoulder joint (J 31 ), right upper arm joint (J8), left upper arm joint (J 32 ), right forearm joint (J9), left forearm joint (J 33 ), right hand joint (J 10 ), left hand joint (J 34 The joint data of ) can be calculated as in mathematical formula 14.
[0095]
[0096]
[0097]
[0098]
[0099]
[0100]
[0101]
[0102]
[0103] 3) Generation of joint data for Leg classification
[0104] Since both legs are symmetrical, their length ratios are equal, and the joint data generation unit (220) can estimate the positions of the leg joints by extracting the surrounding vertex positions and calculating the median values of the vertices. However, the joint data generation unit (220) can estimate the hip joint (J0) and the ankle joint (J 57 , J 62 Estimating the Z-values of the thigh joint (J 55 , J 60 The joint data of ) can be calculated as in mathematical formula 15.
[0105]
[0106]
[0107]
[0108] The joint data generation unit (220) calculates the value J of the distance between the thigh and the ankle. Y After producing J Y By extracting vertices based on the Y-axis between and the ankle to generate intermediate Z values, the knee joint (J 56 , J 61 The joint data of ) can be calculated as in mathematical formula 16.
[0109]
[0110]
[0111]
[0112]
[0113]
[0114] The joint data generation unit (220) is the ankle joint (J 57 , J 62 Use the joint data entered as ) as is ( , ) can, and ankle joint (J 57 , J 62 By extracting lower vertices along the Y-axis based on ), the maximum and minimum virtual Z values can be obtained. Additionally, the joint data generation unit (220) can calculate an intermediate value by extracting new vertices based on the X value of the hip joint, the Y value of the ankle joint, and the virtual Z value, and by applying the calculated intermediate value, the toe joint (J 58 , J 63 ) can calculate joint data. That is, the joint data generation unit (220) can calculate the toe joint (J) as in mathematical formula 17. 58 , J 63 Can calculate joint data of ).
[0115]
[0116]
[0117]
[0118]
[0119]
[0120]
[0121]
[0122]
[0123]
[0124]
[0125]
[0126] The joint data generation unit (220) measures the right toe tip joint (J 59 ) and the tip joint of the left toes (J 64 To generate joint data of ), vertices can be extracted using the Y-axis of the ankle and the Z-value of the toes to generate Y-values, and vertices can be extracted once again using the Y-value of the toes to generate Z-values. That is, the joint data generation unit (220) can generate the right toe tip joint (J) as in Equation 18. 59 ) and the tip joint of the left toes (J 64 Joint data of ) can be generated. Since both feet are symmetrical, they are not distinguished by definition.
[0127]
[0128]
[0129]
[0130]
[0131]
[0132]
[0133] 4) Generation of joint data for head classification
[0134] The joint data generation unit (220) divides the entire head into three equal parts starting from the neck joint (J4), and the highest value J along the Y-axis from the entire vertex of the character MaxY Extract, and between the neck and upper chest Joint data for the neck joint (J4), head joint (J5), and head end joint (J6) can be generated by calculating. That is, the joint data generation unit (220) can calculate joint data for the neck joint (J4), head joint (J5), and head end joint (J6) as in Equation 19.
[0135]
[0136]
[0137]
[0138]
[0139]
[0140]
[0141] 5) Generation of joint data for Finger classification
[0142] The human body has a total of 5 fingers (thumb, index finger, middle finger, ring finger, and little finger). The joint data generation unit (220) can generate a total of 20 finger joints by classifying each of the 5 fingers into 4 segments. The joint data generation unit (220) first designates the end point of each finger, and then the end point and both hands (J 10 , J 34 Joints can be generated by calculating the distance to ) as a ratio. The positions of the fingers are located on the right or left side of both hands along the X-axis, and the joint data generation unit (220) can first extract the Hand vertices of the right hand and left hand as in Equation 20 and estimate the end point of each finger based on the extracted vertices. FIG. 9 shows a set of hand vertices for estimating the end point of each finger.
[0143]
[0144]
[0145] The joint data generation unit (220) has a minimum value along the X-axis when the human body is in T pose ( )(right) or maximum value( )(left) can be calculated as the end point of the middle finger. Fig. 10 shows the hand joint (J 10 , J 34 From ) to the end point of the middle finger, the 3rd joint of the middle finger (J 22 , J 46 It indicates the distance to ).
[0146] And the joint data generation unit (220) is from the end point of the middle finger to the hand joint (J 10 , J34 By setting the ratio up to ) as Hand : Middle : Middle 1 : Middle 2 : Middle 3 = 0 : 5 : 2 : 1.5 : 2, the right middle finger joint (J) as in mathematical formula 21 19 ), left middle finger joint (J 43 ), right middle finger joint 1 (J 20 ), left middle finger joint 1 (J 44 ), right middle finger joint 2 (J 21 ), left middle finger joint 2 (J 45 ), right middle finger 3rd joint (J 22 ), left middle finger 3rd joint (J 46 Can calculate joint data of ).
[0147]
[0148]
[0149]
[0150]
[0151]
[0152]
[0153]
[0154]
[0155]
[0156] The joint data generation unit (220) is from the tip of the thumb to the hand joint (J 10 , J 34 The ratio up to ) can be set to Hand : Thumb : Thumb1 : Thumb2 : Thumb3 = 0 : 3 : 2.7 : 2.5 : 1.8. The joint data generation unit (220) extracts the end point of the thumb, V Hand Extract the maximum Z value from the set, and the extracted vertex as a temporary endpoint (J t3 After designating as ), calculate the ratio to the thumb 2 point and the temporary thumb 2 joint (J t2 The position of ) can be found, and again VHand In the set, the minimum value along the Z-axis is the temporary thumb joint (J t2 The set V satisfying ). thumb After finding the set, V thumb After extracting the maximum value along the X-axis of the set, V thumb Maximum value (J t4 ) and the previously created temporary endpoint (J t3 The point between ) is finally the thumb 3rd joint (J 14 or J 38 It can be generated using the joint data of ). Fig. 11 shows the thumb 3 joint (J 14 , J 38 It represents a temporary thumb joint for generating joint data of ). The joint data generation unit (220) represents the thumb 3 joint (J) as in Equation 22. 14 , J 38 Can calculate joint data of ).
[0157]
[0158]
[0159]
[0160]
[0161]
[0162] And the joint data generation unit (220) is the thumb joint (J) which is the end point of the thumb. 14 , J 38 From ) to the hand joint (J 10 , J 34 By applying the ratio up to ), the right thumb joint (J) as in mathematical formula 23 11 ), left thumb joint (J 35 ), right thumb joint 1 (J 12 ), left thumb joint 1 (J 36 ), right thumb joint (J 13 ), left thumb joint (J 37 ), right thumb 3rd joint (J 14 ), left thumb 3rd joint (J 38Can calculate joint data of ).
[0163]
[0164]
[0165]
[0166]
[0167]
[0168]
[0169]
[0170]
[0171]
[0172] The joint data generation unit (220) divides the middle finger into three equal parts as shown in Equation 24 to generate joint data for the remaining fingers, namely the index finger, ring finger, and pinky finger, and sets the reference position ( Create ) and, and middle finger joint 2 (J 21 , J 45 A temporary midpoint of the index, ring, and little fingers can be created using ). Fig. 12 shows the positions of the index, ring, and little fingers centered on the middle finger.
[0173]
[0174]
[0175]
[0176]
[0177] And the joint data generation unit (220) uses the generated temporary intermediate point After extracting the vertices of each finger from, the end points (maximum or minimum values) of each finger (J EndBy determining the position, joints can be generated according to the ratio of each joint. The ratio of the index finger joints can be set to 5:2:1.6, the ratio of the ring finger joints to 5:1.9:1.7, and the ratio of the pinky finger joints to 5:2.3:1.3. Fingers are not distinguished by left-right symmetry. That is, the joint data generation unit (220) can calculate the joint data of the index finger, ring finger, and pinky finger as shown in Equation 25. Here, the scale number is assigned to each ratio.
[0178]
[0179]
[0180]
[0181]
[0182]
[0183]
[0184]
[0185]
[0186]
[0187] In this way, the joint data generation unit (220) can generate the 3D coordinate positions of the remaining 57 joints based on the 3D coordinate positions of the essential joints (input joints in Table 2). FIG. 13 shows the joints (left) and original joints (right) generated by the joint data generation unit.
[0188] The joint data validation unit (230) can perform a joint data validation process to check whether the joints generated by the joint data generation unit (220) are properly generated in the correct position. The joint data validation unit (230) can perform validation as in Equation 26 to check whether all generated joints are located inside the vertices of the character.
[0189]
[0190] The joint data validation unit (230) can perform validation as in Equation 27 to check whether the four joints of the Body classification are higher than the hip joint (J0) on the Y-axis.
[0191]
[0192] The joint data validation unit (230) has four joints of the Arm classification along the X-axis, right-hand joint (J 10 Larger than ) and left hand joint (J 34 Validation can be performed as in mathematical formula 28 to check if it is smaller than ).
[0193]
[0194]
[0195] The joint data validation unit (230) has four joints of the Leg classification along the Y-axis, including the right thigh joint (J 55 ) or left thigh joint (J 60 Validation can be performed as in mathematical formula 29 to check if it is smaller than ).
[0196]
[0197] The joint data validation unit (230) can perform validation as in Equation 30 to check whether the three joints of the Head classification are larger than the neck joint (J4) on the Y-axis.
[0198]
[0199] The joint data validation unit (230) can perform validation as in Equation 31 to check whether the 20 joints of each hand in the Finger classification are inside the vertex.
[0200]
[0201] The generated joints are created according to their positions regardless of their relationship, and the joint hierarchy data generation unit (240) can perform a joint hierarchy generation process that matches the relationships between the joints according to the characteristics of the human body. The joint hierarchy data generation unit (240) can match the relationships between the joints into a tree structure or a hierarchy structure. The relationships between the joints can be connected as a parent, which means the upper part of the joint, and a child, which means the lower part of the joint. The joint hierarchy data generation unit (240) can generate a joint hierarchy in which the generated joints are connected as a parent and a child, as shown in Table 3.
[0202]
[0203]
[0204]
[0205]
[0206]
[0207]
[0208]
[0209]
[0210]
[0211] The skinning data generation unit (250) can perform a weight painting process that sets the sensitivity of deformation and movement of the mesh according to the movement of the bone for the data matched to the joint hierarchy.
[0212] More specifically, the skinning data generation unit (250) has a center position J for each generated joint. stand Points can be generated, and vertices around the generated joints can be extracted, along with the conditional expression for each joint and J for extracting vertices around the joints. standAn α value can be generated to set the point. The skinning data generation unit (250) is J stand After setting the range of the set of vertices to be extracted by setting the points and α values, V satisfying the conditions can be extracted from all vertices, and J stand The distance can be calculated using the vertex position of each V extracted from the point and the Euclidean definition, and the calculated distance can be normalized to a scale between 0 and 1 and then stored in a data collection in the form of JSON data. That is, the skinning data generation unit (250) can perform a weight painting process for all joints as in Equation 32.
[0213]
[0214]
[0215]
[0216]
[0217]
[0218] The skinning data generation unit (250) can perform a weight painting process for the Spine bone in the Body classification as shown in Equation 33.
[0219]
[0220] The skinning data generation unit (250) can perform a weight painting process for the Chest bone in the Body classification as in Equation 34.
[0221]
[0222] The skinning data generation unit (250) can perform a weight painting process for the Upper Chest bone in the Body classification as in Equation 35.
[0223]
[0224]
[0225] The skinning data generation unit (250) can perform a weight painting process for the Shoulder bone in Arm classification as in Equation 36.
[0226]
[0227] The skinning data generation unit (250) can perform a weight painting process for the Upper Arm bone in Arm classification as in Equation 37.
[0228]
[0229] The skinning data generation unit (250) can perform a weight painting process for the Lower Arm bone in Arm classification as in Equation 38.
[0230]
[0231] The skinning data generation unit (250) can perform a weight painting process for the Hand bone in Arm classification as in Equation 39.
[0232]
[0233] The skinning data generation unit (250) can perform a weight painting process for the Upper Leg bone in the Leg classification as shown in Equation 40.
[0234]
[0235] The skinning data generation unit (250) can perform a weight painting process for the knee bone in the Leg classification as in Equation 41.
[0236]
[0237] The skinning data generation unit (250) can perform a weight painting process for the Foot bone in Leg classification as in Equation 42.
[0238]
[0239] The skinning data generation unit (250) can perform a weight painting process for the Toes bone in Leg classification as in Equation 43.
[0240]
[0241]
[0242] The skinning data generation unit (250) can perform a weight painting process for the neck bone in the head classification as shown in Equation 44.
[0243]
[0244] The skinning data generation unit (250) can perform a weight painting process for the Head bone in the Head classification as in Equation 45.
[0245]
[0246] Fingers can be classified into four categories: five fingers, three fingers, two fingers, and zero fingers. The measurement algorithm may differ for each classification method.
[0247] The K-Means algorithm, which belongs to unsupervised machine learning, can be used for classifying the five fingers. The K-Means algorithm is an algorithm that groups data with similar characteristics for classification. The set of vertices around the hand can be classified into five groups to create the boundary lines for each finger. After creating the boundary lines for each finger, the previously generated J n Assign weight values according to the position.
[0248] The classification method for the three fingers can utilize the position of the middle finger joint. A boundary line can be generated by calculating proportionally, and weight values can be assigned to the three fingers (thumb, middle finger, little finger).
[0249] The classification method for two fingers can utilize the positions of the thumb and middle finger joints. A boundary line can be generated by calculating proportionally, and weight values can be assigned to the two fingers (thumb, middle finger).
[0250] Zero fingers do not generate fingers, and can concentrate weights on the Hand joints using sets of vertices around the hand.
[0251] The skinning data generation unit (250) can perform a weight painting process for the fingers by classifying the five fingers using a method of classifying five fingers, extracting vertices from the entire set of vertices based on the Hand joint position values, with the right hand having a maximum value and the left hand having a minimum value, and applying the extracted sets of vertices to the K-Means algorithm based on the Z-axis to classify them into five clusters. FIG. 14 shows an example of classifying vertices into five clusters by applying the K-Means algorithm as a method of classifying five fingers.
[0252] The skinning data generation unit (250) can perform a weight painting process on the fingers using a classification method of three fingers as in mathematical formula 46.
[0253]
[0254]
[0255]
[0256]
[0257]
[0258]
[0259] The skinning data generation unit (250) can perform a weight painting process on the fingers using a classification method of two fingers as in Equation 47.
[0260]
[0261]
[0262]
[0263]
[0264] The skinning data generation unit (250) can perform a weight painting process for the first joint of each of the thumb, index finger, middle finger, ring finger, and little finger as in Equation 48.
[0265]
[0266] The skinning data generation unit (250) can perform a weight painting process for the second joint of each of the thumb, index finger, middle finger, ring finger, and little finger as in Equation 49.
[0267]
[0268] The skinning data generation unit (250) can perform a weight painting process for the third joint of each of the thumb, index finger, middle finger, ring finger, and little finger as in Equation 50.
[0269]
[0270] The data application unit (300) may include a character joint generation unit (310), a character skinning generation unit (320), and a character generation unit (330) to perform a data application process of applying data generated through a data generation process to character data.
[0271] The character data initially input into the character automatic rigging generation device has the rigging removed and only has mesh nodes, and the character joint generation unit (310) can perform a character joint generation process by creating a Skeleton node under the Root node in the character data and creating the joints of Table 3. At this time, the Type of all joints is eLimbNode type, the J0 joint is eRoot type, and the size of each joint can be the same as 1.0. The character joint generation unit (310) can generate each joint using a recursive function according to the structure of Table 3.
[0272] The character skinning generation unit (320) can perform a character skinning process that applies skinning according to the joint positions after the character's joints are generated. First, the character skinning generation unit (320) can search for mesh nodes and skeleton nodes of the character data, then create a skin node, and subsequently create a cluster node. The character skinning generation unit (320) can link the skeleton node to the cluster node and apply vertex and weight values to each cluster node. The character skinning generation unit (320) can add the cluster node with the applied weight value to the skin node, and then add the skin node to the mesh node to apply it.
[0273] The character generation unit (330) can perform a character generation process by exporting character data in which a character skinning process has been performed and creating (checking) a character using a character viewer program (e.g., Blender, Unity, etc.). FIG. 15 shows a character (left) created through the character generation process and the original character data.
[0274] The character automatic rigging generation device and method according to an embodiment of the present invention may be implemented as a computer device or a computer program executable on a computer device.
[0275] The drawings and detailed description of the invention referenced so far are merely exemplary of the invention and are used only for the purpose of explaining the invention, not to limit the meaning or the scope of the invention as defined in the claims. Therefore, those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the invention should be determined by the technical spirit of the appended claims. Explanation of the symbols
[0276] 100: Preprocessing section 110: Character Data Input Section 120: Essential Joint Input Section 130: Character Data Preprocessing Section 140: Essential Joint Preprocessing Unit 150: Essential Joint Validation Unit 200: Data generation section 210: Joint creation part 220: Joint Data Generation Unit 230: Joint Validation Unit 240: Joint Hierarchy Data Generation Unit 250: Skinning Data Generation Unit 300: Data Application Section 310: Character Joint Generation Unit 320: Character Skinning Generation Section 330: Character Creation
Claims
Claim 1 The preprocessing unit includes: a preprocessing unit that receives character data and essential joints, quantifies character mesh data, and performs a preprocessing process to correct the essential joints; a data generation unit that uses the data generated through the preprocessing process to generate joints of each segment according to the joint ratios of the human body, matches the relationships and structures of the generated joints, and generates skinning data; and a data application unit that performs a data application process to apply the data generated through the data generation process to the character data. The preprocessing unit comprises: a character data input unit that renders in 2D based on the character mesh data included in the character data, which is a 3D model file; an essential joint input unit that adds the positions of the essential joints to the data rendered in 2D based on the character mesh data; a character data preprocessing unit that quantifies the character mesh data of the mesh nodes in the character data; and an essential joint preprocessing unit that performs an essential joint preprocessing process to convert the 2D positions of the essential joints into positions on 3D coordinates. A character automatic rigging generation device comprising an essential joint validation unit that validates the position of the preprocessed essential joint to prevent the position of the essential joint from being outside the vertex position of the character or being entered at a different location after the essential joint preprocessing process. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 A character automatic rigging generation device according to claim 1, wherein the data generation unit receives information on 8 essential joints preprocessed through the preprocessing process and includes a joint generation unit that generates a total of 65 joints to fit the character by adding 57 joints to the 8 essential joints. Claim 7 In claim 6, the data generation unit further comprises a joint data generation unit that generates the 3D coordinate positions of the 57 joints based on the 3D coordinate positions of the 8 preprocessed essential joints, forming a character automatic rigging generation device. Claim 8 A character automatic rigging generation device comprising: a preprocessing unit that receives character data and essential joints, quantifies character mesh data, and performs a preprocessing process to correct the essential joints; a data generation unit that uses the data generated through the preprocessing process to generate joints of each segment according to the joint ratio of the human body, matches the relationships and structures of the generated joints, and generates skinning data; and a data application unit that performs a data application process to apply the data generated through the data generation process to the character data, wherein the data generation unit comprises: a joint generation unit that receives information on 8 essential joints preprocessed through the preprocessing process and adds 57 joints to the 8 essential joints to generate a total of 65 joints tailored to the character; a joint data generation unit that generates the 3D coordinate positions of the 57 joints based on the 3D coordinate positions of the 8 preprocessed essential joints; and a joint data validation unit that performs validation on the joints generated by the joint data generation unit. Claim 9 In claim 8, the data generation unit further comprises a joint hierarchy data generation unit that matches the relationship between joints to a joint hierarchy structure according to the characteristics of the human body, thereby creating a character automatic rigging generation device. Claim 10 In claim 9, the data generation unit further comprises a skinning data generation unit that performs a weight painting process for setting the sensitivity of mesh deformation and movement according to bone movement for data matched to the joint hierarchy, thereby forming a character automatic rigging generation device. Claim 11 A method for generating automatic character rigging, comprising: a step of receiving character data and essential joints as input, quantifying character mesh data, and performing a preprocessing process to correct the essential joints; a step of performing a data generation process to generate joints of each segment according to the joint ratio of the human body using the data generated through the preprocessing process, matching the relationships and structures of the generated joints, and generating skinning data; and a step of performing a data application process to apply the data generated through the data generation process to the character data, wherein the step of performing the preprocessing process includes: a step of rendering in 2D based on the character mesh data included in the character data, which is a 3D model file; a step of adding the positions of the essential joints to the data rendered in 2D based on the character mesh data; a step of quantifying the character mesh data of the mesh nodes in the character data; a step of performing an essential joint preprocessing process to convert the 2D positions of the essential joints into positions on 3D coordinates; and a step of validating the preprocessed positions of the essential joints after the essential joint preprocessing process to prevent cases where the positions of the essential joints deviate from the vertex positions of the character or are entered at different locations. Claim 12 delete Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 A method for generating automatic character rigging according to claim 11, wherein the step of performing the data generation process comprises receiving information on 8 essential joints preprocessed through the preprocessing process, and adding 57 joints to the 8 essential joints to generate a total of 65 joints to fit the character. Claim 17 A character automatic rigging generation method according to claim 16, wherein the step of performing the data generation process further comprises the step of performing a joint data generation process that generates the 3D coordinate positions of the 57 joints based on the 3D coordinate positions of the 8 preprocessed essential joints. Claim 18 A method for generating automatic character rigging, comprising: a step of receiving character data and essential joints as input, quantifying character mesh data, and performing a preprocessing process to correct the essential joints; a step of performing a data generation process to generate joints of each segment according to the joint ratio of the human body using the data generated through the preprocessing process, matching the relationships and structures of the generated joints, and generating skinning data; and a step of performing a data application process to apply the data generated through the data generation process to the character data, wherein the step of performing the data generation process includes: a step of receiving information on 8 essential joints preprocessed through the preprocessing process and adding 57 joints to the 8 essential joints to generate a total of 65 joints to fit the character; a step of performing a joint data generation process to generate the 3D coordinate positions of the 57 joints based on the 3D coordinate positions of the 8 preprocessed essential joints; and a step of performing validation on the joints generated by the joint data generation process. Claim 19 In claim 18, the step of performing the data generation process further includes the step of matching the relationship between joints to a joint hierarchy structure according to the characteristics of the human body, using the joints generated by the joint data generation process. Claim 20 A method for generating automatic character rigging according to claim 19, wherein the step of performing the data generation process further includes the step of performing a weight painting process for the data matched to the joint hierarchy, which sets the sensitivity of deformation and movement of the mesh according to the movement of the bone.
Citation Information
Patent Citations
Method and Apparatus For Using Rigging Character
KR1020200069786A
Method for generating character animation through extracting a character from an image and device using thereof
KR102264803B1
