Method for generating gesture connection in sign language
By dividing the hand action areas and establishing a transition action library, the coherent transition between 3D virtual human gesture language actions is achieved, and the problem of unnatural connection between gesture language actions in the prior art is solved, and visual effects and computing efficiency are improved.
Patent Information
- Application Number
- CN202210318044.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-29
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-03-29
AI Technical Summary
The prior art is difficult to achieve a coherent transition between 3D virtual human gesture language actions, resulting in poor visual effects. Especially when the two actions are far apart, the connection may appear weird and cannot simulate the natural transition of real human gestures.
By dividing the range of hand movements into multiple areas, recording the center position of each area, collecting and establishing a transition action library, generating gesture language connection actions, and using linear interpolation to achieve the transition of hand positions, from the current position to the center of the area to the center of the area Target position, reduce calculation amount, and enhance visual effects.
It realizes a smooth and realistic transition between gesture language actions, enhances visual effects, reduces calculation amount, and improves computing processing speed.
Smart Images

Figure CN115035275B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of sign language translation, and in particular relates to a method for generating gesture language action connections. Background Art
[0002] The "National Common Sign Language Vocabulary" and "National Common Braille Scheme" were officially released and implemented in 2018. They clearly include the sign language widely used by the majority of hearing-impaired people in real life, replace many sign languages corresponding to Chinese characters, pay attention to describing the changes in body movements and facial expressions when expressing sign language, and focus on reflecting the language characteristics of sign language in expressing shape and meaning. The expression of sign language relies on real-life comparative movements. With the development of artificial intelligence and the improvement of algorithm computing power, it is now possible to use 3D virtual people to make sign language movements, replace real people to express sign language, and serve the hearing-impaired. In order to make the sign language movements coherent and real, and to ensure the accuracy of the meaning of the sign language, the research on the sign language movement connection technology is of great significance.
[0003] The method for converting language text into 3D virtual human gestures in real time includes the following steps: 1. According to the standard word library of gestures, all words in the word library are used to create corresponding 3D virtual human gestures as an action library. 2. The text in a sentence is split into standard word sequences, and the 3D virtual human actions corresponding to each standard word are matched from the action library. 3. The 3D virtual human actions corresponding to each standard word in the standard word sequence are connected in series to form a set of continuous actions, which drive the 3D virtual human to perform gestures and output them to the screen.
[0004] The starting positions and postures of the hands of each individual standard sign language action are different. In step 3 of the above "implementation method", it is assumed that the end position of the left hand of the previous standard action is A, and the starting position of the left hand of the next standard action is B. Because A and B are in different positions, if A and B are simply connected in series, after a gesture is completed, the hand will jump from position A to position B, and there will be no transition between the two points, resulting in a poor visual effect. If you simply use linear interpolation to connect A and B, when A and B are far apart, the connection between A and B may be very strange, which is quite different from real-life actions. Summary of the invention
[0005] In view of the defects of the above-mentioned prior art, the purpose of the present invention is to provide a method for generating gesture language action connection to solve the problem of continuous transition of the hand from one position A to the next position B between two standard actions of 3D virtual human gesture language.
[0006] To achieve the above purpose, the technical solution adopted by the present invention is: a method for generating gesture language action connection, the specific steps are as follows:
[0007] Step 1: According to the standard word library of sign language, define the hand movement range of all sign languages, divide the hand movement range into N areas, and record the center position of each area;
[0008] Step 2: Collect the hand transition motions between the center positions of every two regions and establish a transition motion library;
[0009] Step 3: Select the area according to the hand position, correspond the hand position transition in the standard word sequence to the transition between areas, read the corresponding hand transition action in the transition action library, and generate the sign language connection action.
[0010] Furthermore, after the hand movement range is divided into N areas, there are a total of ((N-1) + (N-2) + ... + 2 + 1) types of hand transition movements between the center positions of every two areas, that is, a total of N (N-1) / 2 types of hand transition movements, forming a transition movement library.
[0011] Preferably, N is an integer and is not less than three.
[0012] Furthermore, the sign language standard vocabulary refers to a vocabulary formed based on the sign language representation of words in the "National Common Sign Language Vocabulary".
[0013] Furthermore, in step three, when the hand moves across areas according to the sign language action corresponding to the standard word sequence, the hand moves from the current position to the target position, first from the current position to the center position of the area where the current position is located, and then the hand transition action in the transition action library is called up, moving from the center position of the area where the current position is located to the center position of the area where the target position is located, and finally moving from the center position of the area where the target position is located to the target position.
[0014] Furthermore, the movement from the current position to the center position of the area where the current position is located is completed by linear interpolation; the movement from the center position of the area where the target position is located to the target position is completed by linear interpolation.
[0015] Furthermore, in step three, when the hand moves in the same area according to the sign language action corresponding to the standard word sequence, it moves from the current position to the target position, which is directly completed by linear interpolation.
[0016] Furthermore, the current position is the hand position at the end of the previous sign language action; the target position is the hand position at the start of the next sign language action, and the current position and the target position are the hand positions of the same hand.
[0017] Further, in step one, the region is a spatial range region, and the center position of the region is the center position of the spatial range region.
[0018] Furthermore, in step 2, the hand transition movement is formed by collecting motion data of various parts of the hands of multiple objects, and through sorting, feature analysis, evaluation and element extraction.
[0019] Furthermore, in step three, the area where the hand is located is selected based on the hand position. When the hand position is within a certain area, the area is selected as the area where the hand is located; when the hand position is simultaneously at the junction of multiple areas, the straight-line distance between the hand position and each area is determined, and the area with the shortest straight-line distance to the hand position is selected as the area where the hand is located.
[0020] Furthermore, the hand position is determined by the hand center position, and the position of each area is determined by the center position of each area; when the hand center position is located at the junction of multiple areas and the straight-line distance between the hand position and each area is the same, the area closest to the area to be transitioned is selected as the area.
[0021] Furthermore, when the center position of the hand is located at the junction of multiple areas and the distances between each area and the area to be transitioned are the same, one area is selected as the area according to an artificially set priority.
[0022] The beneficial effects of the present invention are as follows: the technical solution of the present invention ensures that the connection between two gesture language actions is smooth and real, and can enhance the visual effect. In addition, directly calling the transition action from the transition action library can reduce the amount of calculation, save programs, and speed up the calculation processing speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 This is a schematic diagram of the area division in step 1;
[0024] Figure 2 Sign language movements for the word "hope";
[0025] Figure 3 The hand gestures for the word "Qigong";
[0026] In the figure: A is the starting point of the sign language action. DETAILED DESCRIPTION
[0027] In order to make the structure and function of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0028] The present invention provides a method for generating gesture language action connection, and the specific steps are as follows:
[0029] Step 1: According to the standard word library of sign language, define the hand movement range of all sign languages, divide the hand movement range into N areas, and record the center position of each area;
[0030] Step 2: Collect the hand transition motions between the center positions of every two regions and establish a transition motion library;
[0031] Step 3: Select the area according to the hand position, correspond the hand position transition in the standard word sequence to the transition between areas, read the corresponding hand transition action in the transition action library, and generate the sign language connection action.
[0032] Furthermore, after the hand movement range is divided into N areas, there are a total of ((N-1) + (N-2) + ... + 2 + 1) types of hand transition movements between the center positions of every two areas, that is, a total of N (N-1) / 2 types of hand transition movements, forming a transition movement library.
[0033] Preferably, N is an integer and is not less than three.
[0034] Furthermore, the sign language standard vocabulary refers to a vocabulary formed based on the sign language representation of words in the "National Common Sign Language Vocabulary".
[0035] Furthermore, in step three, when the hand moves across areas according to the sign language action corresponding to the standard word sequence, the hand moves from the current position to the target position, first from the current position to the center position of the area where the current position is located, and then the hand transition action in the transition action library is called up, moving from the center position of the area where the current position is located to the center position of the area where the target position is located, and finally moving from the center position of the area where the target position is located to the target position.
[0036] Furthermore, the movement from the current position to the center position of the area where the current position is located is completed by linear interpolation; the movement from the center position of the area where the target position is located to the target position is completed by linear interpolation.
[0037] Furthermore, in step 3, when the hand moves in the same area according to the sign language action corresponding to the standard word sequence, the hand moves from the current position to the target position, and linear interpolation is directly used. When the current position and the target position are in the same area, linear translation is directly used, because in the same area, they are already very close, and direct translation does not look inconsistent.
[0038] Furthermore, the current position is the hand position at the end of the previous sign language action; the target position is the hand position at the start of the next sign language action, and the current position and the target position are the hand positions of the same hand.
[0039] Further, in step one, the region is a spatial range region, and the center position of the region is the center position of the spatial range region.
[0040] Based on the above technical solution, it should be noted that the area is divided within the spatial range, but when doing sign language, the hand moves within a range not too far in front of the body at the beginning of the sign language word, so although it is a spatial division, the thickness of the space in front of the body is not large.
[0041] Furthermore, in step 2, the hand transition movement is formed by collecting motion data of various parts of the hands of multiple objects, and through sorting, feature analysis, evaluation and element extraction.
[0042] Based on the above technical solution, it should be noted that the hand transition movement is produced by motion capture, hand adjustment, etc., and since the number of hand transition movements is limited, they can be produced. However, the transition in real sign language may be from any position to any position, which cannot be produced, so the method of the present invention is needed to generate it.
[0043] Furthermore, in step three, the area where the hand is located is selected based on the hand position. When the hand position is within a certain area, the area is selected as the area where the hand is located; when the hand position is simultaneously at the junction of multiple areas, the straight-line distance between the hand position and each area is determined, and the area with the shortest straight-line distance to the hand position is selected as the area where the hand is located.
[0044] Furthermore, the hand position is determined by the hand center position, and the position of each area is determined by the center position of each area; when the hand center position is located at the junction of multiple areas and the straight-line distance between the hand position and each area is the same, the area closest to the area to be transitioned is selected as the area.
[0045] Furthermore, when the center position of the hand is located at the junction of multiple areas and the distances between each area and the area to be transitioned are the same, one area is selected as the area according to an artificially set priority.
[0046] Based on the above technical solution, when the hand is at the intersection of multiple areas at the same time, the one with the closest straight-line distance to the center of the area is selected; in the extreme case, the hand center is at the intersection of multiple areas, and the straight-line distances to the centers of each area are almost the same, then the area with the center closest to the target position is selected as the area to which it belongs. If the centers of two areas are also at the same distance from the target position, then one area can be selected as the area to which it belongs.
[0047] See attached Figure 1 Taking the left hand as an example, the front section of the human body is divided into 6 areas, K1, K2...K6. The action of the 3D virtual human hand transitioning from the center Ki of any area to the center Kj of another area is collected and made as the transition action Gij to form a transition action library.
[0048] Find out the area Ka closest to the hand position A and the area Kb closest to the hand position B, then find the transition action Gab between the areas Ka and Kb in the transition action library.
[0049] The movement of A to the center area of area Ka is completed by linear interpolation. Because A is relatively close to Ka, the connection effect is better even if simple linear interpolation is used.
[0050] Similarly, linear interpolation is also used for the movement of the center of area Kb to B.
[0051] In this way, the transition from A to B becomes:
[0052]
[0053] In this way, the hand can be transitioned from position A to position B to achieve a smooth and realistic effect.
[0054] The above method divides the section into 6 areas. In practice, if more precise actions are required, more areas can be divided.
[0055] See attached Figure 2 and attached Figure 3 , using the hand sign "hope" to the hand sign "qigong" to explain the transition. Figure 2 and Figure 3 According to the zone division in the exercise, when the "Hope" movement ends, the right hand is in the K1 zone; when the "Qigong" movement begins, the right hand is in the K3 zone.
[0056] When you need to string together "hope" and "qigong" to express it coherently, the following method can be used to achieve it:
[0057] 1) Move the right hand linearly from the "desired" end position directly to the center of K1
[0058] 2) Link the G13 action in the transition action library and move the right hand to the center of K3
[0059] 3) The right hand moves linearly from the center of K3 directly to the starting position of "Qigong"
[0060] The above examples are only the best embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments, and many variations are possible. All variations that can be directly derived or associated with the contents disclosed by a person skilled in the art should be considered as the protection scope of the present invention.
Claims
1. A method for generating gesture language action connection, characterized in that: The specific steps are as follows: Step 1: According to the standard word library of sign language, define the hand movement range of all sign languages, divide the hand movement range into N areas, and record the center position of each area; The region is a spatial range region, and the center position of the region is the center position of the spatial range region; Step 2: Collect the hand transition motions between the center positions of every two regions and establish a transition motion library; The hand transition motion is formed by collecting motion data of various parts of the hands of multiple subjects, and by sorting, feature analysis, evaluation and element extraction; Step 3: Select the region according to the hand position, correspond the hand position transition in the standard word sequence to the transition between regions, read the corresponding hand transition action in the transition action library, and generate the sign language connection action; When the hand moves across regions according to the sign language action corresponding to the standard word sequence, the hand moves from the current position to the target position, first from the current position to the center of the region where the current position is located, then the hand transition action in the transition action library is called, and the hand moves from the center of the region where the current position is located to the center of the region where the target position is located, and finally from the center of the region where the target position is located to the target position; The movement from the current position to the center position of the area where the current position is located is completed by linear interpolation; the movement from the center position of the area where the target position is located to the target position is completed by linear interpolation.
2. The method for generating gesture language action connection according to claim 1, characterized in that: For step three, when the hand moves in the same area according to the sign language action corresponding to the standard word sequence, it moves from the current position to the target position, which is directly completed by linear interpolation.
3. The method for generating gesture language action connection according to claim 1 or 2, characterized in that: The current position is the hand position at the end of the previous sign language action; the target position is the hand position at the start of the next sign language action, and the current position and the target position are the hand positions of the same hand.
4. The method for generating gesture language action connection according to claim 1, characterized in that: In step three, the area is selected based on the hand position. When the hand position is within a certain area, the area is selected as the area. When the hand position is at the junction of multiple areas at the same time, the straight-line distance between the hand position and each area is determined, and the area with the shortest straight-line distance to the hand position is selected as the area.
5. The method for generating gesture language action connection according to claim 4, characterized in that: The hand position is determined by the hand center position, and the positions of each area are determined by the center positions of each area; when the hand center position is located at the junction of multiple areas, the area closest to the area to be transitioned is selected as the area.
6. The method for generating gesture language action connection according to claim 1, characterized in that: The sign language standard vocabulary refers to a vocabulary formed based on the sign language representation of words in the "National Common Sign Language Vocabulary".
Citation Information
Patent Citations
Computer assisted hand language communication method under special session context
CN101527092A
Sign language video synthesis method, sign language translation system, medium and electronic equipment
CN112073749A
Sign language inter-translation method and system
CN113407034A
Method and system for playing animation of sign language
CN113538632A
Sign language editing method and device
JP2005099977A
