Information processing device and program
The device addresses the issue of unnatural sign language animations by extracting and synthesizing fixed expressions in sign language animations from Japanese text, ensuring natural and understandable movements.
Patent Information
- Application Number
- JP2023080792
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-16
- Publication Date
- 2024-11-28
AI Technical Summary
Conventional methods for generating sign language CG animation from Japanese text result in unnatural movements due to the use of fixed expressions unique to sign language, as they connect sign language words in a standard form without considering contextual changes.
An information processing device that includes a sign language translation unit to extract fixed expressions from Japanese text and a motion connection unit to synthesize motion data using template data for these expressions, ensuring natural sign language movements are reproduced.
The device generates motion data that accurately reproduces fixed expressions unique to sign language, resulting in more natural and understandable sign language CG animations.
Smart Images

Figure 2024164997000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an information processing device and a program. [Background technology]
[0002] For example, automatic sign language video generation technology that combines Japanese-to-sign language translation with computer graphics (CG) animation to generate sign language videos from Japanese text has the potential to be widely used to provide information to hearing-impaired people whose native language is sign language, or in situations where communication is required between hearing people and hearing-impaired people.
[0003] When translating from a source language to sign language, a method has been known in the art in which a string of sign language words is first converted from the source language, CG corresponding to each word is extracted from the converted string of sign language words, and the extracted CGs are linked to generate a sign language image (see, for example, Patent Document 1). [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2014-21180 A Summary of the Invention [Problem to be solved by the invention]
[0005] Conventional methods for generating sign language images (sign language CG animation) using Japanese text as input involve reading the motion data of sign language words according to the sequence of sign language words that is the translation result, and then interpolating between the motion data arranged in chronological order to generate the motion of the sign language image that corresponds to the original Japanese text.
[0006] Conventional methods for generating sign language CG animations using Japanese text as input connect the motions of sign language words in their standard form, known as dictionary form, which means that it is not possible to use standard expressions specific to sign language that are frequently used in sign language, resulting in the generated sign language CG animations having unnatural movements.
[0007] An object of the present disclosure is to generate motion data for sign language video that reproduces standard expressions specific to sign language from Japanese text. [Means for solving the problem]
[0008] The present disclosure is an information processing device having a sign language translation unit that translates Japanese text to generate a sign language video into a string of sign language words, and a motion connection unit that generates motion data of a sign language video corresponding to the Japanese text in accordance with the string of sign language words, wherein the sign language translation unit extracts standard expressions contained in the Japanese text and translates the Japanese text excluding the standard expressions into a string of sign language words, and the motion connection unit generates motion data of a sign language video connecting between motion data of standard expressions contained in the Japanese text synthesized using template data for standard expression synthesis, and motion data of sign language words in the Japanese text excluding the standard expressions. Effect of the Invention
[0009] According to the present disclosure, motion data for sign language video that reproduces standard expressions specific to sign language can be generated from Japanese text. [Brief description of the drawings]
[0010] [Figure 1] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device according to the present embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device according to the present embodiment. [Diagram 3]FIG. 11 is an explanatory diagram of an example of a technique for connecting motion data according to a string of sign language words resulting from translation when generating sign language CG animation from Japanese text. [Figure 4] FIG. 13 is an explanatory diagram of an example of a method for reproducing fixed expressions in sign language when generating sign language CG animation from Japanese text. [Diagram 5] FIG. 11 is an explanatory diagram of an example of a fixed expression other than a date. [Figure 6] 5 is a flowchart illustrating an example of processing of the information processing device according to the present embodiment. [Figure 7] FIG. 1 is a diagram illustrating an example of an information processing system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] Hereinafter, an embodiment of the present disclosure (hereinafter, referred to as the present embodiment) will be described with reference to the drawings.
[0012] <Summary> In this embodiment, when translating Japanese text into a string of sign language words, parts that become standard expressions in sign language are extracted, and when connecting motion data, a synthesis process is applied to the standard expression parts using template data for standard expression synthesis, thereby reproducing movements that are closer to actual sign language expressions in sign language CG animation, which is an example of sign language video.
[0013] When speaking in sign language, a sentence is constructed by listing a number of sign words in chronological order, and each sign word frequently undergoes inflection in accordance with the context or the words before and after it.
[0014] For example, when a date such as May 8th is expressed in sign language, a standard expression is used in which the numbers for the month and day are presented simultaneously using both hands, one above the other. Also, when a number is accompanied by a unit such as % (percent) or m (meter), a standard expression is used in which the number and unit are presented by shifting them to the side in order. Furthermore, when expressing a year or time, a standard expression is used in which only the first number is presented with the left hand, held there, and the remaining numbers are presented by shifting them with the right hand.
[0015] Such fixed expressions are frequently used in sign language, which is a visual language, to convey information more intuitively. Therefore, if fixed expressions are not used, the sign language CG animation will be unnatural and difficult to understand.
[0016] On the other hand, in generating sign language CG animation using Japanese text as input, there is a method that uses machine translation to translate from Japanese to sign language (JP Patent Publication 2014-21180 A). In machine translation from Japanese to sign language, a string of sign language words in chronological order is output as the translation result.
[0017] The sign language movements used in sign language CG animation are motion captured from actual sign language movements, including hand and facial expressions, and saved as motion data in file formats such as BVH (Biovision Hierarchy). Motion data for sign language words is recorded on a word or phrase basis. Motion data for sign language videos is generated by arranging the motion data for sign language words in chronological order and connecting the sign language words by interpolating between them. Similarly, when generating sign language CG animation from Japanese text, the motion data for multiple sign language words included in the translated sign language word string is read, and the motion data to be used in the final sign language CG animation is generated by interpolating between the motion data and connecting them.
[0018] The motion data of the sign language words used here is the standard form found in dictionaries, known as dictionary form. Standard form motion data is basically expressed in front of the speaker, and the right hand is used when only one hand is used. Therefore, simply connecting the motion data of a string of sign language words as is will not be able to reproduce the fixed expressions unique to sign language, and will simply present the motion data of the sign language words in order, resulting in an unnatural and difficult to understand sign language CG animation.
[0019] Therefore, in this embodiment, when translating Japanese text into a string of sign language words, parts that become standard expressions in sign language are extracted and the standard expression parts are replaced with standard expression flags, and template data for standard expression synthesis is loaded and used when connecting motion data.
[0020] In this embodiment, motion data for sign language CG animation that reproduces fixed expressions specific to sign language can be automatically generated from input Japanese text, making it possible to generate sign language CG animation that reproduces natural sign language. Furthermore, in this embodiment, by adding a pair of a flag for a fixed expression in sign language and template data for synthesis of the fixed expression, it is possible to easily accommodate new fixed expressions.
[0021] <Hardware configuration> An information processing device 10 according to this embodiment is realized, for example, by the configuration shown in Fig. 1. Fig. 1 is a diagram showing an example of the hardware configuration of the information processing device according to this embodiment.
[0022] The information processing device 10 in FIG. 1 includes, for example, a CPU (Central Processing Unit) 500, a storage device 502, a communication device 504, an input device 506, an output device 508, and a media I / F 510.
[0023] The CPU 500 controls the information processing device 10 according to a program. The storage device 502 is, for example, a memory such as a ROM (Read Only Memory) or a RAM (Random Access Memory), or a storage such as a HDD (Hard Disk Drive) or an SSD (Solid State Drive). The storage device 502 stores the programs and data executed by the CPU 500.
[0024] The communication device 504 is a communication device such as a network circuit that controls communication. The input device 506 is an input device such as a touchpad, a controller, a mouse, a keyboard, a camera, a microphone, etc. The output device 508 is an output device such as a display, a speaker, etc. A touch panel is realized by combining a touchpad, which is an example of the input device 506, with a display, which is an example of the output device 508. The media I / F 510 controls reading and writing of data from and to a media 512 such as a USB memory, a flash memory, etc. The hardware configuration in FIG. 1 is an example.
[0025] <Functional configuration> Fig. 2 is a diagram showing an example of the functional configuration of an information processing device according to this embodiment. Note that the functional configuration of Fig. 2 is just an example, and for example, at least a part of the functional configuration of the information processing device 10 shown in Fig. 2 may be provided in another information processing device so as to perform processing in cooperation with the other information processing device.
[0026] The functional configuration of the information processing device 10 in Fig. 2 is realized by processing executed by one or more programs on the CPU 500. The information processing device 10 has a Japanese text storage unit 20, a sign language translation unit 30, a motion connection unit 40, and an image generation unit 50. The sign language translation unit 30 has a text morphological analysis unit 31, a fixed expression information extraction unit 32, a translation unit 33, and a fixed expression information insertion unit 34. The motion connection unit 40 has a fixed expression synthesis template reading unit 41, a motion data reading unit 42, a motion interpolation unit 43, a fixed expression synthesis template DB 44, and a sign language word motion DB 45. The image generation unit 50 has a CG animation generation unit 51 and a sign language avatar data storage unit 52.
[0027] The Japanese text storage unit 20 stores Japanese text for generating sign language CG animation. The sign language translation unit 30 translates the Japanese text for generating sign language CG animation into a string of sign language words. The text morphological analysis unit 31 of the sign language translation unit 30 divides the input Japanese text into words, classifies them into parts of speech, and outputs the results of the morphological analysis.
[0028] When a specific noun (such as a date, a number + unit, a year, a time, or an abbreviation) that is a standard expression in sign language is included in the Japanese text, the standard expression information extraction unit 32 extracts the specific noun portion as a standard expression. The standard expression information extraction unit 32 outputs the morphological analysis result in a state in which the standard expression portion (specific noun portion) is temporarily replaced with a standard expression flag. The standard expression information extraction unit 32 extracts and outputs information on the standard expression portion (standard expression information).
[0029] The translation unit 33 translates the morphological analysis result in which the fixed expression portion is temporarily replaced with the fixed expression flag into a sign language word string, and outputs the translated result. The fixed expression information insertion unit 34 inserts the fixed expression information extracted by the fixed expression information extraction unit 32 into the fixed expression flag portion of the sign language word string, which is the translation result, and outputs the translated result to the fixed expression synthesis template reading unit 41 of the motion connection unit 40 as a sign language word string with the fixed expression flag.
[0030] As a method for translating the result of morphological analysis in which the fixed expression part is temporarily replaced with the fixed expression flag into a sign language word string, a method using replacement tag information called a placeholder (see, for example, JP 2020-166672 A) can be applied, but is not limited to this. In addition, if the Japanese text for generating the sign language CG animation does not contain a noun that corresponds to a fixed expression, the processing of the fixed expression information extraction unit 32 and the fixed expression information insertion unit 34 is omitted. The sign language translation unit 30 outputs the sign language word string with the fixed expression flag attached to the motion connection unit 40.
[0031] The motion connection unit 40 receives a string of sign language words with a fixed expression flag attached thereto from the sign language translation unit 30. The fixed expression synthesis template reading unit 41 reads out template data for fixed expression synthesis corresponding to the fixed expression flag from the fixed expression synthesis template DB 44. The fixed expression synthesis template DB 44 stores the fixed expression flag and the fixed expression synthesis template data in association with each other.
[0032] The motion data reading unit 42 reads out the motion data of the sign language words listed in the sign language word string from the sign language word motion DB 45. The sign language word motion DB 45 stores the sign language words and the motion data in association with each other.
[0033] In addition, the motion data reading unit 42 inserts the template data for fixed expression synthesis read by the fixed expression synthesis template reading unit 41 into the corresponding location in the sign language word string, and also reads out the motion data of the sign language words used in the template from the sign language word motion DB 45.
[0034] The motion interpolation unit 43 generates motion data for sign language CG animation by interpolating and connecting the motion data read by the motion data reading unit 42. Note that for the portion of template data for synthesis of fixed expressions inserted at the corresponding position in the sign language word string, a process for synthesis of fixed expressions, described below, is performed before the interpolation process by the motion interpolation unit 43 to create a state of one sign language word arranged in time series, and then the motion interpolation unit 43 applies the interpolation process and connects it to the previous and next motion data. For the portion of template data for synthesis of fixed expressions inserted at the corresponding position in the sign language word string, natural sign language CG animation motion data reproducing fixed expressions specific to sign language can be generated, so that motion data with movements closer to actual sign language can be generated compared to the case where motion data for sign language word strings is directly connected to generate motion data for sign language CG animation.
[0035] The motion connection unit 40 outputs the motion data of the generated sign language CG animation to the image generation unit 50. The image generation unit 50 receives the motion data of the sign language CG animation from the motion connection unit 40.
[0036] The CG animation generation unit 51 generates a sign language video in which an avatar moves according to the motion data of the sign language CG animation input from the motion connection unit 40. The sign language avatar data storage unit 52 stores sign language avatar data, which is an example of model data for sign language. The CG animation generation unit 51 combines the motion data of the sign language CG animation input from the motion connection unit 40 with the sign language avatar data read from the sign language avatar data storage unit 52 and in a standby state, renders the sign language CG animation, and presents it to the user.
[0037] <Processing> A specific process in this embodiment will be described with an example of generating a sign language CG animation from Japanese text that includes an expression for "date," which is a fixed expression in sign language.
[0038] Fig. 3 is an explanatory diagram of an example of a method for connecting motion data according to the sign language word string of the translation result when generating sign language CG animation from Japanese text. Fig. 3 shows an example of generating sign language CG animation from Japanese text, with the Japanese text being "The day after tomorrow, April 28th, is expected to be sunny nationwide."
[0039] "April 28th" in the Japanese text in Figure 3 is the date. In sign language, a fixed expression is used in which the numbers of the month and day are presented simultaneously, one above the other, with the left and right hands. In the method in Figure 3, the motion data of each sign language word read out according to the sign language word string "{N} {the day after tomorrow} {April} {28} {day} {N} {Japan} ..." translated by a translation AI (Artificial Intelligence) is interpolated and connected in dictionary form to generate motion data for a sign language CG animation. The sign language word {N} represents a "nod."
[0040] The part of the date expression that should be a fixed expression, "April 28th," is translated into the three sign language words "{April} {28} {day}" in the sign language word sequence. Each motion data read out according to the translated sign language word sequence is all in dictionary form.
[0041] For example, the motion data of the sign language word "{April}" is a movement expressed in front of the face of the speaker, which is a movement representing the number "4" with the left hand and a movement representing "month" with the right hand. The motion data of the sign language word "{28}" is a movement expressed in front of the face of the speaker, which is a movement representing the number "28" with the right hand. The motion data of the sign language word "{day}" is a movement expressed in front of the face of the speaker, which is a movement representing the Chinese character "day" with both hands.
[0042] In a sign language CG animation that does not use set expressions, the motion data between sign language words is interpolated and connected, and the three dictionary-form motion data of "{April} {28} {day}" are played in order. Therefore, the set expressions unique to sign language cannot be reproduced, and it becomes a sign language CG animation with movements different from actual sign language. Specific examples of set expressions other than date expressions in sign language will be described later.
[0043] FIG. 4 is an explanatory diagram of an example of a method for reproducing a set expression part in sign language when generating a sign language CG animation from Japanese text. The Japanese text in FIG. 4 is the same as that in FIG. 3. When the Japanese text in FIG. 4 is input, the text morphological analysis unit 31 of the sign language translation unit 30 divides the input Japanese text into words for each word, classifies the part-of-speech, and outputs it as a morphological analysis result.
[0044] Since the set expression information extraction unit 32 includes a specific noun (date) that is a set expression in sign language in the Japanese text, it extracts the date "April 28th" as set expression information and temporarily replaces the "April 28th" part of the date with the set expression flag "date". The set expression information extraction unit 32 outputs the morphological analysis result in the state where the "April 28th" part of the date is temporarily replaced with the set expression flag "date" ("the day after tomorrow", "date", "is", ",", "the whole country",...) to the translation unit 33.
[0045] The translation unit 33 translates the morphological analysis result in which the date "April 28" has been temporarily replaced with the fixed expression flag "date" into a sign language word string and outputs the translated result. The fixed expression information insertion unit 34 receives the sign language word string "{N} {the day after tomorrow} {date} {N} {Japan} ..." output from the translation unit 33 as the translation result.
[0046] The fixed expression information insertion unit 34 replaces the fixed expression flag {date} portion of the sign language word string "{N} {the day after tomorrow} {date} {N} {Japan} ..." output as the translation result with {date: April 28th}, using fixed expression information that the fixed expression flag "date" is "April 28th". The fixed expression information insertion unit 34 inputs the sign language word string "{N} {the day after tomorrow} {date: April 28th} {N} {Japan} ..." with the fixed expression flag attached to it to the motion connection unit 40 as the output result of the sign language translation unit 30. Note that if it can be determined that the Japanese text does not contain fixed expressions in sign language at the time of morphological analysis, the processes of the fixed expression information extraction unit 32 and the fixed expression information insertion unit 34 may be omitted.
[0047] The motion connection unit 40 performs the process of synthesizing a fixed expression using template data as follows.
[0048] The fixed expression synthesis template reading unit 41 checks the fixed expression flag in the input sign language word string "{N} {the day after tomorrow} {date: April 28th} {N} {Japan} ...." In this example, based on the fixed expression flag {date: April 28th}, template data for date fixed expression synthesis is read from the fixed expression synthesis template DB 44.
[0049] Template data for synthesizing standard expressions for dates is information for synthesizing motion data in which the numbers of the month and day are presented simultaneously, one above the other, with the left and right hands. For example, with template data for synthesizing standard expressions for dates, the month motion and day motion are connected to synthesize standard expressions for dates, as shown in the example in the lower part of Figure 4. The month motion represents the month of the date selected from January to December. The day motion has only the left hand of the month motion remaining (retaining the rotation information of the final frame of the left arm joint), and the presentation position of the day motion representing the day of the date selected from January to December is moved in parallel (the wrist position is moved below the left hand of the moon motion using IK (Inverse Kinematics) processing), and the month and day of the date are presented simultaneously, one above the other.
[0050] When the input sign language word string does not include a standard expression flag, standard expression synthesis template reading unit 41 omits the process of reading template data for standard expression synthesis from standard expression synthesis template DB 44.
[0051] The motion data reading unit 42 reads out motion data corresponding to each sign language word in the sign language word string from the sign language word motion DB 45. The motion data reading unit 42 also reads out from the sign language word motion DB 45 motion data used in template data for synthesis of fixed expression.
[0052] The motion interpolation unit 43 performs a process of synthesizing a standard expression of the standard expression flag "Date: April 28th", and generates motion data of a standard expression of a date in which the left hand holds the number "April" and the right hand expresses "28th" below it, as shown in "Process of synthesizing standard expression using a template" in Fig. 4. After that, the previously generated standard expression of the date is treated as motion data of a single sign language word, "April 28th". The motion interpolation unit 43 generates motion data of a sign language CG animation by interpolating and connecting the motion data of the sign language words.
[0053] Therefore, when the Japanese text for which the sign language CG animation is to be generated contains a specific noun that is a standard expression in sign language, the motion connection unit 40 can generate natural sign language CG animation motion data for the specific noun that reproduces the standard expression specific to sign language.
[0054] Fig. 5 is an explanatory diagram of an example of a fixed expression other than a date. Fig. 5 shows fixed expressions of numbers + units, year / time, and abbreviations as examples of fixed expressions other than dates. The left side of Fig. 5 shows an example of presenting dictionary-form sign language words in order. The right side of Fig. 5 shows an example of presenting a fixed expression by performing fixed expression synthesis processing using template data on the fixed expression portion.
[0055] If dictionary-type sign language words are presented in order as shown on the left side of Fig. 5, fixed expressions specific to sign language cannot be reproduced, resulting in a sign language CG animation with movements different from those of actual sign language. In this embodiment, fixed expressions specific to sign language are reproduced by performing fixed expression synthesis processing using template data on the fixed expression portion and presenting it as shown on the right side of Fig. 5.
[0056] The CG animation generation unit 51 of the video generation unit 50 combines the motion data of the sign language CG animation input from the motion connection unit 40 with the sign language avatar data read out from the sign language avatar data storage unit 52 and in a standby state, renders the sign language CG animation, and presents it to the user.
[0057] 6 is a flowchart showing an example of processing of the information processing device according to this embodiment. In step S100, Japanese text for generating a sign language CG animation is input to the information processing device 10. In step S102, the text morphological analysis unit 31 of the sign language translation unit 30 divides the input Japanese text into words, classifies them into parts of speech, and outputs the results of the morphological analysis.
[0058] In step S104, the fixed expression information extraction unit 32 determines whether or not a specific noun that is a fixed expression in sign language (such as date, number + unit, year, time, or abbreviation) is included in the Japanese text for generating the sign language CG animation.
[0059] If a specific noun that is a standard expression in sign language is included in the Japanese text for generating the sign language CG animation, the standard expression information extraction unit 32 proceeds to step S108. In step S108, the standard expression information extraction unit 32 performs a standard expression flag replacement process and then proceeds to step S110. The standard expression flag replacement process is a process for temporarily replacing a standard expression part (specific noun part) with a standard expression flag. If a specific noun that is a standard expression in sign language is not included in the Japanese text for generating the sign language CG animation, the standard expression information extraction unit 32 proceeds to step S110.
[0060] In step S110, the translation unit 33 translates the morphological analysis result in which the fixed expression portion has been temporarily replaced with the fixed expression flag into a sign language word string, and outputs the translated result. In other words, the translation unit 33 translates the Japanese text excluding the fixed expression portion into a sign language word string by temporarily replacing it with the fixed expression flag.
[0061] In step S112, the fixed expression information insertion unit 34 performs a fixed expression information insertion process on the sign language word string that is the translation result. The fixed expression information insertion process is a process of inserting the fixed expression information extracted and output by the fixed expression information extraction unit 32 into the fixed expression flag portion of the sign language word string that is the translation result. The fixed expression information insertion unit 34 outputs the sign language word string with the fixed expression flag attached to the motion connection unit 40.
[0062] In step S114, the fixed expression synthesis template reading unit 41 of the motion connecting unit 40 receives the sign language word string with the fixed expression flag from the sign language translating unit 30. The fixed expression synthesis template reading unit 41 reads out template data for fixed expression synthesis corresponding to the fixed expression flag from the fixed expression synthesis template DB 44.
[0063] In step S116, the motion data reading unit 42 inserts the template data for fixed expression synthesis read by the fixed expression synthesis template reading unit 41 into the corresponding location in the sign language word string, and reads out from the sign language word motion DB 45 the motion data of the sign language words used in the template and the motion data of the sign language words listed in the sign language word string.
[0064] In step S118, the motion interpolation unit 43 performs a process of synthesizing fixed expressions on the template data inserted at the corresponding position in the sign language word string, to make it into a state of one sign language word arranged in time series, before the interpolation process by the motion interpolation unit 43. The motion interpolation unit 43 also generates motion data of a sign language CG animation by interpolating and connecting between the motion data that has been processed into a state of one sign language word by performing the process of synthesizing fixed expressions and the motion data of sign language words other than the fixed expressions.
[0065] In step S120, the video generation unit 50 receives the motion data of the sign language CG animation from the motion connection unit 40. The CG animation generation unit 51 generates a sign language video in which the avatar moves in accordance with the input motion data of the sign language CG animation.
[0066] According to this embodiment, when the input Japanese text contains specific nouns that are fixed expressions in sign language, by applying a fixed expression synthesis template to the corresponding parts of the translated sign language word string, it is possible to automatically generate more natural sign language CG animation that is closer to the actual sign language expression.
[0067] <Other embodiments> Although the above embodiment has been described by way of example with the configuration of the information processing device 10 in Fig. 1, it may also be realized in an information processing system in which a plurality of information processing devices 10 perform processing in cooperation with each other. Also, the above embodiment may be realized by the information processing device 10 calling and using an API (Application Programming Interface) provided by the sign language translation device 101, the motion connection device 102, and the image generation device 103 via a network 110, as in the information processing system in Fig. 7. The network 110 may be any communication network capable of data communication, such as the Internet or a LAN (Local Area Network).
[0068] The sign language translation device 101 is an example of an information processing device that provides the function of the sign language translation unit 30 in Fig. 2 via an API. The motion connection device 102 is an example of an information processing device that provides the function of the motion connection unit 40 in Fig. 2 via an API. The image generation device 103 is an example of an information processing device that provides the function of the image generation unit 50 in Fig. 2 via an API.
[0069] In addition, in an information processing system in which a plurality of information processing devices 10 cooperate to perform processing, an information processing device 10 having the functions of the sign language translation unit 30 and the motion connection unit 40 in Fig. 2 and an information processing device 10 having the functions of the image generation unit 50 in Fig. 2 may cooperate to perform processing. Furthermore, in an information processing system in which a plurality of information processing devices 10 cooperate to perform processing, an information processing device 10 having the functions of the sign language translation unit 30 in Fig. 2, an information processing device 10 having the functions of the motion connection unit 40 in Fig. 2, and an information processing device 10 having the functions of the image generation unit 50 in Fig. 2 may cooperate to perform processing.
[0070] The information processing system of the disclosed embodiment should be considered as an example and not a limitation. The above embodiment can be modified and improved in various forms without departing from the scope of the appended claims and the spirit thereof. In addition, the matters described in the above-mentioned embodiments can be configured in other ways as long as they are not inconsistent, and can be combined as long as they are not inconsistent. [Explanation of symbols]
[0071] 10. Information processing device 20 Japanese text storage section 30 Sign Language Translation Department 31 Text Morphological Analysis Unit 32 Fixed expression information extraction part 33 Translation Department 34 Fixed expression information insertion section 40 Motion Connection 41 Fixed expression synthesis template reading section 42 Motion data reading section 43 Motion Interpolation Section 44 Fixed expression synthesis template DB 45 Sign Language Word Motion DB 50 Image Generation Unit 51 CG Animation Generation Department 52 Sign Language Avatar Data Storage Unit
Claims
1. a sign language translation unit that translates Japanese text into a string of sign language words to generate a sign language image; a motion connection unit that generates motion data of a sign language image corresponding to the Japanese text according to the sign language word string; having the sign language translation unit extracts fixed expressions included in the Japanese text, and translates the Japanese text excluding the fixed expressions into a sign language word string; The motion connection unit is an information processing device that generates motion data of a sign language image by connecting between motion data of a standard expression contained in the Japanese text synthesized using template data for standard expression synthesis, and motion data of sign language words in the Japanese text excluding the standard expressions.
2. the sign language translation unit extracts fixed expressions included in the Japanese text, translates the Japanese text in a state in which the extracted parts of the fixed expressions are replaced with flags for fixed expressions, into a string of sign language words, and outputs the string of sign language words with the flags for fixed expressions attached thereto; The motion connection unit reads out template data for synthesis of fixed expressions corresponding to the flag for fixed expressions, synthesizes motion data of fixed expressions included in the Japanese text using the template data for synthesis of fixed expressions to create a state of one sign language word, and then generates motion data of a sign language video in which the motion data of the fixed expression and the motion data of the sign language word of the Japanese text excluding the fixed expression are connected.
2. The information processing device according to claim 1.
3. an image generating unit that generates a sign language image in which an avatar moves in accordance with the motion data generated by the motion connecting unit; 3. The information processing apparatus according to claim 1, further comprising:
4. The fixed expression is a date, a number + unit, a year, a time, or an abbreviation.
3. The information processing device according to claim 1 or 2.
5. The template data for the fixed expression synthesis includes information on the motion data used to synthesize the fixed expression with the motion data, and information on synthesis processing using the motion data.
3. The information processing device according to claim 1 or 2.
6. In the information processing device, A sign language translation step for translating Japanese text into a string of sign language words to generate a sign language image; a motion connection step of generating motion data of a sign language image corresponding to the Japanese text according to the sign language word string; Run the command, The sign language translation procedure includes: Extracting fixed expressions contained in the Japanese text; Translating the Japanese text, excluding the fixed expressions, into a string of sign language words; The motion connection procedure includes: Generate motion data of a sign language video by connecting motion data of a fixed expression included in the Japanese text synthesized using template data for synthesizing fixed expressions and motion data of sign language words of the Japanese text excluding the fixed expressions. program.
Citation Information
Patent Citations
Sign language translation device and sign language translation program
JP2014021180A
