Sign language CG generator and program thereof

Abstract

To provide a sign language CG generator which can generate sign language CG which is natural and easily conveyed, and in which correction of the sign language CG is easy.SOLUTION: A sign language CG generator 1 comprises: a machine translation unit 21 which executes machine translation on a Japanese sentence into a sign language word string; a motion database 30 in which sign language word motions are stored in advance; a sign language grammar element generation unit 22; a motion generation unit 42 which combines sign language word motions acquired from the motion database 30 to generate sentence motions; a motion correction unit 43 which receives correction of one or more of the Japanese sentence, the sign language word string or the sign language grammar element in a text format; and a CG rendering unit 50 which refers to the sign language grammar element, and generates the sign language CG from the sentence motions.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a sign language CG generating device and a program therefor. [Background technology]

[0002] For native sign language speakers, it is important to present information in sign language. In the broadcasting field, the number of sign language interpreters is insufficient, so the level of fulfillment is not sufficient. Therefore, a method has been proposed to translate Japanese text into sign language and automatically generate computer graphics (CG) of sign language from the sign language.

[0003] Patent Document 1 proposes a method for generating sign language CG from prepared fixed sign language phrases. However, the method described in Patent Document 1 cannot generate sign language CG from phrases other than fixed phrases, making it difficult to use in the broadcasting field.

[0004] Patent Document 2 also proposes a method for translating any Japanese sentence into sign language using machine translation. In the method described in Patent Document 2, in order to carry over part-of-speech information in a Japanese sentence to a sign language word string as a translation result, a marker is added to a proper noun included in the sign language word string to distinguish it from other parts of speech. For example, the Japanese sentence "The amount of snow that will fall by tonight will be 20 cm on the Pacific coast of southern Tohoku at its heaviest." will be explained as an example. In this case, a PN (property noun) marker for a proper noun is added to the sign language word corresponding to "southern Tohoku," and the sign language word string is expressed as, for example, "PN{east, north, hot}."

[0005] Also, Patent Document 3 proposes a method for correcting the generated sign language CG using an editing tool operated by a GUI. In other words, in the method described in Patent Document 3, when generating sign language CG from an arbitrary sentence, there is a possibility that the sign language word string may contain translation errors, so manual corrections are made on dedicated software. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2016-38748 A [Patent Document 2] JP 2022-16436 A [Patent Document 3] JP 2018-128543 A Summary of the Invention [Problem to be solved by the invention]

[0007] For example, in the above Japanese sentence, "until tonight" and "the amount of snow falling" are considered to be one unit of context. However, in the method described in Patent Document 2, the sign language word string of the translation result is expressed in a form in which only independent sign language words are arranged in chronological order, so much of the context and meaning information related to the context of the original Japanese sentence is lost.

[0008] Here, in the method described in Patent Document 2, for proper nouns, the Japanese sentence "Tohoku Nanbu" is output as a continuous block of sign language words such as "PN{east, north, hot}," and the CG character's mouth moves to say "Tohoku Nanbu." On the other hand, in the method described in Patent Document 2, context information other than proper nouns is lost, so the CG character's mouth does not move. Furthermore, when generating the sign language CG, all sign language words are combined at a fixed speed and interval, making the sign language CG unnatural and difficult to understand.

[0009] The method described in Patent Document 3 requires the user to master the operation of dedicated software to perform manual corrections, and has the problem that even minor corrections require a lot of work.

[0010] Therefore, an object of the present invention is to provide a sign language CG generation device and a program therefor that can generate natural and easy-to-understand sign language CG and also allows easy modification of the sign language CG. [Means for solving the problem]

[0011] In order to solve the above problems, the sign language CG generation device of the present invention is a sign language CG generation device that generates sign language CG from any Japanese sentence, and is configured to include a machine translation unit, a motion database, a sign language grammar element generation unit, a motion generation unit, a motion correction unit, and a sign language CG generation unit.

[0012] According to this configuration, the machine translation unit receives a Japanese sentence, and machine-translates the input Japanese sentence into a sign language word string to generate a sign language word string. The motion database stores in advance a sign language word motion corresponding to each sign language word. The sign language grammar element generating unit generates sign language grammar elements that represent the mouth shapes and nods of a CG character performing sign language actions in sign language CG, and groups of sign language words, and associates them with sign language word strings. The motion generation unit acquires, from a motion database, sign language word motions corresponding to the string of sign language words machine-translated by the machine translation unit, and generates a sentence motion by combining the acquired sign language word motions.

[0013] The motion correction unit accepts, in text format, corrections to one or more of the Japanese sentence input to the machine translation unit, the sign language word string machine-translated by the machine translation unit, or the sign language grammar elements generated by the sign language grammar element generation unit. The sign language CG generation unit refers to sign language grammar elements and generates sign language CG from the sentence motion generated by the motion generation unit.

[0014] In this way, the sign language CG generation device reflects the sign language grammar elements that are important in sign language expression in the sign language CG, so it can generate natural and easy-to-understand sign language CG. Furthermore, the sign language CG generation device can make corrections in text format or JSON format, so users do not need to master the operation of dedicated software, making it easy to correct the sign language CG.

[0015] The present invention can also be realized by a program for causing a computer to function as the sign language CG generating device described above. Effect of the Invention

[0016] According to the present invention, it is possible to generate natural and easily understandable sign language CG, and also to easily modify the sign language CG. [Brief description of the drawings]

[0017] [Figure 1] 1 is a block diagram showing a configuration of a sign language CG generation device according to an embodiment. [Diagram 2] FIG. 1A is a diagram for explaining an example of a conventional sign language word string, and FIG. 1B is a diagram for explaining an example of a sign language grammar element in an embodiment. [Diagram 3] FIG. 4 is an explanatory diagram for explaining a correction screen in the embodiment. [Figure 4] 4 is a flowchart showing the operation of the sign language CG generation device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, each embodiment described below is intended to embody the technical idea of ​​the present invention, and unless otherwise specified, the present invention is not limited to the following. In addition, the same symbols are used for the same means, and descriptions thereof may be omitted.

[0019] (Embodiment) [Configuration of the sign language CG generation device] The configuration of a sign language CG generating device 1 according to an embodiment will be described with reference to FIG. The sign language CG generation device 1 generates sign language CG from any Japanese sentence. As shown in Fig. 1, the sign language CG generation device 1 includes an external data analysis unit 10, a sign language translation unit 20, a motion database 30, a motion synthesis unit 40, and a CG rendering unit (sign language CG generation unit) 50.

[0020] The external data analysis unit 10 receives an XML telegram and extracts an arbitrary Japanese sentence from the input XML telegram. For example, the external data analysis unit 10 receives an XML telegram of disaster prevention information from the Japan Meteorological Agency and extracts a weather overview sentence from the input XML telegram. For example, the weather overview sentence is a Japanese sentence that describes the general weather conditions, such as "There will be snow in northern Kyoto Prefecture on February 28th." The external data analysis unit 10 then outputs the extracted Japanese sentence to the machine translation unit 21.

[0021] The sign language CG generation device 1 may not include the external data analysis unit 10, and may directly receive input of a Japanese sentence for which a sign language CG is to be generated. Furthermore, it goes without saying that the Japanese text for which sign language CG is generated is not limited to weather overview text.

[0022] As shown in FIG. 1, the sign language translation unit 20 includes a machine translation unit 21 and a sign language grammar element generation unit 22.

[0023] The machine translation unit 21 receives a Japanese sentence and machine-translates the input Japanese sentence into a string of sign language words. For example, the machine translation unit 21 receives a weather summary sentence from the external data analysis unit 10.

[0024] Here, the machine translation unit 21 can perform machine translation by any method. In this embodiment, the machine translation unit 21 machine translates a Japanese sentence into a sign language word string by a method using a pre-learning model and post-replacement of proper nouns. This method uses a translation model to translate a Japanese sentence into sign language, and then estimates which part of the sign language a Japanese named entity has been translated into from the obtained translation result and the Japanese sentence, and replaces the named entity later (see Reference 1).

[0025] Reference 1: Taro Miyazaki et al., “Japanese to Sign Language Machine Translation Using Pre-Training Models and Post-Replacement of Proper Nouns,” Association for Natural Language Processing, Proceedings of the 29th Annual Conference, March 2023

[0026] Thereafter, the machine translation unit 21 associates the input Japanese sentence with the machine-translated sign language word string and outputs them to the motion synthesis unit 40.

[0027] There are cases where the Japanese sentence is corrected by the motion correction unit 43 described later. In this case, the machine translation unit 21 performs machine translation of the Japanese sentence corrected by the motion correction unit 43, associates the corrected Japanese sentence with a sign language word string corresponding to the Japanese, and outputs the result to the motion synthesis unit 40 again. In addition, the sign language word string may be corrected by the motion correction unit 43 described later. In this case, the machine translation unit 21 associates the input Japanese sentence with the sign language word string corrected by the motion correction unit 43 and outputs it again to the motion synthesis unit 40.

[0028] The sign language grammar element generating unit 22 generates sign language grammar elements that represent the mouth shape and nod of a CG character making a sign language sign, and groups of each sign language word, and associates them with a string of sign language words. Hereinafter, the generation of sign language grammar elements by the sign language grammar element generating unit 22 will be specifically described with reference to FIG. 2.

[0029] <Generating sign language grammar elements> FIG. 2(a) shows an example of a conventional sign language word string 9. As shown in FIG. 2(a), the sign language word string 9 is a translation of a weather summary sentence into a sign language word string. The sign language word string 9 is written in text format. The sign language word "pt3" represents pointing, and the sign language word "N" represents a CG character nodding.

[0030] FIG. 2(b) shows a table 100 showing an example of a sign language word sequence 110 and a sign language grammar element 120 in the sign language CG generation device 1. 2(b), unlike the conventional sign language word string 9, the sign language word string 110 is in JSON or CSV format, and each sign language word is associated with a sign language grammar element 120. The sign language grammar element 120 includes three elements: a mouth shape 121, a nod 122, and a group 123. In this embodiment, the sign language grammar element 120 is described as tags (B, I, O). Because of this data structure, when adding a new sign language grammar element 120 required for generating a sign language CG, it is sufficient to simply add a new row to the table 100.

[0031] <<Mouth type>> The mouth shape represents that the mouth shape of the sign language speaker (CG character) is adjusted to match the reading of the word when performing hand and finger movements. In addition, the mouth shape is an essential element for correctly conveying sign language when the hand and finger movements are similar but the mouth shape of the sign language speaker changes to express sign language words with different meanings. When expressing the Japanese word "Kyoto" in sign language, the hand and finger movements are similar to the sign language word "west," so by moving the mouth shape to "kyoto," it is possible to convey the meaning of "Kyoto" to people who see the sign language instead of "west." In the example of FIG. 2(b), in the mouth shape 121, the mouth shape "kyoto" is described for the sign language word "Kyoto," and the mouth shape "hokubu" is described for the sign language word "kita." Note that a sign language word with a blank mouth shape 121 is not associated with a mouth shape as a sign language grammar element.

[0032] Here, the sign language grammar element generation unit 22 performs morpheme analysis on the Japanese sentence, and when the result of the morpheme analysis is stored in a mouth-pattern dictionary that stores mouth patterns in advance, adds the mouth pattern as a sign language grammar element. Specifically, the sign language grammar element generation unit 22 has a mouth-pattern dictionary that stores mouth patterns "Kyoto" and "Hokubu". The sign language grammar element generation unit 22 also analyzes the weather overview sentence, and obtains the Japanese words "Kyoto" and "Hokubu". Then, the sign language grammar element generation unit 22 searches the mouth pattern reading dictionary for the Japanese words "Kyoto" and "Hokubu", and adds them to the sign language grammar element.

[0033] <<nod>> A nod represents an action of a CG character lowering its head once and then raising it. In addition, a nod has a role of a punctuation mark or a particle in sign language, and is an essential element for reading sign language correctly. Conventionally, a nod is described independently like other sign language words (e.g., "N"), and a CG character performs a nodding action after the previous sign language action is completed. In the example of FIG. 2(a), after the CG character finishes the sign language action of the third word "28" and lowers both hands, the CG character performs a nodding action at the fourth word "N". In the example of FIG. 2(a), the sign language word sequence 9 includes four "N"s and there are four nods, two of which represent nods that correspond to particles and the remaining two represent nods that correspond to punctuation marks. Ideally, these nods should be controlled so that the angle of the head tilt and the time of nodding are different. However, in reality, these nods are the exact same action, which is one of the reasons why sign language is difficult to understand.

[0034] Therefore, the sign language CG generation device 1 distinguishes between these nods and attaches the nods that correspond to particles to sign language words such as nouns and verbs. In the example of Fig. 2(b), the nods that correspond to particles are the two places represented by "B" in the nod 122. On the other hand, the nods that correspond to punctuation marks are the two places represented by "N" in the sign language word string 110. The "O" in the nod 122 indicates that the sign language word does not include a nod that corresponds to a particle.

[0035] In the sign language CG generation device 1, when the nod corresponding to the particle is completed, the sign language speaker nods only with his / her head while keeping the fingers in the same position, recreating a more natural nod. As described above, in the past, the sign language speaker would lower both hands after completing the sign language movement for the third word, "28," and then perform the nod. In contrast, in the sign language CG generation device 1, the sign language speaker nods without lowering both hands even after completing the sign language movement for the first word, "February 28," reducing the sense of incongruity and making it easier to understand the sign language.

[0036] Here, the sign language grammar element generation unit 22 detects phrases and clauses from the Japanese sentence, and adds a nod as a sign language grammar element at the boundary position of the detected phrase and clause. For example, the sign language grammar element generation unit 22 detects phrases and clauses from the Japanese sentence by general natural language processing (e.g., word alignment), and writes "B" in the sign language word nod 122 that appears immediately before the boundary position of the detected phrase or clause.

[0037] <<Group>> A group represents a single unit of meaning, and the speed of sign language movements and the interval between sign language words are controlled on a group-by-group basis. By shortening the interval between sign language words constituting a group and increasing the speed of movements, it becomes easier to grasp the unity of meaning in the entire sign language sentence when looking at the CG hand image. In the example of FIG. 2(b), in group 123, the string of sign language words starting with "B" and continuing with "I": "Kyoto", "Place", and "North" constitutes one group. In other words, the group is described in a form that applies to the sign language words "Place" and "North". Note that "O" in group 123 indicates that the sign language word does not belong to any group.

[0038] Here, the sign language grammar element generation unit 22 extracts phrase groups from the Japanese sentence, and adds groups corresponding to the extracted groups to the sign language grammar elements. For example, the sign language grammar element generation unit 22 detects phrase groups from the Japanese sentence by general natural language processing (e.g., word alignment), writes "B" in the group of the first sign language word of the detected phrase group, and writes "I" in the group 123 of the following sign language word.

[0039] 2(b), the sign language grammar element generation unit 22 can associate sign language grammar elements 120 consisting of a mouth shape 121, a nod 122, and a group 123 with the sign language word string 110. Then, the sign language CG generation device 1 can generate easy-to-understand sign language CG by referring to the sign language grammar elements associated with each sign language word.

[0040] Returning to FIG. 1, the description of the configuration of the sign language CG generation device 1 will continue. The motion database 30 stores in advance sign language word motions corresponding to each sign language word. For example, the administrator of the sign language CG generation device 1 registers the motion database 30 in advance. A sign language word motion is a sign language action that represents each sign language word. For example, the motion database 30 stores general sign language word motions for sign language words, as well as sign language word motions corresponding to mouth shapes, which will be described later.

[0041] As shown in FIG. 1, the motion synthesis unit 40 includes a motion combination unit 41, a motion generation unit 42, and a motion correction unit 43.

[0042] The motion combining unit 41 acquires, from the motion database 30, sign language word motions corresponding to the sign language word strings machine-translated by the machine translation unit 21, and combines the acquired sign language word motions. The motion generating unit 42 generates an intermediate file (sentence motion) from the sign language word motions combined by the motion combining unit 41. The motion combination unit 41 and the motion generation unit 42 correspond to the "motion generation unit" recited in the claims.

[0043] <Sentence Motion Generation> <<Mouth type>> The generation of sentence motions by the motion combining unit 41 and the motion generating unit 42 will be specifically described below. The motion combining unit 41 and the motion generating unit 42 obtain sign language word motions specified by the mouth shapes of sign language grammar elements from the motion database 30, and generate sentence motions by combining the sign language word motions specified by the mouth shapes around the mouth of the CG character.

[0044] That is, the motion combining unit 41 and the motion generating unit 42 combine the sign language word motion corresponding to the mouth shape only with the skeleton around the mouth of the CG character, and combine the sign language word motion corresponding to the sign language word with the skeleton other than the mouth, thereby generating a sentence motion. The mouth shape "KYOTO" for the sign language word "Kyoto" in Fig. 2(b) will be explained as an example. The motion generating unit 42 combines the sign language word motions of the mouth shapes "KYOTO", "U", and "TO" with the skeleton around the mouth of the CG character, and combines the sign language word motion of the sign language word "Kyoto" with the skeleton other than the mouth (for example, the skeleton of the fingers and back), thereby generating a sentence motion.

[0045] <<nod>> The motion combining unit 41 and the motion generating unit 42 generate a sentence motion in which the head of the CG character nods while maintaining the position of the fingers of the character for a sign language word to which a nod, which is a sign language grammar element, has been added.

[0046] That is, the motion combining unit 41 and the motion generating unit 42 combine the sign language word motion of the sign language word to which a nod has been added with a sign language word motion of the CG character "nodding" only from the neck up without lowering both hands of the fingers. The sign language word "location" in Fig. 2(b) will be explained as an example. The motion generating unit 42 freezes the CG character's hands without lowering them for the sign language word motion of the sign language word "location," combines it with the sign language word motion of the CG character "nodding" only from the neck up, and generates a sentence motion.

[0047] <<Group>> The motion combining unit 41 and the motion generating unit 42 generate sentence motions in which the sign language actions are made faster by a preset value and the connection intervals between sign language words are shortened by a preset value for sign language words specified in a group of sign language grammar elements.

[0048] That is, the motion combining unit 41 and the motion generating unit 42 set the speed and blend span of the sign language action for the sign language words specified in the group of sign language grammar elements to values ​​different from those of the sign language word strings other than the group. This blend span indicates the length of the frames connecting between the sign language word motions, and the longer the blend span, the slower the transition to the next sign language word motion. The group "Kyoto, place, north" in FIG. 2(b) will be explained as an example. The motion generating unit 42 sets the movement speed of the sign language words "Kyoto", "place", and "north" 20% faster than the other sign language word strings. Also, the motion generating unit 42 sets the blend span of the sign language words "Kyoto", "place", and "north" 20% shorter than the other sign language word strings.

[0049] It goes without saying that the value by which the sign language action is made faster and the value by which the blend span is shortened for the sign language words specified in the group are not limited to 20%. For example, the sign language action may be made faster by 10% to 50% and the blend span may be shortened by 10% to 60% for the sign language words specified in the group.

[0050] The motion correction unit 43 accepts, in text format, corrections to one or more of the Japanese sentence input to the machine translation unit 21, the sign language word string machine-translated by the machine translation unit 21, or the sign language grammar elements generated by the sign language grammar element generation unit 22.

[0051] <Motion correction by the motion correction department> The modification by the motion modifying unit 43 will now be described in detail with reference to FIG. In this embodiment, the motion correction unit 43 accepts corrections from the user of the sign language CG generation device 1 using a correction screen 200 in Fig. 3. As shown in Fig. 3, the correction screen 200 has a sign language CG display area 210 and a display / correction area 220 for Japanese / sign language word strings.

[0052] The sign language CG display area 210 is an area for displaying sign language CG generated by the CG rendering unit 50 described later. In other words, the sign language CG display area 210 displays sign language CG reflecting the correction made in the Japanese / sign language word string display / correction area 220. Also, a CG character 211 performing sign language actions in the sign language CG is arranged in the sign language CG display area 210.

[0053] The Japanese / sign language word string display / edit area 220 is an area where the user edits Japanese sentences and sign language word strings, and has a topic field 230 , a Japanese sentence field 240 , a sign language word string field 250 , and a date / time field 260 .

[0054] The item field 230 is a field that displays an item corresponding to the Japanese sentence displayed in the Japanese sentence field 240. For example, the item is a sequential number assigned to each Japanese sentence, or a label such as "weather forecast."

[0055] The Japanese sentence field 240 is a field that displays, in text format, the Japanese sentence input to the machine translation unit 21. In addition, in the Japanese sentence field 240, the user can input, in text format, the Japanese sentence to be corrected, using operation means such as a keyboard and a mouse (not shown).

[0056] The sign language word string field 250 is a field that displays, in text format, a sign language word string machine-translated by the machine translation unit 21. That is, the sign language word string field 250 displays a sign language word string that corresponds to the Japanese sentence in the Japanese sentence field 240. In addition, in the sign language word string field 250, the user of the sign language CG generation device 1 can input, in text format, a sign language word string that he or she wishes to correct, using an operation means.

[0057] In this way, on the correction screen 200, the user can correct Japanese sentences or sign language word strings in text format, so that the user does not need to become familiar with the operation of dedicated software, and correction is easy.

[0058] The date and time field 260 is a field that displays the date and time when the XML message was generated.

[0059] When the Japanese sentence in the Japanese sentence field 240 or the sign language word string in the sign language word string field 250 is corrected, the motion correction unit 43 outputs the correction result to the machine translation unit 21. Furthermore, the motion correcting unit 43 allows the user to correct the sign language grammar element by editing the sign language word string displayed in the sign language word string field 250. In this case, the sign language grammar element corrected by the motion correcting unit 43 is associated with the sign language word string.

[0060] Thereafter, the motion synthesis unit 40 outputs the sign language word strings associated with the sign language grammar elements and the sentence motions to the CG rendering unit 50.

[0061] Returning to FIG. 1, the description of the configuration of the sign language CG generation device 1 will continue. The CG rendering unit 50 refers to sign language grammar elements and generates sign language CG from the sentence motion generated by the motion generation unit 42. That is, the CG rendering unit 50 refers to the sign language grammar elements input from the motion synthesis unit 40 and renders the sign language CG from the sentence motion generated by the motion generation unit 42. The CG rendering unit 50 then outputs the generated sign language CG to an external device (e.g., a storage device). Furthermore, the CG rendering unit 50 outputs the generated sign language CG to the motion modification unit 43 in order to display the sign language CG on the modification screen 200 of FIG. 3.

[0062] [Operation of the sign language CG generator] The operation of the sign language CG generating device 1 will be described with reference to FIG. As shown in FIG. 4, in step S1, the external data analysis unit 10 receives an XML message and extracts any Japanese sentence from the received XML message.

[0063] In step S2, the machine translation unit 21 machine-translates the Japanese sentence extracted in step S1 into a string of sign language words. For example, the machine translation unit 21 machine-translates the Japanese sentence into a string of sign language words by a method using a pre-learning model and post-substitution of proper nouns.

[0064] In step S3, the sign language grammar element generating unit 22 generates sign language grammar elements representing the mouth shape and nodding of the CG character and groups of each sign language word. Specifically, the sign language grammar element generating unit 22 performs a morphological analysis on the Japanese sentence, and when the result of the morphological analysis is stored in a mouth shape dictionary in which mouth shapes are stored in advance, adds the mouth shape as a sign language grammar element, detects phrases and clauses from the Japanese sentence, adds nodding as a sign language grammar element at the boundary positions of the detected phrases and clauses, extracts groups of phrases from the Japanese sentence, and adds groups corresponding to the extracted groups as sign language grammar elements.

[0065] In step S4, the motion combination unit 41 and the motion generation unit 42 acquire sign language word motions corresponding to the sign language word string machine-translated in step S2 from the motion database 30, and combine the acquired sign language word motions to generate a sentence motion. Specifically, the motion combination unit 41 and the motion generation unit 42 acquire sign language word motions designated by the mouth shapes of the sign language grammar elements from the motion database. Then, the motion combination unit 41 and the motion generation unit 42 combine the sign language word motions designated by the mouth shapes for the CG character's mouth area to generate a sentence motion. Furthermore, the motion combination unit 41 and the motion generation unit 42 generate a sentence motion in which the head of the CG character nods while keeping the position of the fingers of the CG character constant, for a sign language word to which a nod of the sign language grammar element is added. Furthermore, the motion combination unit 41 and the motion generation unit 42 generate a sentence motion in which the sign language action of the CG character is made faster by a preset value and the connection interval between sign language words is shortened by a preset value, for a sign language word designated by a group of sign language grammar elements.

[0066] In step S5, the motion correction unit 43 accepts, in text format, corrections to one or more of the Japanese sentence input to the machine translation unit 21, the sign language word string machine-translated by the machine translation unit 21, or the sign language grammar elements generated by the sign language grammar element generation unit 22.

[0067] If the Japanese sentence is corrected in step S5, the sign language CG generation device 1 re-executes the processes from step S2 onwards using the corrected Japanese sentence. Furthermore, if the sign language word string or the sign language grammar element is corrected in step S5, the sign language CG generation device 1 re-executes the processes from step S3 onwards using the corrected sign language word string or the sign language grammar element.

[0068] In step S6, the CG rendering unit 50 refers to the sign language grammar elements and renders the sign language CG from the sentence motion generated by the motion generating unit .

[0069] [Actions and Effects] As described above, the sign language CG generation device 1 according to the embodiment reflects important sign language grammar elements in sign language expression in the sign language CG, so that it can generate natural and easy-to-understand sign language CG. Furthermore, the sign language CG generation device 1 can make corrections in text format or JSON format, so that the user does not need to become familiar with the operation of dedicated software, and can easily correct the sign language CG.

[0070] Although the embodiment has been described in detail above, the present invention is not limited to the above-described embodiment, and includes design modifications and the like within the scope of the present invention.

[0071] In the above embodiment, the sign language CG generation device is described as an independent piece of hardware, but the present invention is not limited to this. For example, the present invention can be realized by a program for causing hardware resources such as a CPU, memory, and hard disk of a computer to function as the sign language CG generation device. This program may be distributed via a communication line, or may be written on a recording medium such as a CD-ROM or flash memory and distributed. [Explanation of symbols]

[0072] 1. Sign language CG generation device 10 External Data Analysis Department 20 Sign Language Translation Department 21 Machine Translation Department 22 Sign language grammar element generation unit 30 Motion Database 40 Motion synthesis section 41 Motion Joint 42 Motion Generation Section 43 Motion Correction Department 50 CG rendering unit (sign language CG generation unit)

Claims

1. A sign language CG generation device that generates sign language CG from any Japanese sentence, comprising: a machine translation unit to which the Japanese sentence is input and which machine translates the input Japanese sentence into a sign language word string to generate a sign language word string; A motion database in which sign language word motions corresponding to each sign language word are stored in advance; a sign language grammar element generating unit for generating sign language grammar elements representing mouth shapes and nodding of a CG character performing sign language actions in the sign language CG and groups of each sign language word, and for associating the sign language grammar elements with the sign language word string; a motion generation unit that acquires, from the motion database, sign language word motions corresponding to the sign language word strings machine-translated by the machine translation unit, and generates a sentence motion by combining the acquired sign language word motions; a motion correction unit that receives, in a text format, corrections to one or more of the Japanese sentence input to the machine translation unit, the sign language word string machine-translated by the machine translation unit, or the sign language grammar element generated by the sign language grammar element generation unit; a sign language CG generation unit that generates the sign language CG from the sentence motion generated by the motion generation unit by referring to the sign language grammar elements; A sign language CG generation device comprising:

2. The sign language grammar element generation unit morphologically analyzing the Japanese sentence, and if the result of the morphological analysis is stored in a mouth pattern dictionary in which the mouth patterns are previously stored, adding the mouth patterns as the sign language grammar elements; detecting phrases and clauses from the Japanese sentence, and adding the nod as the sign language grammar element at the boundary positions of the detected phrases and clauses; A group of phrases is extracted from the Japanese sentence, and a group corresponding to the extracted group is added as the sign language grammar element.

2. The sign language CG generating device according to claim 1.

3. The motion generation unit is A sign language word motion designated by a mouth shape of the sign language grammar element is obtained from the motion database, and a sentence motion is generated by combining the sign language word motion designated by the mouth shape with respect to the mouth area of ​​the CG character; generating a sentence motion in which the head of the CG character nods while maintaining the position of the fingers of the CG character in the sign language word to which the sign language grammar element nods; A sentence motion is generated in which the sign language action of the CG character is made faster by a preset value for the sign language words designated by the group of sign language grammar elements, and the connection interval between the sign language words is shortened by a preset value.

2. The sign language CG generating device according to claim 1.

4. The machine translation unit machine-translates the Japanese sentence into the sign language word string using a method that uses a pre-learning model and post-substitution of proper nouns.

2. The sign language CG generating device according to claim 1.

5. A program for causing a computer to function as the sign language CG generation device according to any one of claims 1 to 4.

Images