Translation device and program

Abstract

To provide a translation apparatus and program for outputting sign language while maintaining the meaning of a proper noun included in an original natural language text.SOLUTION: A translation apparatus includes a translation unit, a sign language motion database, and a proper noun motion synthesis unit. The translation unit converts a natural language text to sign language label strings. In the sign language label string, a part equivalent to a proper noun is specified. The sign language motion database holds associations between sign language labels and sign language motions. The proper noun motion synthesis unit reads, based on a sign language label string with a proper noun marker, out of the sign language label strings output by the translation unit, a sign language motion corresponding to each of the sign language label from the sign language motion database, and sequentially connects the sign language motions with a glide length shorter than usual to generate a synthetic sign language motion corresponding to the proper noun.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a translation device and a program. [Background technology]

[0002] The technology of translating Japanese text into sign language (Japanese Sign Language) using computer graphics (CG) animation has the potential to be widely used in various fields.

[0003] In conventional sign language translation methods for CG animation, an example corpus is constructed by combining textualized Japanese (input information) with a corresponding string of sign language labels (output information). Then, using a statistical model or neural network generated from the example corpus, translation is realized from Japanese text to a string of sign language labels that lists multiple sign language labels in chronological order. Furthermore, based on the translated string of sign language labels, motion data corresponding to each sign language label is read, and the motion is played on a CG avatar to generate sign language animation.

[0004] Motion data basically corresponds to the unit of sign language words (labels). By connecting the motion data in a time series corresponding to the sign language label string, it is possible to generate motion corresponding to a sign language sentence.

[0005] When a proper noun is included in the Japanese text that is input to the translation process, the string of sign language labels that constitute the proper noun is included in the output sign language label string.

[0006] The following methods can be considered as a method for generating CG animation of sign language based on Japanese text. First, the Japanese text is converted into a string of sign language labels using one of the following two methods. In the first method, multiple pieces of motion data are prepared for each sentence of a fixed phrase that has been translated from the Japanese text into sign language. Also, parts of the fixed phrase, such as proper nouns, are replaced depending on the content of the Japanese text. In the second method, any Japanese sentence text is translated into a string of sign language labels. By using one of these methods, Japanese text can be converted into a string of sign language labels. Next, the sign language word motion data corresponding to each word in the string of sign language labels is read, and the motion synthesized on a sentence-by-sentence basis is synthesized using a CG avatar.

[0007] When generating a sign language label string using the above first method, a fixed phrase is used, so that it is possible to output highly accurate sign language without the need for translation processing. However, there is a problem that the sign language sentences that can be generated are limited to patterns prepared in advance. When generating a sign language label string using translation, which is the second method, there is no limit to the types of sentence patterns that can be generated, but there may be problems with the accuracy of translating Japanese text into sign language labels.

[0008] Patent Documents 1 and 2 describe techniques for performing machine translation to generate a sign language label string for generating sign language CG animation based on Japanese text. The techniques described in these documents use a corpus, which is bilingual data of Japanese text and sign language label strings, to perform learning for machine translation processing. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] JP 2013-186673 A [Patent Document 2] JP 2014-021180 A Summary of the Invention [Problem to be solved by the invention]

[0010] In conventional technology, the part-of-speech information of the Japanese text is not inherited in the sign language label string that is the translation result. Therefore, even if the word order of the output sign language labels is correct, it is impossible to reproduce motion synthesis that takes into account the relationships between each sign language label, such as spatial transformation or grouping by part of speech.

[0011] In other words, even if the translation accuracy of the conventional technology to the sign language labels is high, the sign language expressions corresponding to all the sign language labels end up as independent CG animations. This causes a problem that the meaning of the expressions contained in the original Japanese text is changed in the generated CG animations.

[0012] The sign language labels that make up proper nouns included in the translation process are treated indistinguishably from the sign language labels that correspond to other general words, which means that when a CG animation is generated, it no longer expresses the original meaning of the proper noun. In other words, the meaning of the proper noun changes when expressed in sign language.

[0013] The present invention has been made to solve the above problems. That is, the present invention aims to provide a translation device and a program that can output sign language while maintaining the meaning of proper nouns contained in the original natural language text when translating the natural language text into a sign language label string to generate a sign language animation. [Means for solving the problem]

[0014] [1] In order to solve the above problem, a translation device according to one aspect of the present invention includes a morpheme analysis unit that performs morpheme analysis processing on input text to output a morpheme string, and if a morpheme included in the morpheme string is a proper noun, adds data indicating that the morpheme is a proper noun; a translation unit that converts the morpheme string output from the morpheme analysis unit into a sign language label string based on a machine learning model, and adds a proper noun marker to a portion of the sign language label string after the conversion that corresponds to the morpheme to which the data indicating that it is a proper noun has been added; a sign language motion database that holds a correspondence between sign language labels and sign language motions; and a translation unit that reads out sign language motions corresponding to each of the sign language labels from the sign language motion database based on the sign language label string to which the proper noun markers have been added from the sign language label string output by the translation unit, and translates the sign language motions into the sign language motions. The system comprises a proper noun motion synthesis unit which synthesizes a sign language motion corresponding to a proper noun by sequentially connecting sign language motions at a first transition length, and a sign language motion synthesis unit which reads out sign language motions corresponding to each of the sign language labels from the sign language motion database based on a sign language label string not having the proper noun marker among the sign language label string output by the translation unit, and sequentially connects the sign language motions at a second transition length to synthesize sign language motions corresponding to entities other than proper nouns, and synthesizes sign language motions corresponding to the text by further connecting the sign language motions corresponding to the proper nouns synthesized by the proper noun motion synthesis unit based on the sign language label string output by the translation unit to sign language motions corresponding to entities other than the proper nouns, wherein the first transition length is shorter than the second transition length.

[0015] [2] Another aspect of the present invention is that, in the above-mentioned translation device, the playback speed of each sign language label included in the sign language motion corresponding to the proper noun is made faster than the playback speed of each sign language label included in the sign language motion corresponding to something other than the proper noun.

[0016] [3] In one aspect of the present invention, the above-mentioned translation device further includes a proper noun reading estimation unit that estimates the reading of a proper noun for a morpheme to which the morpheme analysis unit has assigned data indicating that the morpheme is a proper noun, and a mouth motion database that maintains a correspondence between the sounds of the reading and mouth motions that are part of a sign language motion, and the proper noun motion synthesis unit reads out mouth motions corresponding to each sound contained in the estimated reading of the proper noun from the mouth motion database, and synthesizes a mouth motion corresponding to the proper noun by sequentially connecting these mouth motions based on the reading, and replaces data regarding the mouth shape in the sign language motion corresponding to the proper noun with the mouth motion corresponding to the synthesized proper noun.

[0017] [4] In another aspect of the present invention, in the above-mentioned translation device, the translation device further includes an animation generation unit that generates an animation image based on the sign language motion data corresponding to the text synthesized by the sign language motion synthesis unit.

[0018] [5] Also, one aspect of the present invention includes a morpheme analysis unit that performs morpheme analysis processing on input text to output a morpheme string, and if a morpheme included in the morpheme string is a proper noun, adds data indicating that the morpheme is a proper noun; a translation unit that converts the morpheme string output from the morpheme analysis unit into a sign language label string based on a machine learning model, and adds a proper noun marker to a portion of the sign language label string after the conversion that corresponds to the morpheme to which the data indicating that it is a proper noun has been added; a sign language motion database that holds a correspondence between sign language labels and sign language motions; and a translation unit that reads out sign language motions corresponding to each of the sign language labels from the sign language motion database based on the sign language label string to which the proper noun markers have been added from the sign language label string output by the translation unit, and sequentially connects the sign language motions with a first span. and a sign language motion synthesis unit that reads out sign language motions corresponding to each of the sign language labels from the sign language motion database based on a sign language label string not having the proper noun marker among the sign language label string output by the translation unit, sequentially connects the sign language motions at a second transition length to synthesize sign language motions corresponding to words other than proper nouns, and further connects the sign language motions corresponding to the proper nouns synthesized by the proper noun motion synthesis unit based on the sign language label string output by the translation unit to sign language motions corresponding to words other than proper nouns, wherein the first transition length is shorter than the second transition length. Effect of the Invention

[0019] According to the present invention, the translation device makes the length of the gap between sign language labels corresponding to proper nouns shorter than the length of the gap between sign language labels corresponding to non-proper nouns, which means that the translation device can synthesize sign language motion data that is easy to recognize as a unit of proper nouns. [Brief description of the drawings]

[0020] [Figure 1] 1 is a block diagram showing a schematic functional configuration of a translation device that translates text into sign language according to an embodiment of the present invention. [Diagram 2] 2 is a schematic diagram showing an example of Japanese text externally acquired by a text acquisition unit according to the embodiment. FIG. [Diagram 3] 4 is a schematic diagram showing an example of translation result data (data including an output label string, the pronunciation of proper nouns, etc.) output by a translation unit according to the embodiment. FIG. [Figure 4] 13 is a schematic diagram showing an example of a format of data held in a mouth shape motion database according to the embodiment. FIG. [Diagram 5] 2 is a schematic diagram showing an example of a format of data stored in a sign language motion database according to the embodiment. FIG. [Figure 6] 13 is a schematic diagram showing an example of a CG animation based on motion data synthesized by a sign language motion synthesis unit according to the embodiment. FIG. [Figure 7] 10 is a flowchart showing a procedure of a translation process from text to sign language by the translation device of the embodiment. [Figure 8] 4 is a schematic diagram showing at least some of the setting values ​​referred to by the translation device according to the embodiment. FIG. [Figure 9] 2 is a block diagram showing an example of an internal configuration of the translation device according to the embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Next, an embodiment of the present invention will be described with reference to the drawings. The translation device of this embodiment translates natural language text into a string of sign language labels. The translation device also generates a sign language animation video based on the string of sign language labels obtained by the translation process. In this embodiment, the natural language text to be translated is Japanese text. The sign language output as the translation result is Japanese Sign Language. However, the translation source may be text written in a language other than Japanese. Also, a sign language other than Japanese Sign Language may be output.

[0022] The translation device of this embodiment translates Japanese text into a sign language label string while maintaining part-of-speech information (specifically, information on proper nouns). In other words, the sign language label string output as a result of the translation process has information indicating which labels correspond to proper nouns. Specifically, in the sign language label string output by the translation device as a translation result, sign language label strings that correspond to one proper noun are grouped together, and a marker indicating that the group is a proper noun is attached to the group.

[0023] When the translation device synthesizes motion from the sign language label sequence based on the information on the proper noun, it applies processing specific to the label sequence corresponding to the proper noun. Specifically, for a group of label sequences to which a marker indicating that it is a proper noun has been added, the translation device shortens the span between the sign language label sequences and adjusts the speed of the motion. In other words, when the translation device synthesizes motion, it is possible to perform processing that takes into account the group as a proper noun. In addition, the translation device generates animation with mouth shapes that correspond to the reading of the proper noun.

[0024] A transition is a transition (connection) part of a motion from the end point of a motion corresponding to one sign language label to the start point of a motion corresponding to another sign language label. The span length is the length of time corresponding to the span portion. The span length can be expressed, for example, in seconds or by a number such as the number of frames. Furthermore, the mouth shape is a part of the sign language motion that corresponds to the mouth movement. The sign language motion can be expressed, for example, as data on the movement of the joint parts of a modeled human skeleton. The mouth shape motion of the sign language motion can be expressed as data on the movement of the joint parts around the mouth. In other words, a part of the sign language motion is the mouth shape motion.

[0025] Translation devices using existing technology generate sign language CG animations by referring only to the word order of the sign language label string, which is the result of translation based on natural language text. Therefore, translation devices using existing technology may generate incorrect sign language CG animations for proper nouns. In contrast, this embodiment can generate CG animations with intonation that reproduce proper nouns as a single unit. In other words, the translation device of this embodiment can output sign language CG animations that are closer to the meaning of the input Japanese text.

[0026] FIG. 1 is a functional block diagram showing a schematic functional configuration of a translation device according to this embodiment. As shown in the figure, the translation device 1 includes a text acquisition unit 11, a morpheme analysis unit 12, a proper noun reading estimation unit 13, a translation unit 14, a mouth shape motion database 21, a sign language motion database 22, a proper noun reading estimation unit 23, a sign language motion synthesis unit 24, and a CG animation generation unit 31. The text acquisition unit 11, the morpheme analysis unit 12, the proper noun reading estimation unit 13, and the translation unit 14 have a function for translating Japanese text into a sign language label string. The mouth shape motion database 21, the sign language motion database 22, the proper noun motion synthesis unit 23, and the sign language motion synthesis unit 24 have a function for generating data of a sign language motion based on the sign language label string. The CG animation generation unit 31 generates an animation based on the data of the sign language motion.

[0027] Each of these functional units can be realized, for example, by a computer and a program. Each functional unit has a storage means as necessary. The storage means is, for example, a variable in the program, or a memory allocated by the execution of the program. A non-volatile storage means such as a magnetic hard disk drive or a solid state drive (SSD) may also be used as necessary. At least a part of the functions of each functional unit may also be realized as a dedicated electronic circuit rather than a program. The specific functions of each unit are as follows:

[0028] The text acquisition unit 11 acquires Japanese text to be translated from an external source. The text acquisition unit 11 acquires Japanese text manually inputted from, for example, a keyboard or a touch panel. Alternatively, the text acquisition unit 11 may acquire Japanese text by recognizing spoken voice using a voice recognition technology. Alternatively, the text acquisition unit 11 may acquire Japanese text recorded on an external recording medium or the like. The text acquisition unit 11 may also acquire Japanese text by other methods. The text acquisition unit 11 passes the acquired Japanese text to the morphological analysis unit 12.

[0029] The morpheme analysis unit 12 performs a morpheme analysis process on the Japanese text passed from the text acquisition unit 11. The morpheme analysis process itself can be performed using existing technology. One example of a technology for realizing the morpheme analysis unit 12 is KyTea (http: / / www.phontron.com / kytea / index-ja.html). However, the technology that can be used here is not limited to KyTea. The morpheme analysis unit 12 divides the Japanese text into a string of morphemes and identifies the part of speech of each morpheme. The morpheme analysis unit 12 assigns part of speech information to each morpheme. Proper nouns are included as one of the parts of speech. In other words, a proper noun included in the original Japanese text is classified as a single morpheme, and information indicating that it is a proper noun is assigned to it.

[0030] The morphological analysis unit 12 passes the result of the morphological analysis process to the translation unit 14. Furthermore, if a proper noun is included in the original Japanese text, the morphological analysis unit 12 also passes the result of the morphological analysis process to the proper noun reading estimation unit 13.

[0031] In other words, the morphological analysis unit 12 performs morphological analysis of the input text to output a morpheme string, and if a morpheme included in the morpheme string is a proper noun, adds data indicating that the morpheme is a proper noun.

[0032] When proper nouns are included in the Japanese text, the proper noun reading estimation unit 13 extracts the proper nouns based on the result of the above-mentioned morphological analysis process, and estimates the reading of each proper noun in Japanese. The process of estimating the reading of the proper noun can also be realized using existing technology. For example, it is possible to estimate the reading of a proper noun written in kanji or the like by using an existing text analysis technology such as the above-mentioned KyTea. However, the reading of a proper noun may be estimated using a method other than KyTea. The proper noun reading estimation unit 13 outputs the estimated reading of the proper noun as data written in katakana, for example. The proper noun reading estimation unit 13 passes the estimated reading data to the translation unit 14.

[0033] That is, the proper noun reading estimation unit 13 estimates the reading of a proper noun for a morpheme to which the morpheme analysis unit 12 has assigned data indicating that the morpheme is a proper noun.

[0034] When the text divided into morphemes contains a proper noun at the stage of being passed to the translation unit 14, the morpheme string is accompanied by a symbol indicating that it is a proper noun (e.g., "$NE$"). In addition, for the proper noun in the text, data on the reading estimated by the proper noun reading estimation unit 13 (e.g., the reading "Tohoku Nanbu" for the proper noun "Tohoku Nanbu") is added.

[0035] The translation unit 14 translates the morpheme sequence corresponding to the acquired Japanese text into a sign language label sequence. The translation unit 14 is realized using a translator technology using a machine learning technique. The translation technology using machine learning itself belongs to existing technology. The text-to-sign language translation technique is also described in the above-mentioned Patent Document 1 and Patent Document 2. The translation unit 14 is realized using a neural network, as an example. In the translation unit 14, a translation model such as a neural network is machine-learned in advance. During machine learning, input data (morpheme sequence based on the Japanese text) and output data (correct sign language label sequence) are used as learning data. The translation unit 14 passes the output sign language label sequence to each of the proper noun motion synthesis unit 23 and the sign language motion synthesis unit 24.

[0036] In this embodiment, the translation unit 14 also includes information indicating a unit of a group as a proper noun in the data of the translation result. In other words, the translation unit 14 adds a marker indicating that the sign language label string corresponding to a proper noun in the original text to the sign language label string to be output. The translation unit 14 also includes data on the reading of the proper noun received from the proper noun reading estimation unit 13 in the data of the translation result. An example of the sign language label string output by the translation unit 14 will be described later with reference to FIG. 3.

[0037] Each sign language label included in the translation result output by the translation unit 14 corresponds to a data number (a number specifying a motion) of the motion capture data described below. In other words, the translation result output by the translation unit 14 corresponds to a sequence of motion data arranged in chronological order.

[0038] In other words, based on a machine-learned model (translation model), the translation unit 14 converts the morpheme string output from the morpheme analysis unit 12 into a sign language label string, and adds a proper noun marker to the part of the sign language label string after the conversion that corresponds to the morpheme to which data indicating that it is a proper noun has been added.

[0039] The mouth shape motion database 21 and the sign language motion database 22 are databases that store data representing sign language motions. The sign language motion data used in generating CG animation is, for example, motion captured sign language movements made by an actual person, including the hands, mouth shapes, facial expressions, etc., and stored in a format such as BVH (Biovision Hierarchy). To be used in generating CG animation, the sign language motion data is recorded on a word-by-word basis (on a sign language label string basis) and stored in a database (mouth shape motion database 21 or sign language motion database 22). The translation device 1 arranges the motion data on a word-by-word basis (on a sign language label string basis) in chronological order and generates sign language motion data as a sentence by connecting the words.

[0040] The mouth motion database 21 stores data representing mouth motions among the above-mentioned motion data. Mouth motions correspond to each sound when speaking. For example, the mouth motion database 21 holds data on mouth motions corresponding to each of the vowels "a", "i", "u", "e" and "o" and the nasal sound "n". Note that the mouth motion database 21 may also hold data on other mouth motions. In other words, the mouth motion database 21 holds the correspondence between the reading sounds and mouth motions that are part of the sign language motions.

[0041] The sign language motion database 22 stores data representing the entirety of the above motion data (including motions other than those of the mouth). In other words, the sign language motion database 22 holds data corresponding to motions such as hand and finger movements and facial expressions. The sign language motion data stored in the sign language motion database 22 may or may not include mouth motion. The sign language motion database 22 stores sign language motion data in a form corresponding to a sign language label (or a number corresponding to a sign language label, etc.). The sign language motion database 22 holds motion data corresponding to the sign language label in a form corresponding to the sign language label. In other words, the sign language motion database 22 holds the correspondence between the sign language label and the sign language motion.

[0042] The proper noun motion synthesis unit 23 synthesizes, among the sign language motions, motions of parts that correspond to proper nouns in the original text. In other words, the proper noun motion synthesis unit 23 synthesizes a sign language motion for expressing a unity as a proper noun for the part of the sign language label string that corresponds to the proper noun. Specifically, the proper noun motion synthesis unit 23 synthesizes a sign language motion specific to the proper noun by referring to the mouth shape motion database 21 and the sign language motion database 22. The proper noun motion synthesis unit 23 passes the motion data that is the result of the synthesis to the sign language motion synthesis unit 24. The processing contents of the proper noun motion synthesis unit 23 are described in more detail below.

[0043] The proper noun motion synthesis unit 23 extracts, from the sign language label string output by the translation unit 14, a portion that corresponds to a proper noun (a portion to which a proper noun marker is added).

[0044] The proper noun motion synthesis unit 23 acquires data on motions other than mouth shapes (motions of fingers and facial expressions (other than mouth shapes)) for each sign language label included in the part corresponding to a proper noun from the sign language motion database 22. Then, the proper noun motion synthesis unit 23 connects the sign language motions corresponding to each of these sign language labels in chronological order. For the "crossing" part connecting one sign language label to another, the proper noun motion synthesis unit 23 linearly interpolates, for example, between the end of the previous motion and the start of the next motion. The process of connecting the motions of the crossing part itself can be performed using existing technology.

[0045] Next, the proper noun motion synthesis unit 23 adjusts the time of the sign language motion of the proper noun part. Specifically, the proper noun motion synthesis unit 23 adjusts the span length (number of connection frames, connection time length) of the span part between the sign language labels in one proper noun so that it is shorter than the span length between the sign language labels other than the proper noun. Furthermore, the proper noun motion synthesis unit 23 may adjust the speed of the motion corresponding to each sign language label included in the proper noun part so that it is faster than the speed of the motion corresponding to the sign language label other than the proper noun part. By these adjustments, the proper noun motion synthesis unit 23 can express a unity of the sign language motion of the part corresponding to one proper noun. In other words, the proper noun motion synthesis unit 23 makes the playback speed of each sign language label included in the sign language motion corresponding to the proper noun faster than the playback speed of each sign language label included in the sign language motion corresponding to the non-proper noun.

[0046] Next, the proper noun motion synthesis unit 23 synthesizes a mouth motion. Specifically, the proper noun motion synthesis unit 23 obtains the time length (utterance length) of the proper noun motion for which the above-mentioned time adjustment has been completed from the total number of frames of the proper noun. The proper noun motion synthesis unit 23 also generates a time-series mouth motion based on the reading data (reading estimated by the proper noun reading estimation unit 13) associated with the proper noun. That is, the proper noun motion synthesis unit 23 acquires mouth motion data for each sound of the reading data from the mouth motion database 21. Then, the proper noun motion synthesis unit 23 arranges the mouth motion data in the order of the sounds of the reading of the proper noun, and performs a process of connecting the mouth shapes between sounds. Here too, the connection of the motions can be realized by using existing technology, for example, a mouth shape interpolation process is used. Furthermore, the proper noun motion synthesis unit 23 adjusts the speed of the mouth motion so that the entire series of mouth motions corresponding to the reading of the proper noun matches the time length (utterance length, number of frames) calculated above. The mouth motion generated here is data representing the movement of the joints around the mouth.

[0047] Then, the proper noun motion synthesis unit 23 synthesizes the adjusted mouth motion with the sign language motion of the proper noun. Specifically, the proper noun motion synthesis unit 23 replaces the joint information around the mouth of the already synthesized sign language motion of the proper noun part with the mouth motion after the speed adjustment. As a result, motion data is generated that integrates all of the fingers, facial expressions, mouth shapes, etc. for the proper noun.

[0048] That is, the proper noun motion synthesis unit 23 reads out sign language motions corresponding to each of the sign language labels from the sign language motion database 22 based on the sign language label string to which the proper noun marker is added in the sign language label string output by the translation unit 14. Then, the proper noun motion synthesis unit 23 synthesizes a sign language motion corresponding to a proper noun by sequentially connecting those sign language motions with a predetermined first transition length. The first transition length is a transition length for synthesizing the motion of a proper noun.

[0049] Furthermore, the proper noun motion synthesis unit 23 reads out mouth motions corresponding to each sound contained in the estimated reading of the proper noun from the mouth motion database 21. The proper noun motion synthesis unit 23 synthesizes a mouth motion corresponding to the proper noun by sequentially connecting those mouth motions based on the estimated reading. The proper noun motion synthesis unit 23 also replaces data related to mouth shapes in the sign language motion corresponding to the proper noun (including cases where there is no mouth motion) with the mouth motion corresponding to the synthesized proper noun.

[0050] The proper noun motion synthesis unit 23 performs the above-mentioned proper noun motion synthesis process for each of all proper nouns included in the translation result output by the translation unit 14.

[0051] The sign language motion synthesis unit 24 receives the sign language label string resulting from the translation from the translation unit 14, and also receives motion data for each proper noun from the proper noun motion synthesis unit 23. The sign language motion synthesis unit 24 then acquires sign language motion data from the sign language motion database 22 for the sign language label string other than the proper noun portion. The sign language motion synthesis unit 24 also connects the sign language motions in an order corresponding to the sign language label string. At that time, the sign language motion synthesis unit 24 also performs processing for connecting between labels (such as the above-mentioned sign language motion interpolation). The sign language motion synthesis unit 24 connects the sign language motions other than the proper noun portion generated in this manner with the sign language motions for the proper noun portion received from the proper noun motion synthesis unit 23, and generates overall sign language motion data that corresponds as a whole to the original Japanese text.

[0052] That is, the sign language motion synthesis unit 24 reads out sign language motions corresponding to each of the sign language labels from the sign language motion database 22 based on the sign language label string not having the proper noun marker in the sign language label string output by the translation unit 14. Then, the sign language motion synthesis unit 24 sequentially connects the sign language motions with the second transition length to synthesize a sign language motion corresponding to a part other than a proper noun. Then, the sign language motion synthesis unit 24 further connects the sign language motion corresponding to the proper noun synthesized by the proper noun motion synthesis unit 23 based on the sign language label string output by the translation unit 14 to the sign language motion corresponding to the part other than the proper noun. In this way, the sign language motion synthesis unit 24 synthesizes data of the sign language motion corresponding to the original text. Note that the above-mentioned second transition length is the transition length between the sign language labels of the part other than the proper noun. Note that the first transition length is set to be shorter than the second transition length (the normal transition length of the part other than the proper noun).

[0053] The sign language motion synthesis unit 24 passes the obtained sign language motion data to the CG animation generation unit 31. This motion data includes all of the motions of the fingers, facial expressions, and mouth shapes (joints around the mouth).

[0054] The CG animation generation unit 31 generates CG animation based on the motion data passed from the sign language motion synthesis unit 24. Specifically, the CG animation generation unit 31 generates animation corresponding to the movement of each joint included in the motion data. The process of generating CG animation from motion data can be realized using existing technology. The CG animation generation unit 31 is also simply called the "animation generation unit."

[0055] Specifically, CG animation generating unit 31 holds data of a plurality of CG avatars in advance. CG animation generating unit 31 applies the motion data passed from sign language motion synthesis unit 24 to a specific CG avatar and performs animation rendering to generate a CG animation in which the CG avatar performs a sign language motion.

[0056] That is, the CG animation generating unit 31 generates an animation image based on the sign language motion data corresponding to the input text synthesized by the sign language motion synthesizing unit 24.

[0057] Next, details of data used in the processing of translation device 1 will be described.

[0058] 2 is a schematic diagram showing an example of Japanese text acquired from the outside by the text acquisition unit 11. The example text shown in the figure is "The amount of snow that will fall by tonight will be 20 cm at its heaviest on the Pacific side of southern Tohoku." In this sentence of text, "southern Tohoku" and "Pacific side" are proper nouns. As a result of the morphological analysis process, the morphological analysis unit 12 assigns part-of-speech information of proper nouns to "southern Tohoku" and "Pacific side."

[0059] FIG. 3 is a schematic diagram showing an example of translation result data output by the translation unit 14. This translation result data includes a string of sign language labels and other information. The example of the translation result shown here is the result of translation by the translation unit 14 based on the Japanese text of FIG. 2. The example of data shown in FIG. 3 is a string of symbols (labels, etc.) separated by commas. Here, each of "dark", "until", "snow", "amount", "a lot", "place", etc. is a label equivalent to a word in sign language. Also, N is a special symbol and corresponds to the action of nodding in sign language. Also, "tilt head" and "air writing" correspond to other actions in sign language.

[0060] The data output by the translation unit 14 in this embodiment is characterized by the notation "PN{···}_M{···}". This is data corresponding to a proper noun included in the original text. In other words, this data "PN{···}_M{···}" is a proper noun marker. As described above, the translation unit 14 receives information on a proper noun included in the original text. The translation unit 14 outputs data on the proper noun marker in correspondence with the input proper noun. The data "PN{···}" included in the proper noun marker is data for storing a sign language label string corresponding to the proper noun. Similarly, the data "M{···}" included in the marker is data for storing a reading related to the proper noun.

[0061] Specifically, "PN{east, north, hot}_M{Tohoku Nanbu}" in the data shown in FIG. 3 corresponds to the proper noun "Southern Tohoku" in the text in FIG. 2. This sign language label string of "east, north, hot" represents the sign language expression of "Southern Tohoku". Furthermore, "Tohoku Nanbu" is the reading data estimated by the proper noun reading estimation unit 13 for the proper noun "Southern Tohoku". Furthermore, "PN{very, around}_M{Taiheiyogawa}" in the data shown in FIG. 3 corresponds to the proper noun "Pacific Ocean side" in the text in FIG. 2. This sign language label string of "very, around" represents the sign language expression of "Pacific Ocean side". Furthermore, "Taiheiyogawa" is the reading data estimated by the proper noun reading estimation unit 13 for the proper noun "Pacific Ocean side".

[0062] That is, when the input Japanese text contains a proper noun, translation unit 14 outputs the translation result corresponding to the proper noun as data represented by a proper noun marker. Translation unit 14 also outputs the reading of the proper noun (reading estimation result) passed from proper noun reading estimation unit 13 as is, in data in the format "M{···}". Translation unit 14 also outputs the sign language label string (translation result) corresponding to the proper noun, in data in the format "PN{···}".

[0063] 4 is a schematic diagram showing an example of the format of data stored in the mouth shape motion database 21. As shown in the figure, the mouth shape motion database 21 stores data in, for example, a table format. This table has items for sounds (reading elements) and mouth shape motions. In other words, the mouth shape motion database 21 stores mouth shape motion data corresponding to sounds. The sounds are, for example, sounds such as a, i, u, e, o, and n, or other sounds. The mouth shape motion data is data that represents the motion around the mouth corresponding to each sound.

[0064] 5 is a schematic diagram showing an example of the format of data stored in the sign language motion database 22. As shown in the figure, the sign language motion database 22 stores data in, for example, a table format. This table has items of sign language labels and sign language motions. In other words, the sign language motion database 22 stores data of sign language motions corresponding to each sign language label. The sign language motion data is data that represents motions (movements of fingers and facial expressions) corresponding to sign language actions.

[0065] Fig. 6 is a schematic diagram showing an example of CG animation based on the motion data synthesized by the sign language motion synthesis unit 24. The animation shown here corresponds to the part "···, say, N, PN{east, north, hot}_M{Tohoku Nanbu}, PN{very, around}_M{Taiheiyogawa}, N, 20,···" in the translation result (sign language label string) in Fig. 3. In this series of animations, section A corresponds to the proper noun "Southern Tohoku." Also, section B corresponds to the proper noun "Pacific Ocean side."

[0066] As described above, the proper noun motion synthesis unit 23 makes the speed of the sign language motion corresponding to the label string of "East, North, Hot" faster than the speed of the sign language motion of the portion other than the proper noun. In addition, the proper noun motion synthesis unit 23 makes the connection time between the labels in the label string of "East, North, Hot" shorter than the connection time between the labels in the portion other than the proper noun. In addition, the proper noun motion synthesis unit 23 makes the mouth shape of the portion corresponding to the label string of "East, North, Hot" a mouth shape corresponding to "Tohoku Nanbu" instead of "Higashi / Kita / Atsui". In other words, the proper noun motion synthesis unit 23 reads out the mouth shape motions corresponding to each sound of "o / u / o / u / a / n / u", which is a string of vowels and glottal stops corresponding to "Tohoku Nanbu", from the mouth shape motion database 21. The proper noun motion synthesis unit 23 connects those mouth shape motions to synthesize a motion corresponding to the proper noun "Tohoku Nanbu". As a result, the CG animation generating unit 31 generates a sign language animation that is easily recognized as the proper noun "Southern Tohoku."

[0067] Similarly, the proper noun motion synthesis unit 23 makes the speed of the sign language motion corresponding to the label string "very, atari" faster than the speed of the sign language motion of the portion other than the proper noun. The proper noun motion synthesis unit 23 also makes the connection time between labels in the label string "very, atari" shorter than the connection time between labels in the portion other than the proper noun. The proper noun motion synthesis unit 23 also sets the mouth shape of the portion corresponding to the label string "very, atari" to the mouth shape corresponding to "Taiheiyogawa" instead of "Totemo / Atari". That is, the proper noun motion synthesis unit 23 reads out the mouth shape motions corresponding to each sound of "a / i / e / i / o / u / a / a", which is the string of vowels corresponding to "Taiheiyogawa", from the mouth shape motion database 21. The proper noun motion synthesis unit 23 connects those mouth shape motions to synthesize a motion corresponding to the proper noun "Pacific Ocean side". As a result, the CG animation generating unit 31 generates a sign language animation that is easily recognized as a proper noun, "Pacific Ocean side."

[0068] 7 is a flowchart showing the procedure of processing by translation device 1. The operation procedure will be described below with reference to this flowchart.

[0069] First, in step S11, text acquisition unit 11 of translation apparatus 1 acquires Japanese text to be translated into sign language from outside.

[0070] Next, in step S12, the morpheme analysis unit 12 performs a morpheme analysis process on the Japanese text (sentence, paragraph) acquired in step S11. As a result, the morpheme analysis unit 12 divides the Japanese text into morphemes and assigns part-of-speech information to each morpheme. Here, some morphemes may be proper nouns.

[0071] Next, in step S13, translation device 1 determines whether or not a proper noun is included in the result of the morphological analysis process in step S12. If a proper noun is included (step S13: YES), the process proceeds to step S14. If a proper noun is not included (step S13: NO), the process jumps to step S15.

[0072] Next, when the process proceeds to step S14, in that step, the proper noun reading estimation unit 13 estimates the reading of the detected proper noun. When multiple proper nouns are included in the original Japanese text, the proper noun reading estimation unit 13 estimates the reading of each of those proper nouns.

[0073] Next, in step S15, the translation unit 14 translates the Japanese text acquired in step S11 into a sign language label string. If the Japanese text contains a proper noun, information indicating a grouping of the proper noun (proper noun marker) is maintained in the sign language label string that is the translation result. If the Japanese text contains a proper noun, the translation unit 14 includes the reading data of the proper noun that is the result estimated in step S14 in the label string that is the translation result (see also FIG. 3).

[0074] Next, in step S16, translation device 1 determines whether the translation result (sign language label string data) generated in step S15 includes a proper noun marker. If the translation result data includes a proper noun marker (step S16: YES), the process proceeds to step S17. If the translation result data does not include a proper noun marker (step S16: NO), the process jumps to step S19.

[0075] Next, if the process proceeds to step S17, in that step the proper noun motion synthesis unit 23 reads data of sign language motion for proper nouns and synthesizes them.

[0076] Specifically, the proper noun motion synthesis unit 23 reads sign language motion data for each proper noun included in the translation result output by the translation unit 14 from the sign language motion database 22. The proper noun motion synthesis unit 23 reads sign language motion data for each sign language label included in the sign language label string corresponding to the proper noun from the sign language motion database 22. That is, as a specific example, for one of the proper noun parts in the translation result shown in FIG. 3, "PN{east, north, hot}_M{Tohoku Nanbu}", the proper noun motion synthesis unit 23 reads corresponding sign language motions for each label, "east", "north", and "hot", from the sign language motion database 22. Then, the proper noun motion synthesis unit 23 sequentially connects the read sign language motions according to the order of the sign language label string. In the case of the label string "{east, north, hot}", the proper noun motion synthesis unit 23 connects the sign language motion corresponding to "east", the sign language motion corresponding to "north", and the sign language motion corresponding to "hot". However, for the transition portion from one sign language label to another, the proper noun motion synthesis unit 23 performs connection by linear interpolation or the like. Note that the proper noun motion synthesis unit 23 executes the sign language motion at a speed that is predetermined for the proper noun. The proper noun motion synthesis unit 23 also sets the length of connection time for the transition portion to the length that is predetermined for the proper noun.

[0077] By the process of step S17, proper noun motion synthesis unit 23 creates data of motion other than the mouth shape for the proper noun. At the stage where the process of step S17 is completed, the mouth shape motion of the proper noun part remains in the state of the motion data read from sign language motion database 22.

[0078] Next, in step S18, the proper noun motion synthesis unit 23 reads data of mouth motions for proper nouns and synthesizes them.

[0079] Specifically, the proper noun motion synthesis unit 23 reads mouth-shaped motion data for each sound of the reading of each proper noun included in the translation result output by the translation unit 14 from the mouth-shaped motion database 21. As a specific example, for one of the proper noun parts in the translation result shown in FIG. 3, "PN{Higashi, Kita, Atsui}_M{Tohoku Nanbu}", the proper noun motion synthesis unit 23 reads corresponding mouth-shaped motions for each sound of "To", "U", "Ho", "Ku", "Na", "N" and "Bu" from the mouth-shaped motion database 21. When the mouth-shaped motion database 21 holds data of mouth-shaped motions corresponding to each of the vowels and glottal stops as shown in FIG. 4, the proper noun motion synthesis unit 23 reads data of mouth-shaped motions corresponding to each of the sounds of "o", "u", "o", "u", "a", "n" and "u". Then, the proper noun motion synthesis unit 23 sequentially connects the read mouth-shaped motions according to the order of the sound sequence. In the case of the reading "{Tohoku Nanbu}" above, the proper noun motion synthesis unit 23 connects mouth motions corresponding to the sounds o, u, o, u, a, n, and u in that order. At this time, the proper noun motion synthesis unit 23 adjusts the speed of the mouth motions so that the total time length of the connected mouth motions matches the time length of the sign language motion of the proper noun synthesized in step S17. Furthermore, the proper noun motion synthesis unit 23 replaces the mouth motion data included in the sign language motion of the proper noun synthesized in step S17 with the mouth motion data corresponding to the reading of the proper noun generated in step S17.

[0080] By the process of step S18 described above, proper noun motion synthesis unit 23 can generate mouth motion data corresponding to the reading of the proper noun. Furthermore, proper noun motion synthesis unit 23 replaces the mouth motion data in the sign language motion data corresponding to the proper noun (synthesized in step S17) with the mouth motion data corresponding to the reading. In other words, by the processes of steps S17 and S18, proper noun motion synthesis unit 23 generates sign language motion data that can be correctly recognized as a proper noun. Note that proper noun motion synthesis unit 23 passes the synthesized sign language motion data of the proper noun to sign language motion synthesis unit 24.

[0081] Next, in step S19, the sign language motion synthesis unit 24 reads sign language motion data for sign language labels other than the proper noun portion from the sign language motion database 22. Specifically, the sign language motion synthesis unit 24 reads motion data from the sign language motion database 22 for all sign language labels (including action labels such as N, [air writing], and [head tilt]) other than the proper noun portion in the translation result output by the translation unit 14.

[0082] Next, in step S20, the sign language motion synthesis unit 24 synthesizes the entire sign language motion for the input text. Specifically, in this step, the sign language motion synthesis unit 24 connects the motion data read in step S19 in the order of the sign language label string that is the translation result. Also, the sign language motion synthesis unit 24 connects the sign language data synthesized in the processes of steps S17 and S18 in the order of the sign language label string output by the translation unit 14. When the sign language motion synthesis unit 24 connects multiple sign language motions in this step, it generates the motion of the transition part by an interpolation process or the like. Also, the sign language motion synthesis unit 24 adjusts the time length (number of frames) of the transition part so that it becomes a length that is predetermined as the transition length other than for proper nouns.

[0083] The processing in step S20 completes sign language motion data corresponding to the entire text acquired in step S11. The sign language motion synthesis unit 24 passes the completed entire motion data to the CG animation generation unit 31.

[0084] Next, in step S21, CG animation generator 31 generates CG animation based on the sign language motion data completed in step S20. Specifically, CG animation generator 31 uses a definition of a pre-stored avatar or a specified avatar to render an animation image of each frame so that the avatar moves according to the motion data. These frame images are arranged in chronological order and appropriately coded as necessary to create a video representing the sign language that is the result of translating Japanese text.

[0085] FIG. 8 is a schematic diagram showing at least some of the setting values ​​that the translation device 1 refers to in order to perform processing. As shown in the figure, the translation device 1 sets the inter-label span length within a proper noun as a settable value. The setting value of this item is, for example, "0.25 seconds." The translation device 1 also sets the inter-label span length within a non-proper noun as a settable value. The setting value of this item is, for example, "0.60 seconds." The translation device 1 also sets the playback speed ratio of the label within the proper noun (how many times the playback speed of the label other than the proper noun) as a settable value. The setting value of this item is, for example, "1.30." The translation device 1 may also have other setting items. Note that the setting values ​​shown here are merely examples. When using the translation device 1, these setting values ​​can be set to appropriate values ​​that are determined in advance for each item.

[0086] FIG. 9 is a block diagram showing an example of the internal configuration of the translation device 1 described as an embodiment. The translation device 1 can be realized using a computer. As shown in the figure, the computer includes a central processing unit 901, a RAM 902, an input / output port 903, input / output devices 904 and 905, and a bus 906. The computer itself can be realized using existing technology. The central processing unit 901 executes instructions included in a program read from the RAM 902 or the like. The central processing unit 901 writes data to the RAM 902, reads data from the RAM 902, and performs arithmetic and logical operations according to each instruction. The RAM 902 stores data and programs. Each element included in the RAM 902 has an address and can be accessed using the address. The RAM is an abbreviation for "random access memory." The input / output port 903 is a port through which the central processing unit 901 exchanges data with an external input / output device or the like. The input / output devices 904 and 905 are input / output devices. Input / output devices 904 and 905 exchange data with the central processing unit 901 via an input / output port 903. A bus 906 is a common communication path used inside the computer. For example, the central processing unit 901 reads and writes data from and to a RAM 902 via the bus 906. Also, for example, the central processing unit 901 accesses the input / output port via the bus 906.

[0087] At least some of the functions of the translation device 1 in the above-mentioned embodiment can be realized by a computer. In that case, a program for realizing the functions may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the functions. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, CD-ROMs, DVD-ROMs, and USB memories, and storage devices such as hard disks built into a computer system. In other words, the term "computer-readable recording medium" may be a non-transitory computer-readable recording medium. The term "computer-readable recording medium" may also include a medium that temporarily and dynamically holds a program, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in such a case. The above-mentioned program may be a program for realizing some of the above-mentioned functions, or may be a program that can be realized in combination with a program already recorded in the computer system.

[0088] As described above, when a proper noun is included in the input text, the translation device 1 according to this embodiment can output a sign language motion with a pause that is easily recognized as the proper noun. Also, when a proper noun is included in the input text, the translation device 1 according to this embodiment can output a sign language motion with a mouth shape that corresponds to the reading of the proper noun.

[0089] Although the embodiment has been described above, the present invention can also be embodied in the following modified examples. Note that a plurality of modified examples may be combined insofar as they can be combined.

[0090] [Variation 1] In the above embodiment, the text to be translated is a text written in Japanese. Alternatively, text written in another natural language may be imported as the translation target. In the above embodiment, the sign language label string output as the translation result is a label string in Japanese Sign Language. The translation device 1 has a function of generating a CG animation of Japanese Sign Language based on the label string of Japanese Sign Language that is the translation result. Alternatively, the translation result may be a label string of another type of sign language. Furthermore, the translation device 1 may generate a CG animation of a type of sign language (other than Japanese Sign Language) corresponding to the label string of that sign language (sign language other than Japanese Sign Language) based on the label string of that sign language. In these cases, too, the translation device can output a sign language that is correctly recognized as a proper noun by treating proper nouns that appear in the input text as a set of label strings.

[0091] [Variation 2] In the above embodiment, the time length of the transition is shortened and the speed of the sign language motion corresponding to each label is increased so that the sign language motion corresponding to the part of the proper noun can be recognized as a group of proper nouns. Such an adjustment is conveniently called time adjustment. In the above embodiment, in addition to the time adjustment, mouth shape motion data based on the estimation result of the reading of the proper noun is synthesized, and the movement around the mouth corresponding to the mouth shape motion data is realized by CG animation. Such an adjustment is conveniently called mouth shape adjustment. As a second modification, the translation device 1 may only perform the above-mentioned time adjustment and not perform the mouth shape adjustment. Conversely, the translation device 1 may only perform the above-mentioned mouth shape adjustment and not perform the time adjustment. Even when such a second modification is implemented, it is possible to realize a sign language motion that is more easily recognized as a proper noun than the sign language translation by the conventional technology.

[0092] [Variation 3] In the above embodiment, the proper noun motion synthesis unit 23 makes the playback speed of each sign language label included in the sign language motion corresponding to a proper noun faster than the playback speed of each sign language label included in the sign language motion corresponding to a non-proper noun. As a modified example, this is not necessarily required. That is, the proper noun motion synthesis unit 23 may make the playback speed of each sign language label included in the sign language motion corresponding to a proper noun approximately the same as the playback speed of each sign language label included in the sign language motion corresponding to a non-proper noun.

[0093] [Variation 3] In the above embodiment, the proper noun motion synthesis unit 23 replaces data related to mouth shapes in the sign language motion corresponding to the proper noun (including the case where there is no mouth motion) with the mouth motion corresponding to the proper noun. As a modified example, the proper noun motion synthesis unit 23 may not replace the mouth motion based on the estimated reading. Even in this case, the sign language motion output by the translation device 1 makes it easy to recognize the proper noun by the length of the transition other than the mouth motion and the playback speed of the motion corresponding to the sign language label.

[0094] The above describes in detail an embodiment of the present invention (including modified examples) with reference to the drawings. However, the specific configuration is not limited to this embodiment, and also includes designs that do not deviate from the gist of the present invention. [Industrial Applicability]

[0095] The present invention can be used, for example, to generate sign language. Industries that can benefit from efficient generation of sign language include content production and distribution businesses, for example. However, the scope of use of the present invention is not limited to the examples given here. [Explanation of symbols]

[0096] 1 Translation device 11 Text Acquisition Section 12 Morphological analysis section 13 Proper noun pronunciation estimation unit 14 Translation Department 21 Mouth Shape Motion Database 22 Sign Language Motion Database 23 Proper noun motion synthesis unit 24 Sign Language Motion Synthesis Unit 31 CG animation generation section (animation generation section) 901 Central Processing Unit 902 RAM 903 Input / Output Ports 904,905 Input / Output Devices 906 Bus

Claims

1. a morpheme analysis unit that performs a morpheme analysis process on an input text, outputs a morpheme string, and, when a morpheme included in the morpheme string is a proper noun, adds data indicating that the morpheme is a proper noun; a translation unit that converts the morpheme string output from the morpheme analysis unit into a sign language label string based on a machine learning model, and adds a proper noun marker to a portion of the sign language label string after the conversion that corresponds to a morpheme to which data indicating that the morpheme is a proper noun is added; a sign language motion database that holds a correspondence between a sign language label and a sign language motion; a proper noun motion synthesis unit that reads out sign language motions corresponding to each of the sign language labels from the sign language motion database based on a sign language label string to which the proper noun marker is added from among the sign language label strings output by the translation unit, and synthesizes a sign language motion corresponding to a proper noun by sequentially connecting the sign language motions with a first span; a sign language motion synthesis unit that reads out sign language motions corresponding to each of the sign language labels from the sign language motion database based on a sign language label string not marked with a proper noun marker from among the sign language label string output by the translation unit, sequentially connects the sign language motions at a second span to synthesize a sign language motion corresponding to a word other than a proper noun, and further connects the sign language motion corresponding to the proper noun synthesized by the proper noun motion synthesis unit based on the sign language label string output by the translation unit to the sign language motion corresponding to the word other than the proper noun, thereby synthesizing a sign language motion corresponding to the text; Equipped with The first span length is shorter than the second span length; Translation device.

2. a playback speed of each sign language label included in the sign language motion corresponding to the proper noun is made faster than a playback speed of each sign language label included in the sign language motion corresponding to a word other than the proper noun; The translation device according to claim 1.

3. a proper noun reading estimation unit that estimates the reading of a proper noun for the morpheme to which data indicating that the morpheme is a proper noun is added by the morpheme analysis unit; a mouth motion database that holds a correspondence between the pronunciation of the pronunciation and a mouth motion that is a part of the sign language motion; Further equipped with the proper noun motion synthesis unit reads out mouth motions corresponding to each sound included in the estimated reading of the proper noun from the mouth motion database, synthesizes a mouth motion corresponding to the proper noun by sequentially connecting the mouth motions based on the reading, and replaces data relating to mouth shapes in a sign language motion corresponding to the proper noun with the synthesized mouth motion corresponding to the proper noun; 3. The translation device according to claim 1 or 2.

4. an animation generation unit that generates an animation image based on the sign language motion data corresponding to the text generated by the sign language motion synthesis unit; The translation device according to claim 1 , further comprising:

5. a morpheme analysis unit that performs a morpheme analysis process on an input text, outputs a morpheme string, and, when a morpheme included in the morpheme string is a proper noun, adds data indicating that the morpheme is a proper noun; a translation unit that converts the morpheme string output from the morpheme analysis unit into a sign language label string based on a machine learning model, and adds a proper noun marker to a portion of the sign language label string after the conversion that corresponds to a morpheme to which data indicating that the morpheme is a proper noun is added; a sign language motion database that holds a correspondence between a sign language label and a sign language motion; a proper noun motion synthesis unit that reads out sign language motions corresponding to each of the sign language labels from the sign language motion database based on a sign language label string to which the proper noun marker is added from among the sign language label strings output by the translation unit, and synthesizes a sign language motion corresponding to a proper noun by sequentially connecting the sign language motions with a first span; a sign language motion synthesis unit that reads out sign language motions corresponding to each of the sign language labels from the sign language motion database based on a sign language label string not marked with a proper noun marker from among the sign language label string output by the translation unit, sequentially connects the sign language motions at a second span to synthesize a sign language motion corresponding to a word other than a proper noun, and further connects the sign language motion corresponding to the proper noun synthesized by the proper noun motion synthesis unit based on the sign language label string output by the translation unit to the sign language motion corresponding to the word other than the proper noun, thereby synthesizing a sign language motion corresponding to the text; Equipped with The first span length is shorter than the second span length; A program that enables a computer to function as a translation device.

Images