Display device, display method, and program

Abstract

To provide a display device for displaying a plurality of character strings having been converted into a plurality of languages from inputted handwritten data.SOLUTION: The present invention provides a display device comprising: acceptance means for accepting input of handwritten data; display control means for displaying a plurality of character strings having been converted into a plurality of languages from the handwritten data accepted by the acceptance means; and selection means for accepting selection of a character string from the plurality of character strings displayed by the display control means.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a display device, a display method, or a program. [Background technology]

[0002] There is a known display device that uses handwriting recognition technology to convert handwritten data into characters and display them on a display. Display devices equipped with relatively large touch panels are placed in conference rooms and public facilities and are used by multiple users as electronic whiteboards. Display devices are also sometimes used as communication tools by writing.

[0003] Also, a technique has been devised for converting handwritten data into another language using handwriting recognition technology (see, for example, Patent Document 1). Patent Document 1 discloses a technique for performing character recognition on handwritten data to convert it into text data, and then converting the text data into a different language. Summary of the Invention [Problem to be solved by the invention]

[0004] However, while conventional technology displays a character string converted into one language from a single input handwritten data, in situations where the display device is used in workplaces or on-site where speakers of multiple different languages ​​coexist, there are cases where it is desired to display multiple character strings converted into multiple languages ​​for a single input handwritten data.

[0005] In view of the above problems, an object of the present invention is to provide a display device capable of displaying a plurality of character strings converted into a plurality of languages ​​from input handwritten data. [Means for solving the problem]

[0006] In view of the above problems, the present invention provides a display control device that converts the handwritten data received by the handwritten data input into a plurality of languages ​​and displays a plurality of character strings, a selection means for accepting a selection of a character string from the plurality of character strings displayed by the display control means, the display control means converting the handwritten data to the character string selected by the selection means and displaying the character string, setting the language of the character string selected by the selection means to a conversion destination language, and converting the handwritten data accepted by the acceptance means into the conversion destination language after the selection means accepts the selection of the character string, and the display control means displays the character string converted into the conversion destination language by the conversion means.A display device is provided. Effect of the Invention

[0007] It is possible to provide a display device that displays a plurality of character strings converted from input handwritten data into a plurality of languages. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram for explaining an outline of language conversion by the display device of the present embodiment. FIG. [Diagram 2] FIG. 1 is a diagram illustrating an overall configuration of an example of a display device. [Diagram 3] FIG. 2 is a diagram illustrating a hardware configuration of an example of a display device. [Figure 4] FIG. 2 is a functional block diagram of an example for explaining functions of the display device by dividing them into blocks. [Diagram 5] FIG. 13 is a diagram showing an example of a manipulation guide and selectable candidates displayed by the manipulation guide. [Figure 6] FIG. 13 is a diagram showing handwritten data of Case 1a and an example of its conversion. [Figure 7] FIG. 7 is a flowchart illustrating an example of the conversion procedure of FIG. 6. [Figure 8] FIG. 13 is a diagram showing handwritten data of Case 1b and an example of its conversion. [Figure 9] FIG. 9 is a flowchart illustrating an example of the conversion example of FIG. 8. [Figure 10] FIG. 2 is a functional block diagram of an example for explaining functions of the display device by dividing them into blocks. [Figure 11] FIG. 13 is a diagram showing handwritten data and an example of its conversion in case 2a. [Figure 12] FIG. 12 is a flowchart illustrating an example of the conversion example of FIG. [Figure 13] FIG. 13 is a diagram showing handwritten data and an example of its conversion in case 2b. [Figure 14] FIG. 13 is a flowchart illustrating an example of the conversion example of FIG. 12. [Figure 15]FIG. 13 is a diagram showing handwritten data and an example of its conversion in case 2c. [Figure 16] FIG. 16 is a flowchart illustrating an example of the conversion example of FIG. 15. [Figure 17] FIG. 13 is a diagram showing an example of an icon button for setting a conversion target language. [Figure 18] FIG. 18 is a flowchart illustrating an example of the conversion example of FIG. 17. [Figure 19] FIG. 13 is a diagram showing an example of displaying both a source language and a destination language. [Figure 20] FIG. 13 is a diagram illustrating an example of conversion into English and Chinese. [Figure 21] FIG. 13 is a diagram illustrating an example of conversion into English and Chinese. [Figure 22] FIG. 22 is a flowchart illustrating an example of the conversion example of FIG. 21. [Figure 23] 13 is a diagram illustrating a recommendation window in which a plurality of operation guides are added as starting points and displayed; FIG. [Figure 24] FIG. 13 is a diagram showing an example of an operation guide in which a display device displays selectable candidates in the operation guide like a caption. [Diagram 25] FIG. 13 is a diagram showing another configuration example of the display device. [Figure 26] FIG. 13 is a diagram showing another configuration example of the display device. [Figure 27] FIG. 13 is a diagram showing another configuration example of the display device. [Figure 28] FIG. 13 is a diagram showing another configuration example of the display device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A display device and a display method performed by the display device will be described below as an example of an embodiment of the present invention with reference to the drawings. EXAMPLES

[0010] There are situations where display devices are used in workplaces or on-site where multiple language speakers coexist. In such situations, when a person from one country wants to convey information by handwriting to a person from another country, communication can be made easier if the characters shown on the display are converted into a language that the person from the other country can understand. However, there may be cases where the person from one country does not know the language of the person from the other country very well. Furthermore, there may be cases where a person from one country does not know what language the person from the other country understands.

[0011] <Summary of string modification> FIG. 1 is a diagram for explaining an outline of language conversion by the display device of this embodiment. (1) First, the user writes "Hello." The user does not necessarily have to write the entire word "Hello." (2) The display device converts the handwritten data into the character string "Hello" through character recognition. The display device then displays a list of character strings in other languages ​​that have the same meaning as "Hello." In FIG. 1, character strings that mean "Hello" in English, Chinese, French, and Korean are displayed. "Other languages" refers to languages ​​other than the language of the handwritten data that has been character-recognized. (3) For example, a user who wants to convert to Chinese selects a Chinese string.

[0012] In this way, the display device of this embodiment can display a list of character strings with the same meaning in a language different from the language handwritten, even if the user does not set the conversion language in advance. Therefore, communication is possible even when a person from one country does not know the language of a person from another country well, or when a person from one country does not know what language a person from another country understands.

[0013] In the past, the database that established the relationships between Japanese words and English words (data in a computer having a relational data structure that collects and organizes related data to make searches and updates efficient) was managed as a separate database from the database that established the relationships between Japanese words and Chinese words. Therefore, before inputting handwritten data, the user had to specify the language to which the data would be converted (English or Chinese in the above example) in order to identify the database to be used.

[0014] This poses a problem in that the user must set the language into which the handwritten data is to be converted beforehand.

[0015] <Terminology> The input means may be any means capable of specifying coordinates on a touch panel and writing by hand, such as a pen, a human finger or hand, or a rod-shaped member.

[0016] A stroke refers to a series of operations in which a user presses an input means against a display, moves it continuously, and then releases it from the display. Stroke data is information displayed on the display based on a trajectory of coordinates input by the input means. Stroke data may be appropriately interpolated. Handwritten data is data having one or more stroke data. Handwritten input refers to handwritten data being input by a user.

[0017] An object is an object that is displayed on a display based on stroke data. Object means a target, but in this embodiment it means a display target. The character string converted from handwritten data through character recognition may include not only text data, but also data displayed based on user operations, such as fixed characters such as "Done," stamps displayed as marks, shapes such as circles and stars, and straight lines.

[0018] The confirmed data is data that has been converted into a character code (font) by character recognition of handwritten data and selected by the user, or handwritten data that has been confirmed not to be converted into a character code (font).

[0019] An operation command is a command prepared for operating the handwriting input device to instruct the execution of a specific process. In this embodiment, as an example, a case will be described in which the user instructs the display device 2 to rotate the entire image and to correspond the display direction to the converted character string. In addition, operation commands are also used in the editing system, modification system, input / output system, etc. of character strings.

[0020] A string is one or more characters that are handled by a computer. The actual entity of a string is a character code. Characters include numbers, alphabets, and symbols. Strings are also called text data.

[0021] Conversion means to change or replace. Converting a string of characters from one language to another is sometimes called translation.

[0022] <Example of device configuration> The overall configuration of the display device 2 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing the overall configuration of the display device 2. Fig. 2(a) shows, as an example of the display device 2, a display device 2 used as a horizontally long electronic whiteboard hung on a wall.

[0023] 2(a), a display 220 as an example of a display device is provided on the upper part of the display device 2. A user U can handwrite (also referred to as input or drawing) characters or the like on the display 220 using a pen 2500.

[0024] FIG. 2(b) shows a display device 2 used as a vertically long electronic whiteboard suspended on a wall.

[0025] 2(c) shows the display device 2 placed flat on a desk 230. Since the display device 2 is about 1 cm thick, there is no need to adjust the height of the desk even if it is placed flat on a normal desk. In addition, the user can easily move the display device 2.

[0026] The input method of the coordinates of the pen 2500 includes an electromagnetic induction method, an active electrostatic coupling method, etc. The pen 2500 may also have functions such as pen pressure detection, tilt detection, and a hover function (displaying the cursor before the pen touches the screen).

[0027] <Hardware configuration of the device> Next, the hardware configuration of the display device 2 will be described with reference to Fig. 3. As shown in the figure, the display device 2 has the configuration of an information processing device or a computer. Fig. 3 is an example of a hardware configuration diagram of the display device 2. As shown in Fig. 3, the display device 2 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and an SSD (Solid State Drive) 204.

[0028] Of these, the CPU 201 controls the overall operation of the display device 2. The ROM 202 stores the CPU 201 and programs used to drive the CPU 201, such as an IPL (Initial Program Loader). The RAM 203 is used as a work area for the CPU 201.

[0029] The SSD 204 stores various data such as an OS and a program for a display device. This program may be an application program that runs on an information processing device equipped with a general-purpose OS (Windows (registered trademark), Mac OS (registered trademark), Android (registered trademark), iOS (registered trademark), etc.).

[0030] The display device 2 also includes a display controller 213, a touch sensor controller 215, a touch sensor 216, a display 220, a power switch 227, a tilt sensor 217, a serial interface 218, a speaker 219, a microphone 221, a wireless communication device 222, an infrared I / F 223, a power control circuit 224, an AC adapter 225, and a battery 226.

[0031] The display controller 213 controls and manages the screen display in order to output an output image to the display 220, etc. The touch sensor 216 detects that the pen 2500, the user's hand, etc. (the pen and the user's hand serve as input means) have touched the display 220. The touch sensor 216 also receives a pen ID.

[0032] The touch sensor controller 215 controls the processing of the touch sensor 216. The touch sensor 216 inputs and detects coordinates. For example, in the case of an optical type, the input of coordinates and the detection of coordinates are performed by two light receiving / emitting devices installed at both ends of the upper side of the display 220 emitting a plurality of infrared rays in parallel to the display 220, and receiving the light reflected by a reflecting member provided around the display 220 and returning along the same optical path as the optical path of the light emitted by the light receiving element. The touch sensor 216 outputs position information of the infrared rays emitted by the two light receiving / emitting devices that are blocked by an object to the touch sensor controller 215, and the touch sensor controller 215 specifies the coordinate position that is the contact position of the object. The touch sensor controller 215 also has a communication unit 215a and can wirelessly communicate with the pen 2500. For example, when communication is performed according to a standard such as Bluetooth (registered trademark), a commercially available pen can be used. If one or more pens 2500 are registered in advance in the communication unit 215a, the user can communicate with the display device 2 without having to perform connection settings to allow the pen 2500 to communicate with the display device 2.

[0033] The power switch 227 is a switch for switching ON / OFF the power of the display device 2. The tilt sensor 217 is a sensor for detecting the tilt angle of the display device 2. It is mainly used to detect whether the display device 2 is being used in the installation state of Fig. 2(a), Fig. 2(b), or Fig. 2(c), and can automatically change the thickness of characters, etc. according to the installation state.

[0034] The serial interface 218 is an external communication interface such as a USB. The serial interface 218 is used for inputting information from the outside. The speaker 219 is used for outputting audio, and the microphone 221 is used for inputting audio. The wireless communication device 222 communicates with a terminal carried by the user, and relays a connection to the Internet, for example. The wireless communication device 222 communicates using Wi-Fi, Bluetooth (registered trademark), or the like, but any communication standard may be used. The wireless communication device 222 forms an access point, and when the user sets an SSID (Service Set Identifier) ​​and password obtained by the user in the terminal carried by the user, the user can connect to the access point.

[0035] It is preferable that the wireless communication device 222 is provided with two access points. a. Access point → Internet b. Access point → Internal network → Internet Access point a is for users outside the company, and users cannot access the company network but can use the Internet. Access point b is for users inside the company, and users can use the company network and the Internet.

[0036] The infrared I / F 223 detects the display device 2 arranged adjacently. The infrared I / F 223 can detect only the display device 2 arranged adjacently by utilizing the linearity of infrared rays. It is preferable that one infrared I / F 223 is provided on each side, and it can detect in which direction of the display device 2 another display device 2 is arranged. This expands the screen, and it is possible to display handwritten information that was previously handwritten on the adjacent display device 2 (handwritten information on another page, with the area of ​​one display 220 being one page) and the like.

[0037] The power supply control circuit 224 controls an AC adapter 225 and a battery 226 which are power sources for the display device 2. The AC adapter 225 converts AC common to commercial power sources into DC.

[0038] If the display 220 is so-called electronic paper, it consumes little or no power to maintain the image display, and therefore can be driven by a battery 226. This makes it possible to use the display device 2 for purposes such as digital signage even in places where it is difficult to connect to a power source, such as outdoors.

[0039] The display device 2 further includes a bus line 210. The bus line 210 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 201 shown in FIG.

[0040] The touch sensor 216 is not limited to an optical type, and may be a capacitive touch panel that identifies the contact position by detecting a change in capacitance, a resistive film touch panel that identifies the contact position by a voltage change between two opposing resistive films, or an electromagnetic induction touch panel that identifies the contact position by detecting electromagnetic induction caused by a contact object touching the display unit. The touch sensor 216 may be of a type that does not require an electronic pen to detect the presence or absence of a touch with the tip of the pen. In this case, a fingertip or a pen-shaped stick can be used to perform a touch operation. The pen 2500 does not need to be a long, thin pen type.

[0041] <About the function> Next, the functions of the display device 2 will be described with reference to Fig. 4. Fig. 4 is an example of a functional block diagram for explaining the functions of the display device 2 in blocks. The display device 2 has a contact position detection unit 21, a drawing data generation unit 22, a character recognition unit 23, a display control unit 24, a data recording unit 25, a network communication unit 26, an operation acceptance unit 27, and a conversion processing unit 28. Each function of the display device 2 is a function or means realized by any of the components shown in Fig. 3 operating according to an instruction from the CPU 201 in accordance with a program loaded from the SSD 204 onto the RAM 203.

[0042] The contact position detection unit 21 detects the coordinates of the position where the pen 2500 touches the touch sensor 216. The drawing data generation unit 22 acquires the coordinates where the tip of the pen 2500 touches from the contact position detection unit 21. The drawing data generation unit 22 generates stroke data by connecting the sequence of coordinate points by interpolating them.

[0043] The character recognition unit 23 performs character recognition processing on one or more stroke data (handwritten data) handwritten by the user, and converts it into a character code. In parallel with the user's pen operation, the character recognition unit 23 recognizes characters (not only Japanese but also multiple languages ​​such as English), numbers, symbols (%, $, &, etc.), shapes (lines, circles, triangles, etc.), etc. Various algorithms have been devised for the recognition method, but details will be omitted since known technology can be used in this embodiment.

[0044] The display control unit 24 displays the handwritten data, character strings converted from the handwritten data, and an operation menu for user operation on the display. The data recording unit 25 stores the handwritten data handwritten on the display device 2, the converted character strings, the PC screen, files, etc. in the storage unit 30. The network communication unit 26 connects to a network such as a LAN and transmits and receives data to and from other devices via the network.

[0045] The conversion processing unit 28 uses the defined control data storage unit 32 and the synonym dictionary 31 to convert a character string input by the user into a character string in another language that has the same meaning as the character string.

[0046] The display device 2 also has a memory unit 30 constructed in the SSD 204 or RAM 203 shown in FIG. 3, and the memory unit 30 has a synonym dictionary 31, a defined control data memory unit 32, and an input data memory unit 33 constructed therein.

[0047] [Table 1] Table 1 shows a schematic of the contents of the synonym dictionary 31. The synonym dictionary 31 is a dictionary in which synonyms in multiple languages ​​are registered in association with each other. Synonyms are words that have different pronunciations or spellings but the same meaning. They are also called synonyms. The conversion processing unit 28 can convert handwritten data into multiple other languages ​​with the same meaning by referring to the synonym dictionary.

[0048] [Table 2] Table 2 shows a schematic diagram of the predefined control data stored in the predefined control data storage unit 32. The predefined control data is the control content of the display device 2 according to the input character string. For example, the predefined control data of the predefined control item "text language" 361 means that the Language attribute is set according to the type of language handwritten and input by the user. The Language attribute indicates the language into which the conversion processing unit 28 converts the character string recognized from the handwritten data. For example, when the user handwrites "English", the Language attribute is set to "English". The handwritten data handwritten by the user is converted into English.

[0049] The predefined control data for the predefined control item "Text Meaning" 362 means that the Text Meaning attribute is set according to the character string entered by the user. The Text Meaning attribute represents the general meaning of the entered character string in English. This meaning is used to search the synonym dictionary 31.

[0050] [Table 3] Table 3 shows a schematic of the contents of the input data storage unit 33. The input data indicates the attributes of the data input by the user for each piece of data. The input data is recorded for each object (one piece of stroke data, one character string, one image, etc.). Each of the input data 363 and 364 is one object. Each attribute will be explained. · DataId is the identification information of the input data. Type is the type of input data, and can be Stroke, Text, or Image. The attributes held by the input data can differ depending on the Type, but Table 3 explains the case where Type="Text". Text represents a character string, and Image is an image. ·PenId is the identification information of the pen 2500 used to input this character string. · colorId indicates the color identification information of the string. ·Angle is the display direction of this string. · StartPoint is the coordinate of the upper left vertex of the bounding rectangle of the string. StartTime is the time when the user started writing the string. · EndPoint is the coordinate of the lower right vertex of the bounding rectangle of the string. · EndTime is the time when the user finishes writing the string. · FontName is the font name as a string. · FontSize is the font size. ·Text is the entered text (character code). ·meaning is the meaning of the string. · Language is the language of the string.

[0051] Input data 363 in Table 3 is a case where "Hello" is entered in Japanese, and input data 364 is a case where "Nihao" is entered in Chinese. The attributes of Meaning and Language are specified by predefined control data. Note that although "Nihao" is written in Chinese in the tables and figures, due to restrictions on the languages ​​that can be used in the application system provided by the Japan Patent Office, in this specification it will be written in Katakana as "Nihao." The same applies to other languages ​​whose use is restricted.

[0052] <Example of selectable candidates> Next, referring to Fig. 5, an operation guide 500 displayed when converting handwritten data will be described. Fig. 5 shows an example of the operation guide 500 and selectable candidates 530 displayed by the operation guide. The user handwrites handwritten data 504, and the operation guide 500 is displayed. The operation guide 500 displays operation command candidates 510, handwritten recognition character string candidates 506, converted character string candidates 507, and character string / predictive conversion candidates 508. The selectable candidates 530 are the operation command candidates 510, the handwritten recognition character string candidates 506, the converted character string candidates 507, and the character string / predictive conversion candidates 508. The selectable candidates 530 excluding the operation command candidates 510 are referred to as character string candidates 539.

[0053] Handwritten data 504 is the character "gi" handwritten by the user. Handwritten data rectangular area display 503 is displayed surrounding handwritten data 504. In Fig. 5, operation guide 500 is displayed when one character is input, but the timing at which operation guide 500 is displayed is the timing at which the user stops handwriting. Therefore, the number of characters in handwritten data 504 is arbitrary.

[0054] The character string candidates are arranged in descending order of probability in handwriting recognition character string candidate 506, conversion character string candidate 507, and character string / predictive conversion candidate 508. "gi" in handwriting recognition character string candidate 506 is a candidate for the recognition result. In this example, character recognition unit 23 correctly recognizes "gi".

[0055] Conversion character string candidate 507 is a conversion character string candidate (e.g., a phrase containing "technique") converted from the result of kana-kanji conversion of "gi" (e.g., "technique"). In this example, "Giryoshi" is an abbreviation for "Technology Mass Production Trial." Character string / predictive conversion candidate 508 is a predicted character string candidate converted from conversion character string candidate 507. In this example, "Approve the giryoshi" and "Address to send the minutes of the meeting" are displayed.

[0056] The operation command candidates 510 are predefined operation command candidates (commands for file operations, editing characters, etc.) that are displayed according to the recognized characters. In the example of Fig. 5, the first character of the line "》" 511 is shown to be an operation command candidate. In Fig. 5, the character string candidate "meiji", which is a character string for "gi", partially matches the definition data, and is therefore displayed as the operation command candidate 510.

[0057] The operation command candidates 510 are displayed when operation command definition data including the converted character string is found, and are not necessarily displayed at all times. In this embodiment, the operation command candidates 510 related to the conversion are displayed.

[0058] The operation header 520 has buttons 501, 509, 502, and 505. Button 501 accepts an operation to switch between predictive conversion and kana conversion. Button 502 operates the page of candidate display. In the example of FIG. 5, there are three candidate display pages, and the first page is currently displayed. Button 505 accepts the deletion of the operation guide 500. When the user presses button 505, the operation acceptance unit 27 accepts it, and the display control unit 24 erases the display except for the handwritten data. Button 509 accepts the deletion of all displays. When the user presses button 509, the operation acceptance unit 27 accepts it, and the display control unit 24 erases all displays shown in FIG. 5, including the handwritten data, allowing the user to start over with handwriting.

[0059] <Conversion example> Below are described some examples of conversion of handwritten data entered by a user.

[0060] <<Case 1>> In the customer service industry, the language used by the person communicating information may not be specified each time. In such cases, it is considered better for the display device 2 not to fix the conversion target language.

[0061] Case 1a FIG. 6 is a diagram showing handwritten data and an example of its conversion in case 1a. (1) The user handwrites “Hello” in Japanese. The drawing data generating unit 22 displays the handwritten data based on the coordinates detected by the contact position detecting unit 21. (2) When the user removes the pen 2500 from the display, the character recognition unit 23 starts character recognition. The character recognition unit 23 generates "Hello" as the direct recognition result (corresponding to the handwriting recognition character string candidate 506).

[0062] Next, the conversion processing unit 28 searches for predefined control data for "hello" to obtain Textmeaning="hello". The conversion processing unit 28 refers to the synonym dictionary 31 to identify character strings in other languages ​​that have the meaning of "hello".

[0063] As described above, the display control unit 24 displays the operation guide 500 in response to the user operations (1) and (2), and displays the character string 350 in a plurality of other languages ​​that are synonymous with Japanese. (3) When the user selects, for example, "Nihao" with the pen 2500, the display control unit 24 displays the character string selected by the user in another language.

[0064] FIG. 7 is a flow chart illustrating the conversion procedure of FIG.

[0065] The drawing data generating unit 22 displays the handwritten data "Hello" based on the coordinates detected by the contact position detecting unit 21 (S1).

[0066] When the user removes the pen 2500 from the display and stops handwriting, the character recognition unit 23 starts character recognition. The character recognition unit 23 converts the recognition result into "Hello" (S2).

[0067] Next, the conversion processing unit 28 searches for predefined control data for "hello" to obtain Textmeaning="hello" (S3). The conversion processing unit 28 also determines the language used by the user based on a default setting or stroke data. Machine learning, for example, may be used to determine the language used by the user based on stroke data. The developer prepares training data that pairs stroke data and the language used, and generates a learning model using an algorithm such as a neural network or support vector machine. Here, it is determined that Language=Japanese.

[0068] Next, the conversion processing unit 28 refers to the synonym dictionary 31 to identify a character string in a language other than Japanese that has the meaning of "hello" (S4).

[0069] The display control unit 24 displays the operation guide 500, and displays "Hello" as the recognition result, as well as multiple character strings in other languages ​​that are synonymous with Japanese (S5).

[0070] When the display control unit 24 receives the selection of the character string candidate 539 from the user, the display control unit 24 displays the character string in the other language selected by the user (S6). The conversion processing unit 28 determines the language of the character string selected by the user. For example, Language=Chinese.

[0071] The data recording unit 25 saves the attributes of the displayed character string in the input data storage unit 33 (S7). The attributes characteristic of this embodiment are Text="Nihao", meaning="hello", and Language="Chinese". The other attributes are set to default values, for example.

[0072] In this way, when a user inputs handwritten data, the display device can display not only Japanese characters, but also characters of the same meaning in other languages.

[0073] Case 1b Instead of displaying the conversion candidate character strings themselves in the operation guide as shown in Figure 6, the conversion target language may be displayed first. For example, the English synonyms of "hello" are "Hello" and "Hi," and "Good morning," which has a similar meaning, is also listed as a candidate. If the display device displays multiple synonymous character strings for one language, the user will have to go through the options due to the large number of candidates. In cases like this where a single language has an enormous number of candidates, it is easier to support multiple languages ​​if the display device 2 employs a user interface that accepts language designation first.

[0074] FIG. 8 is a diagram showing handwritten data and an example of its conversion in case 1b.

[0075] (1) The user handwrites “Hello” in Japanese. The drawing data generating unit 22 displays the handwritten data based on the coordinates detected by the contact position detecting unit 21.

[0076] (2) When the user removes the pen 2500 from the display, the character recognition unit 23 starts character recognition. The character recognition unit 23 generates "Hello" as the direct recognition result (corresponding to the handwriting recognition character string candidate 506).

[0077] Next, since "Hello" is registered in the predefined control data, the conversion processing unit 28 displays the operation command "Translate". The operation command "Translate" may be displayed by a user separately setting the display device 2 to display it. The operation command "Translate" causes the display device 2 to execute a process called "Display list of conversion destination languages" (displaying a list of languages ​​that the display device can convert).

[0078] As described above, the display control unit 24 displays the operation guide 500 in response to the user operations (1) and (2), and displays the operation command candidate 510 for “translate” and the Japanese character string candidate 539.

[0079] In FIG. 8, character string candidates 539 other than the operation commands are displayed as options, so the user can select a character string even if he or she wishes to input Japanese characters rather than convert them.

[0080] (3) When the user selects "Translate" with the pen 2500, the display control unit 24 displays a list of destination languages ​​based on the process "display list of destination languages." The list of destination languages ​​may be a list of language types set in the synonym dictionary 31. These language types are also operation commands.

[0081] (4) When the user selects "English" with the pen 2500, the display control unit 24 displays the English word having the same meaning as "Hello" and its synonyms 351 on the operation guide 500. That is, the conversion processing unit 28 searches for predefined control data for "Hello" to obtain Textmeaning="hello". Next, the conversion processing unit 28 refers to the synonym dictionary 31 to identify the English character string "Hello" that has the meaning of "hello". Furthermore, the conversion processing unit 28 refers to a general-purpose thesaurus to obtain, for example, "Hi", "Good morning", "Good afternoon", and "Good evening" as synonyms of "Hello".

[0082] (5) When the user selects "Hello" with the pen 2500, the display control unit 24 displays "Hello" selected by the user.

[0083] FIG. 9 is a flow chart illustrating an example of the conversion process shown in FIG.

[0084] The drawing data generating unit 22 displays the handwritten data "Hello" based on the coordinates detected by the contact position detecting unit 21 (S1-1).

[0085] The character recognition unit 23 starts character recognition. The character recognition unit 23 generates "Hello" as the direct recognition result (S2-2).

[0086] Next, the conversion processing unit 28 searches for predefined control data for "Hello", and because "Hello" has already been registered or based on the user's settings, it determines to display the predefined operation command "Translate" (S3-3). Note that the conversion processing unit 28 determines that the character string "Hello" is Japanese based on the stroke data that "Hello" has.

[0087] The display control unit 24 displays the operation guide 500 including the Japanese character string candidates and the operation command "translate" (S4-4).

[0088] The operation reception unit 27 receives the operation command "translate" from the operation guide, and the conversion processing unit 28 executes the process of this operation command (S5-5). The process of the operation command is "display list in conversion target language".

[0089] As a result of executing the process of "displaying a list of conversion destination languages", the display control unit 24 displays the languages ​​registered in the synonym dictionary 31 as a list of conversion destination languages ​​on the operation guide 500 (S6-6).

[0090] When the operation reception unit 27 receives "English" from the operation guide, the conversion processing unit 28 executes processing of this operation command (S7-7). The processing of this operation command is "display synonyms of English." The conversion processing unit 28 obtains Textmeaning="hello" by searching the predefined control data for "Hello." The conversion processing unit 28 refers to the synonym dictionary 31 to identify an English character string having the meaning of "hello." The conversion processing unit 28 refers to a general-purpose thesaurus to identify synonyms of "Hello." The display control unit 24 displays the operation guide 500 having "Hello" and its synonyms (S8-8).

[0091] When the selection of character string candidate 539 is accepted from the user, display control unit 24 displays the character string selected by the user (S9-9). Data recording unit 25 saves the attributes of the displayed character string in input data storage unit 33.

[0092] In this way, the display device 2 can first accept the conversion target language and display the conversion candidate character string in that language.

[0093] <Major Effects> As described above, the display device 2 of the present embodiment displays character strings in multiple languages ​​or displays the types of languages. This makes it easier to communicate when the display device 2 is used in a workplace or field where multiple language speakers coexist and the person in front of you does not know the language of the other person's country well or does not know which language the other person understands. EXAMPLES

[0094] In cases where a user can roughly predict or identify the language used by the other party to communicate information, such as a meeting with a known overseas business partner, the conversion target language may be fixed. In this embodiment, this situation is referred to as "Case 2" and several forms will be described.

[0095] Fig. 10 is an example of a functional block diagram for explaining the functions of the display device 2 in blocks. In the explanation of Fig. 10, components with the same reference numerals as in Fig. 4 perform similar functions, so that only the main components of this embodiment may be mainly explained in some cases.

[0096] The display device 2 of this embodiment has a command detection unit 29. The command detection unit 29 judges whether or not an operation command is included in a character string input by the user by referring to an operation command definition data storage unit, and causes the display control unit 24 to display the operation command corresponding to the character string.

[0097] Further, the display device 2 of this embodiment has an operation command definition data storage section 34 in the storage section 30.

[0098] [Table 4] Table 4 is an example of operation command definition data for the user to explicitly set the conversion target language. The contents of each attribute are shown below. ·Name is the display name of the operation command. · String is the string with which the user invokes this operation command. ·Command represents the content of the operation command processing.

[0099] For example, a character string related to English is set as the string of the operation command definition data 401. A character string related to Korean is set as the string of the operation command definition data 402. When the command detection unit 29 detects these character strings, the display control unit 24 displays the Name attribute of the corresponding operation command as an operation command candidate 510.

[0100] <<Case 2a>> The display device 2 fixes the conversion target language to the language of a character string selected by the user once. That is, the conversion processing unit 28 refers to only the language of the character string previously selected in the synonym dictionary. This allows the conversion processing unit 28 to convert to the same language every time.

[0101] 11 is a diagram showing handwritten data and a conversion example thereof in case 2a. (1) to (3) may be the same as in case 1a. In (3), the conversion processing unit 28 fixes the conversion target language to Chinese because "Ni hao" selected by the user is Chinese.

[0102] (4) The user handwrites the Japanese word “Thank you.” The drawing data generating unit 22 displays the handwritten data based on the coordinates detected by the contact position detecting unit 21.

[0103] (5) When the user removes the pen 2500 from the display, the character recognition unit 23 starts character recognition. The character recognition unit 23 generates “Thank you” as the direct recognition result.

[0104] Next, the conversion processing unit 28 searches the predefined control data for "arigatou" to obtain Textmeaning="thanks". The conversion processing unit 28 refers to the synonym dictionary 31 to identify the Chinese word having the meaning of "thanks".

[0105] The display control unit 24 displays the Chinese character "xie xie" 352, which means "thanks", without displaying the operation guide 500.

[0106] Therefore, the user does not need to perform the selection operation every time, and the character string can be displayed in the conversion target language without selecting it. Even if the conversion target language is fixed to Chinese, the display control unit 24 may display the operation guide 500. In this case, words including the Chinese character "xie xie" and the results of predictive conversion are displayed in the same way as when the handwritten data is Japanese.

[0107] In addition, to reset the fixed conversion target language, for example, a "Reset" command is provided as an operation command, and the user can reset the fixed conversion target language.

[0108] Fig. 12 is a flow chart for explaining the conversion example of Fig. 11. Note that Fig. 12 will mainly explain the differences from Fig. 7. First, steps S11 to S15 may be similar to steps S1 to S5 in Fig. 7.

[0109] Next, since "Ni hao" selected by the user is in Chinese, the conversion processing unit 28 fixes the conversion target language to Chinese (S16).

[0110] The drawing data generating unit 22 displays the handwritten data "Thank you" based on the coordinates detected by the contact position detecting unit 21 (S17).

[0111] When the user removes the pen 2500 from the display, the character recognition unit 23 starts character recognition. The character recognition unit 23 generates "Thank you" as the direct recognition result (S18).

[0112] Next, the conversion processing unit 28 searches for predefined control data for "thanks" to obtain Textmeaning="thanks" (S19).

[0113] Next, the conversion processing unit 28 refers to the synonym dictionary 31 to identify a character string in Chinese (the conversion target language is fixed) having the meaning of "thanks" (S20).

[0114] The display control unit 24 displays a Chinese character string meaning "thanks" without displaying the operation guide 500 (S21).

[0115] The data recording unit 25 stores input data for a Chinese character string having the meaning of "thanks" in the input data storage unit 33. The attributes of the input data are Text="Thanks", meaning="thanks" and Language="Chinese".

[0116] In this way, in case 2a, the target language is fixed, allowing users to communicate with each other efficiently.

[0117] <<Case 2b>> In case 2a, the display device 2 may receive from the user whether or not to fix the target language. For example, the display device 2 displays the handwritten data of "Hello" in Japanese as "Ni hao" in Chinese, and then displays the handwritten data of "Thank you" in Japanese as "Xie xie" a second time. The display device 2 detects that the same language has been converted twice in succession, and displays a dialog such as "Do you want to fix the translation target (output language) to Chinese?". The operation receiving unit 27 receives Yes or No from the user to fix the target language.

[0118] Note that the display device 2 may display the dialog when the user selects the Chinese word "Ni hao." In other words, the display device 2 may display the dialog when the user selects a character string in another language once. In this manner, the display device 2 may display the dialog when the user selects a character string in another language a certain number of times or more (including once) in succession.

[0119] 13 is a diagram showing an example of handwritten data and its conversion in case 2b. (1) to (3) are the same as in case 1a.

[0120] (4) The user handwrites "Thank you" in Japanese. The drawing data generation unit 22 displays the handwritten data based on the coordinates detected by the contact position detection unit 21. When the user removes the pen 2500 from the display, the character recognition unit 23 starts character recognition. The character recognition unit 23 generates "Thank you" as the direct recognition result (corresponding to the handwriting recognition character string candidate 506).

[0121] Next, the conversion processing unit 28 searches for predefined control data for "thanks" to obtain Textmeaning="thanks". The conversion processing unit 28 refers to the synonym dictionary 31 to identify character strings in other languages ​​that have the meaning of "thanks".

[0122] The display control unit 24 displays the operation guide 500, and displays a character string in another language that is synonymous with the Japanese phrase "thank you."

[0123] (5) When the display control unit 24 receives the selection of the character string candidate 539 from the user, the display control unit 24 displays the character string in another language selected by the user.

[0124] (6) Because Chinese was selected twice in succession in (2) and (4), the display control unit 24 displays a dialog 410 inquiring whether or not to fix the conversion target language to Chinese. It is assumed that the user selects Yes. The conversion processing unit 28 fixes the conversion target language to Chinese.

[0125] (7) The user handwrites the Japanese word “Conference.” The drawing data generating unit 22 displays the handwritten data based on the coordinates detected by the contact position detecting unit 21.

[0126] (8) Since the conversion target language is fixed to Chinese, the conversion processing unit 28 converts "Conference" into Chinese and displays it. Note that the display device 2 may display the operation guide 500 including the Chinese character string candidate 539.

[0127] Therefore, the display device 2 can fix the destination language with the consent of the user.

[0128] In case 2b, the user can also reset the fixed destination language.

[0129] Fig. 14 is a flow chart for explaining the conversion example of Fig. 12. Note that Fig. 14 will mainly explain the differences from Fig. 12. First, steps S31 to S35 may be similar to steps S11 to S15 in Fig. 12.

[0130] Next, the drawing data generating unit 22 displays the handwritten data "Thank you" based on the coordinates detected by the contact position detecting unit 21 (S36).

[0131] When the user removes the pen 2500 from the display and stops handwriting, the character recognition unit 23 starts character recognition. The character recognition unit 23 generates "Thank you" as the direct recognition result (S37).

[0132] Next, the conversion processing unit 28 searches for predefined control data for "thanks" to obtain Textmeaning="thanks" (S38).

[0133] Next, the conversion processing unit 28 refers to the synonym dictionary 31 to identify a character string in a language other than Japanese that has the meaning of "thanks" (S39).

[0134] The display control unit 24 displays the operation guide 500, and displays "Thank you" as the recognition result, as well as a plurality of character strings in other languages ​​having the same meaning as the Japanese word (S40).

[0135] When accepting the selection of character string candidate 539 from the user, conversion processing unit 28 determines whether or not the character string has been converted into the same language twice consecutively (S41).

[0136] If the determination in step S42 is Yes, the display control unit 24 displays the dialogue 410 (S42).

[0137] If Yes is pressed in the dialogue 410, the conversion processing unit 28 fixes the conversion target language to Chinese (43).

[0138] Note that, not limited to two consecutive conversions, but also one or more conversions into the conversion target language may cause the display device 2 to display the dialog 410 and fix the conversion target language with the user's consent.

[0139] <<Case 2c>> The user may select the language by handwriting a character string related to the target language. Such processing can be realized by the operation command definition data.

[0140] FIG. 15 is a diagram showing handwritten data and an example of its conversion in case 2c.

[0141] (1) The user handwrites "Mexico" in Japanese. In this case, it is known that the person is from Mexico. The drawing data generating unit 22 displays the handwritten data based on the coordinates detected by the contact position detecting unit 21.

[0142] (2) When the user removes the pen 2500 from the display, the character recognition unit 23 starts character recognition. The character recognition unit 23 generates “Mexico” as a direct recognition result (corresponding to the handwritten recognition character string candidate 506).

[0143] Next, the command detection unit 29 searches for predefined control data for "Mexico" to obtain the command names "Translate to Spanish" and "Translate to English."

[0144] The display control unit 24 displays an operation guide 500, and displays Japanese character string candidates and operation command candidates 510. Here, the user has selected "Translate into Spanish." The conversion processing unit 28 executes the operation command and sets the conversion target language to Spanish.

[0145] (3) The user handwrites “Hello” in Japanese. The drawing data generating unit 22 displays the handwritten data based on the coordinates detected by the contact position detecting unit 21.

[0146] (4) When the user removes the pen 2500 from the display, the character recognition unit 23 starts character recognition. The character recognition unit 23 generates "Hello" as the direct recognition result. Since the conversion destination language of the conversion processing unit 28 is fixed to Spanish, it converts "Hello" into Spanish. The conversion method may be the same as that described above.

[0147] In addition, in (4), the display device 2 may display the operation guide 500. In this case, the operation guide 500 displays "Hola", which is "Hello" in Spanish, and a Spanish character string candidate 539.

[0148] FIG. 16 is a flow chart illustrating the conversion example of FIG.

[0149] The drawing data generating unit 22 displays the handwritten data "Mexico" based on the coordinates detected by the contact position detecting unit 21 (S51).

[0150] When the user removes the pen 2500 from the display, the character recognition unit 23 starts character recognition. The character recognition unit 23 generates "Mexico" as the direct recognition result (S52).

[0151] Next, the conversion processing unit 28 searches the operation command definition data for "Mexico" to obtain the command names "Translate to Spanish" and "Translate to English" (S53).

[0152] The display control unit 24 displays the operation guide 500, and displays Japanese character string candidates and operation command candidates 510 (S54).

[0153] If the user selects "Translate into Spanish", the conversion processing unit 28 sets the conversion target language to Spanish by executing an operation command (S55).

[0154] The user handwrites "Hello." The drawing data generating unit 22 displays the handwritten data "Hello" based on the coordinates detected by the contact position detecting unit 21 (S56).

[0155] When the user removes the pen 2500 from the display, the character recognition unit 23 starts character recognition. The character recognition unit 23 generates "Hello" as the direct recognition result. The conversion processing unit 28 obtains Textmeaning="hello" by searching the predefined control data for "Hello". Since the conversion destination language is set to Spanish, the conversion processing unit 28 refers to the synonym dictionary 31 and identifies the Spanish character string "Hola" which has the meaning of "hello". The display control unit 24 displays the Spanish character string "Hola" without displaying the operation guide 500 (S57).

[0156] In this way, users can set the target language by handwriting the strings related to the target language. Even if a user only knows the country of origin of the other person, they can set the appropriate target language.

[0157] <<Case 2d>> The method for setting the destination language can be selected by the user from an icon button or by voice recognition. An icon button is a display part that is displayed with illustrations and characters and accepts selections. Such display parts are also called soft keys.

[0158] FIG. 17 shows an example of an icon button for setting a destination language.

[0159] (1) Three icon buttons 311 to 313 are displayed in Fig. 17. The icon buttons 311 to 313 are icon button 311 for the user to set the conversion target language to English, icon button 312 for setting the conversion target language to Japanese, and icon button 313 for setting the conversion target language to Chinese.

[0160] (2) When the user presses the icon button 313 for setting the conversion target language to Chinese, the conversion processing unit 28 sets the conversion target language to Chinese.

[0161] (3) When the user writes, for example, "Hello," by hand, the conversion processing unit 28 converts it into Chinese and displays it. The display device 2 may display the operation guide 500. In this case, the operation guide 500 displays "Ni hao," which is "Hello" in Chinese.

[0162] In this way, users can set the destination language using the icon button.

[0163] Fig. 18 is a flow chart for explaining the conversion example of Fig. 17. It is assumed that the icon button is already displayed in Fig. 18. The icon button may be always displayed or may be displayed by a user operation.

[0164] The operation reception unit 27 receives the setting of the conversion destination language by using an icon button (S61). Here, the operation reception unit 27 receives Chinese.

[0165] The drawing data generating unit 22 displays the handwritten data "Hello" based on the coordinates detected by the contact position detecting unit 21 (S62).

[0166] When the user removes the pen 2500 from the display, the character recognition unit 23 starts character recognition. The character recognition unit 23 generates "Hello" as the direct recognition result (S63).

[0167] Next, the conversion processing unit 28 searches for predefined control data for "hello" and obtains Textmeaning="hello" (S64).

[0168] Next, since the conversion target language is Chinese, conversion processing unit 28 refers to synonym dictionary 31 to identify the Chinese character string having the meaning of "hello" (S65).

[0169] The display control unit 24 displays the character string in Chinese converted from "Hello" (S66). The display control unit 24 may or may not display the operation guide 500.

[0170] <Major Effects> As described above, the display device 2 of this embodiment can fix the conversion target language, and therefore facilitates communication when the language used by the other party to whom information is to be communicated can be roughly predicted or specified. EXAMPLES

[0171] In this embodiment, a case that can be commonly applied to Case 1 and Case 2 in Embodiments 1 and 2 will be described. For convenience, this case will be referred to as "Case 1 and 2."

[0172] In this embodiment, it is assumed that the functional block diagram of FIG. 10 described in the second embodiment can be used.

[0173] <<Case 1 & 2a>> The display device 2 may display both the source language and the destination language so that the user writing the information and the person to whom the information is to be communicated can understand the meaning at a glance.

[0174] Fig. 19 shows an example of displaying both the source language and the destination language. It is assumed that the destination language is fixed to Chinese. In Fig. 19, "Hello" handwritten by the user is converted into a character string, "Hello" 421, and the character string "Nihao" 422, which is the Chinese version of "Hello", are displayed. Such control can be achieved by the display control unit 24 displaying both the source language and the destination language.

[0175] When the display device 2 displays a plurality of character strings, the operation guide 500 is not displayed, but may be displayed. In this case, the display control unit 24 may display the Japanese and Chinese operation guides 500 separately, or may accept the selection of two character strings from one operation guide 500.

[0176] The processing procedure of the display device 2 may be similar to that of FIGS. 16 and 18, except that there are a plurality of conversion target languages.

[0177] Furthermore, the number of target languages ​​is not limited to one, but may be two or more. In this case, multiple languages ​​are defined as target languages ​​in the operation command definition data. This enables not only "one language" to "one language" conversion, but also "one language" to "multiple languages" conversion.

[0178] [Table 5] Table 5 shows an example of operation command definition data that allows conversion between "one language" and "multiple languages." The recognition character strings are "English and Chinese," "English and Chinese," and "America and China." When these are handwritten, "Translate into English and Chinese" is displayed in operation command candidates 510. When the user selects "Translate into English and Chinese," the conversion destination languages ​​are set to two, English and Chinese.

[0179] FIG. 20 is a diagram for explaining an example of conversion into English and Chinese as two examples of conversion destination languages. (1) The user handwrites “English and Chinese” in Japanese. The drawing data generating unit 22 displays the handwritten data based on the coordinates detected by the contact position detecting unit 21.

[0180] When the user removes the pen 2500 from the display, the character recognition unit 23 starts character recognition. The character recognition unit 23 generates "English and Chinese" as the direct recognition result. The command detection unit 29 searches the operation command definition data for "English and Chinese" and detects the operation command "Translate into English and Chinese". The display control unit 24 displays Japanese character string candidates and candidates for this operation command (an operation command having a pair of two conversion destination languages). When the user selects the operation command "Translate into English and Chinese", the conversion processing unit 28 executes the process "Translate into English and Chinese" and sets the conversion destination languages ​​to English and Chinese.

[0181] (2) The user handwrites “Hello” in Japanese. The drawing data generating unit 22 displays the handwritten data based on the coordinates detected by the contact position detecting unit 21.

[0182] (3) When the user removes the pen 2500 from the display, the character recognition unit 23 starts character recognition. The character recognition unit 23 generates "Hello" as the direct recognition result. The conversion processing unit 28 converts "Hello" into the conversion destination languages, English and Chinese. The display control unit 24 displays the Japanese character string 423, the English character string 424, and the Chinese character string 425.

[0183] The processing procedure of the display device 2 may be the same as that of FIG. 16, except that there are a plurality of conversion target languages.

[0184] <<Case 1·2b>> In cases where there are many conversion candidates in one language, the operation guide 500 may be displayed in a mixture of multiple languages ​​with multiple candidates per language.

[0185] FIG. 21 is a diagram for explaining an example of conversion into English and Chinese.

[0186] (1) The user handwrites “Hello” in Japanese. The drawing data generating unit 22 displays the handwritten data based on the coordinates detected by the contact position detecting unit 21.

[0187] (2) When the user removes the pen 2500 from the display, the character recognition unit 23 starts character recognition. The character recognition unit 23 generates "Hello" as the direct recognition result (corresponding to the handwriting recognition character string candidate 506).

[0188] Next, the conversion processing unit 28 displays the operation command "translate" because "Hello" is registered in the predefined control data or according to the user's setting. The operation command "translate" causes the display device 2 to execute the process of "display list of conversion destination languages".

[0189] As described above, the display control unit 24 displays the operation guide 500 in response to the user operations (1) and (2), and displays the operation command candidate 510 of “translate” and the Japanese character string candidate 539. The content of this operation command is different from that in FIG. 8 of the first embodiment.

[0190] In FIG. 21, character string candidates 539 other than operation commands are displayed as options, so the user can select a character string even if he or she wishes to input Japanese characters rather than convert them.

[0191] (3) When the user selects "Translate" with the pen 2500, the display control unit 24 displays a list 354 of multiple language and multiple character strings on the operation guide 500 based on the process "multiple language conversion". The conversion processing unit 28 searches for predefined control data for "hello" to obtain Textmeaning="hello". Next, the conversion processing unit 28 refers to the synonym dictionary 31 to identify character strings in other languages ​​that have the meaning of "hello". The conversion processing unit 28 refers to a general-purpose thesaurus to obtain synonyms of the Chinese "Ni hao", such as "Hi", which is a synonym of the English word "hello".

[0192] (4) When the user selects "Hi" with the pen 2500, the display control unit 24 displays "Hi."

[0193] In this way, since the display device 2 displays synonyms in multiple languages, the character string that the user intended to select is likely to be displayed. Unlike Fig. 8, the display device 2 does not need to display the conversion target language.

[0194] Fig. 22 is a flow chart for explaining the conversion example of Fig. 21. Note that in the explanation of Fig. 22, differences from Fig. 9 will be mainly explained. Steps S81 to S84 may be the same as S1-1 to S4-4 in Fig. 9.

[0195] The operation reception unit 27 receives the operation command "translate" from the operation guide, and the conversion processing unit 28 executes processing of this operation command (S85). The processing of the operation command is "multiple language conversion".

[0196] As a result of executing the "multiple language conversion" process, the conversion processing unit 28 identifies character strings in other languages ​​that have the same meaning as Textmeaning="hello" from the synonym dictionary 31. The conversion processing unit 28 refers to a general-purpose thesaurus to acquire synonyms such as "Hi" for the English word "hello" and synonyms for the Chinese word "Ni hao."

[0197] The display control unit 24 displays the acquired synonyms in each language on the operation guide 500 (S86).

[0198] The operation acceptance unit 27 accepts the selection of the character string, and the display control unit 24 displays the selected character string (S87).

[0199] In this way, since the display device 2 displays synonyms in multiple languages, the character string that the user intended to select is likely to be displayed. Unlike Fig. 9, the display device 2 does not need to display the conversion target language.

[0200] <User interface example> We will explain some design examples of operation guides.

[0201] FIG. 23 is a diagram for explaining a recommendation window in which a plurality of operation guides are added and displayed from the starting point.

[0202] 23, "Translate" is selected in (2) on the operation guide 500. The display control unit 24 leaves the user's selection in the operation guide 500 and separately displays the recommendation window 330. In the recommendation window 330, a character string in another language that is synonymous with "hello" is displayed.

[0203] Such a user interface allows the user to see the history of their selections.

[0204] Fig. 24 shows an operation guide in which the display device 2 displays selectable candidates 530 while moving them from left to right within the operation guide. In Fig. 24, the operation command "Translate into English and Chinese" does not fit into one square. The display control unit 24 gradually moves "Translate into English and Chinese" from left to right so that the user can read the whole thing. By doing this, it is possible to improve space efficiency when the operation command or the character string that appears is long.

[0205] <Major Effects> As described above, the display device 2 of this embodiment can simultaneously display a plurality of conversion destination languages ​​and provide an easy-to-use user interface. EXAMPLES

[0206] In the following embodiment, another configuration example of the display device 2 will be described.

[0207] <<Another configuration example 1 of the display device>> Although the display device 2 of the present embodiment has been described as having a large touch panel, the display device 2 is not limited to having a touch panel.

[0208] Fig. 25 is a diagram showing another example of the configuration of the display device 2. In Fig. 25, a projector 411 is installed on the upper side of a normal whiteboard 413. This projector 411 corresponds to the display device 2. The normal whiteboard 413 is not a flat panel display integrated with a touch panel, but a whiteboard on which a user directly writes by hand with a marker. The whiteboard may be a blackboard, and may be any flat surface large enough to project an image.

[0209] The projector 411 has an ultra-short focal length optical system and can project an image with little distortion from a distance of about 10 cm onto the whiteboard 413. This image may be transmitted from the PC 400-1 connected wirelessly or via a wire, or may be stored in the projector 411.

[0210] A user writes handwriting on the whiteboard 413 using a dedicated electronic pen 2501. The electronic pen 2501 has a light-emitting part, for example at the tip, that is switched on and emits light when the user presses it against the whiteboard 413 to write. The wavelength of the light is near-infrared or infrared, so it is invisible to the user's eyes. The projector 411 has a camera that captures the light-emitting part, analyzes the image, and identifies the direction of the electronic pen 2501. The electronic pen 2501 also emits sound waves along with the light emission, and the projector 411 calculates the distance based on the arrival time of the sound waves. The projector 411 can identify the position of the electronic pen 2501 based on the direction and distance. Handwritten data is drawn (projected) at the position of the electronic pen 2501.

[0211] Since the projector 411 projects the menu 430, when the user presses a button with the electronic pen 2501, the projector 411 identifies the position of the electronic pen 2501 and the pressed button by the ON signal of the switch. For example, when the save button 431 is pressed, the handwritten data (coordinate point sequence) handwritten by the user is saved by the projector 411. The projector 411 saves the handwritten information in a predetermined server 412 or USB memory 2600 or the like. The handwritten information is saved for each page. Since the information is saved as coordinates rather than as image data, the user can re-edit it. However, in this embodiment, the menu 430 does not need to be displayed because the operation command can be called up by handwriting. EXAMPLES

[0212] <<Another configuration example 2 of the display device>> 26 is a diagram showing another example of the configuration of the display device 2. In the example of FIG.

[0213] The terminal device 600 is connected by wire to the image projection device 700A and the pen operation detection device 810. The image projection device 700A projects image data input by the terminal device 600 onto the screen 800.

[0214] The pen operation detection device 810 communicates with the electronic pen 820 and detects the operation of the electronic pen 820 in the vicinity of the screen 800. Specifically, the electronic pen 820 detects coordinate information indicating a point pointed to by the electronic pen 820 on the screen 800 (the detection method may be the same as that in FIG. 25 ), and transmits the coordinate information to the terminal device 600.

[0215] The terminal device 600 generates image data of handwritten data input by the electronic pen 820 based on the coordinate information received from the pen operation detection device 810, and causes the image projection device 700A to draw an image of the handwritten data on the screen 800.

[0216] Furthermore, the terminal device 600 generates superimposed image data indicating a superimposed image obtained by combining the background image projected by the image projection device 700A and the image of the handwritten data input by the electronic pen 820. EXAMPLES

[0217] <<Another configuration example 3 of the display device>> 27 is a diagram showing an example of the configuration of the display device 2. In the example of FIG. 27, the display device 2 includes a terminal device 600, a display 800A, and a pen motion detection device 810.

[0218] The pen action detection device 810 is disposed near the display 800A, detects coordinate information indicating a point indicated by the electronic pen 820A on the display 800A (the detection method may be the same as that in FIG. 25), and transmits the coordinate information to the terminal device 600. In the example of FIG. 27, the electronic pen 820A may be charged by the terminal device 600 via a USB connector.

[0219] Based on the coordinate information received from the pen operation detection device 810, the terminal device 600 generates image data of the image of the handwritten data input by the electronic pen 820A, and displays it on the display 800A. EXAMPLES

[0220] <<Another Configuration Example 4 of the Display Device>> Fig. 28 is a diagram showing a configuration example of the display device 2. In the example of Fig. 28, the display device 2 includes a terminal device 600 and an image projection device 700A.

[0221] The terminal device 600 performs wireless communication (Bluetooth (registered trademark) or the like) with the electronic pen 820B to receive coordinate information of a point indicated by the electronic pen 820B on the screen 800. The coordinate information may be obtained by the electronic pen 820B reading minute position information formed on the screen 800, or the coordinate information may be received from the screen 800.

[0222] Then, based on the received coordinate information, the terminal device 600 generates image data of an image of the handwritten data input by the electronic pen 820B, and causes the image projection device 700A to project the image of the handwritten data.

[0223] Furthermore, the terminal device 600 generates superimposed image data indicating a superimposed image obtained by combining the background image projected by the image projection device 700A and the image of the handwritten data input by the electronic pen 820.

[0224] As described above, each of the above-described embodiments can be applied to various system configurations.

[0225] <Additional Notes> A supplementary explanation of the problem is provided. The conventional technology has a problem in that a user needs to set a conversion destination language before inputting handwritten data. The conversion destination language refers to another language when a character string in one language is converted to another language, such as conversion from Japanese to English. When a display device can convert to a plurality of conversion destination languages, the user must set the desired conversion destination language before inputting the handwritten data in order to convert the handwritten data to a desired language. In view of the above problem, the present invention also aims to provide a display device that can convert handwritten data into another language and display it without the user having to set a conversion destination language before inputting the handwritten data.

[0226] [Claim A] A receiving means for receiving input of handwritten data; a display control means for displaying a plurality of languages ​​after the handwritten data is accepted by the accepting means; a selection means for accepting a selection of a language from the plurality of language types displayed by the display control means, The display device according to claim 1, wherein the display control means displays the character string converted from the handwritten data in the language selected by the selection means.

[0227] [Claim B] character recognition means for converting the handwritten data received by the receiving means into a character string; the display control means simultaneously displays an operation command for accepting conversion into another language associated with the character string converted by the character recognition means, and the character string converted by the character recognition means from the handwritten data accepted by the accepting means; 8. The display device according to claim 7, wherein, when the selection of the operation command is accepted, the display control means displays a plurality of languages.

[0228] [Claim C] the display control means displays a plurality of character strings into which the handwritten data has been converted in the language type selected by the selection means; 8. The display device according to claim B, wherein the selection means displays the character string selected.

[0229] [Claim D] When the selection of the operation command is accepted, the display control means displays character strings obtained by converting the handwritten data accepted by the accepting means into a plurality of languages; 8. The display device according to claim C, wherein the display control means displays the character string selected by the selection means.

[0230] [Claim E] When the selection of the operation command is accepted, the display control means displays a character string converted from the handwritten data accepted by the accepting means into a plurality of languages, as well as a plurality of character strings that are synonyms for one language; 10. The display device according to claim 9, wherein the display control means displays the character string selected by the selection means.

[0231] [Claim F] The display device according to claim E, wherein, when a selection of the operation command is accepted, the display control means does not erase the operation command, but instead displays a character string in which the handwritten data accepted by the accepting means is converted into a plurality of languages.

[0232] [Claim G] The display device according to claim F, wherein the display control means displays, by moving within a square, an operation command for accepting a conversion into another language associated with the character string converted by the character recognition means, and the character string converted by the character recognition means from the handwritten data accepted by the accepting means.

[0233] <Other application examples> The above describes the best mode for carrying out the present invention using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0234] The character string is stored as character codes, and the handwritten data is stored as coordinate point data in the display device 2. The data can also be stored in various storage media or in a storage device on a network, and later downloaded from the display device 2 for reuse. The display device 2 to be reused may be any display device, or may be a general information processing device. Therefore, the user can reproduce the handwritten content on a different display device 2 and continue a meeting, etc.

[0235] For example, in the present embodiment, an electronic whiteboard has been described as an example, but the electronic whiteboard may be called an electronic whiteboard, an electronic information board, or the like. In addition, the present embodiment can be suitably applied to any information processing device having a touch panel. The information processing device equipped with a touch panel may be, for example, a PJ (Projector), an output device such as digital signage, a HUD (Head Up Display) device, an industrial machine, an imaging device, a sound collecting device, a medical device, a network home appliance, a notebook PC (Personal Computer), a mobile phone, a smartphone, a tablet terminal, a game machine, a PDA (Personal Digital Assistant), a digital camera, a wearable PC, or a desktop PC.

[0236] In this embodiment, the coordinates of the pen are detected by detecting the coordinates of the pen tip with a touch panel, but the coordinates of the pen tip may be detected by ultrasonic waves. The pen emits light and also emits ultrasonic waves, and the display device 2 calculates the distance based on the arrival time of the ultrasonic waves. The position of the pen can be identified based on the direction and distance. The projector draws (projects) the trajectory of the pen as stroke data.

[0237] Moreover, the configuration example in FIG. 4 and the like is divided according to main functions in order to facilitate understanding of the processing by the display device 2. The present invention is not limited by the manner in which the processing units are divided or the names of the processing units. The processing by the display device 2 can be divided into more processing units according to the processing contents. Also, it can be divided so that one processing unit includes more processes.

[0238] Furthermore, part of the processing performed by the display device 2 may be performed by a server connected to the display device 2 via a network.

[0239] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above.

[0240] The contact position detection unit 21 is an example of a receiving means. The display control unit 24 is an example of a display control means. The operation receiving unit 27 is an example of a selecting means. The character recognition unit 23 is an example of a character recognition means. The conversion processing unit 28 is an example of a converting means. [Explanation of symbols]

[0241] 2 Display device [Prior art documents] [Patent documents]

[0242] [Patent Document 1] JP 2003-162527 A

Claims

1. A receiving means for receiving input of handwritten data; a display control means for converting the handwritten data received by the receiving means into a plurality of languages ​​and displaying a plurality of character strings; a selection means for accepting a selection of a character string from the plurality of character strings displayed by the display control means, the display control means converts the handwritten data into the character string selected by the selection means and displays the character string; setting the language of the character string selected by the selection means to a conversion destination language; a conversion means for converting the handwritten data accepted by the acceptance means into the conversion target language after the selection means accepts the selection of the character string, the display control means displays the character string converted into the conversion target language by the conversion means. Display device.

2. When the selection means receives the selection of a character string in the same language a certain number of times in succession, the display control means displays a message inquiring whether or not to set a conversion target language for the character string; 2 . The display device according to claim 1 , wherein when the selection means receives an instruction to set a conversion target language for a character string, the selection means sets the language of the character string selected and received as the conversion target language.

3. character recognition means for converting the handwritten data received by the receiving means into a character string; The display control means displays a list of one or more conversion destination languages ​​associated with the character string converted by the character recognition means, 3. The display device according to claim 2, wherein the conversion means converts the handwritten data accepted by the acceptance means into the conversion target language selected by the selection means from a list.

4. the display control means displays, by a soft key, a conversion destination language of the handwritten data accepted by the accepting means; 3. The display device according to claim 2, wherein the conversion means converts the handwritten data accepted by the acceptance means into the conversion destination language selected by the selection means using a soft key.

5. the display control means converts the handwritten data received by the receiving means into a character string by a character recognition means; 5. The display device according to claim 2, wherein the display device simultaneously displays a character string obtained by converting the handwritten data accepted by the accepting means into the conversion target language by the converting means.

6. The display control means displays a pair of two conversion target languages ​​associated with the character string converted by the character recognition means, the conversion means converts the handwritten data accepted by the acceptance means into the two conversion destination languages ​​selected by the selection means; The display device according to claim 5, wherein the display control means simultaneously displays a character string converted by the character recognition means from the handwritten data accepted by the accepting means, and two character strings converted by the conversion means from the handwritten data accepted by the accepting means into the target language.

7. A step of accepting input of handwritten data by an accepting means; a step of displaying a plurality of character strings obtained by converting the handwritten data accepted by the accepting means into a plurality of languages ​​by a display control means; A selection means receives a selection of a character string from the plurality of character strings displayed by the display control means; a step of the display control means converting the handwritten data into the character string selected by the selection means and displaying the character string; a step of setting a language of the character string selected by the selection means as a conversion target language, and after the selection means accepts the selection of the character string, a conversion means converts the handwritten data accepted by the acceptance means into the conversion target language; a step of the display control means displaying the character string converted into the conversion target language by the conversion means; A display method comprising the steps of:

8. The display device, A receiving means for receiving input of handwritten data; a display control means for displaying a plurality of character strings obtained by converting the handwritten data accepted by the accepting means into a plurality of languages; a selection unit that receives a selection of a character string from the plurality of character strings displayed by the display control unit; the display control means converts the handwritten data into the character string selected by the selection means and displays the character string; The display device, setting the language of the character string selected by the selection means to a conversion destination language, and after the selection means accepts the selection of the character string, causing the selection means to function as a conversion means for converting the handwritten data accepted by the acceptance means into the conversion destination language; the display control means displays the character string converted into the conversion target language by the conversion means. program.

Images