Display device, display method, and program

Abstract

To provide a display device for displaying a plurality of letter string candidates converted in a conversion destination language.SOLUTION: The display device includes: reception means for receiving the input of handwriting data by an electronic pen; letter recognition means for converting the handwriting data to a plurality of letter strings; display control means for displaying the letter strings converted by the letter recognition means; and storage means for storing the conversion destination language in relation to the identification information of the electronic pen. The display control means converts a letter string selected from the letter strings by the electronic pen, into a plurality of letter strings in the conversion destination language related to the identification information of the electronic pen, and displays the letter strings.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a display device, a display method, and a program. [Background technology]

[0002] There is a known display device that uses handwriting recognition technology to convert handwritten data into characters and display them on a display. Display devices equipped with relatively large touch panels are placed in conference rooms and public facilities and are used by multiple users as electronic whiteboards. Display devices are also sometimes used as communication tools by writing.

[0003] Also, a technology for translating handwritten data using handwriting recognition technology has been devised (see, for example, Patent Document 1). Patent Document 1 discloses a device that displays second language information on an image display unit so that the information is upright as seen by a user of a second pen device, based on the positional relationship between the image display unit and a user of a first pen device and a user of a second pen device. Summary of the Invention [Problem to be solved by the invention]

[0004] In communication between speakers of different languages, it may be necessary for the speaker to proceed with the communication while checking whether the conversion result on the display device is being properly conveyed to the other person. In such cases, there has been no proposal in the past to select a character string from multiple character string candidates, convert the selected character string into multiple character string candidates converted into a conversion target language (different language), and display them.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide a display device that displays a plurality of character string candidates converted into a destination language. [Means for solving the problem]

[0006] In view of the above problem, the present invention provides a receiving means for receiving handwritten data input by an electronic pen, a character recognition means for converting the handwritten data into a plurality of character strings, a display control means for displaying the plurality of character strings converted by the character recognition means, and a conversion destination language in association with identification information of the electronic pen. and display language and a storage means for storing therein: the display control means converts the character string selected by the electronic pen from among the plurality of character strings into a plurality of character strings in the conversion target language associated with identification information of the electronic pen, and displays the converted character string; The character recognition means converts the handwritten data into the display language associated with identification information of the electronic pen used to input the handwritten data. . Effect of the Invention

[0007] It is possible to provide a display device that displays a plurality of character string candidates converted into a destination language. [Brief description of the drawings]

[0008] [Figure 1] 11A and 11B are diagrams illustrating an example of conversion of a character string displayed on a display device. [Diagram 2] FIG. 1 is a diagram illustrating an overall configuration of an example of a display device. [Diagram 3] FIG. 2 is a diagram illustrating a hardware configuration of an example of a display device. [Figure 4] FIG. 2 is a functional block diagram of an example for explaining functions of the display device by dividing them into blocks. [Diagram 5] FIG. 13 is a diagram showing an operation guide and an example of selectable candidates displayed by the operation guide. [Figure 6] FIG. 13 is a diagram illustrating a display direction of an example of a character string. [Figure 7] FIG. 10 is a diagram illustrating a correspondence between a user's handwriting direction and a display direction. [Figure 8] FIG. 11 is a diagram illustrating an example of a method for inputting angle information. [Figure 9] 11A and 11B are diagrams illustrating an example of a method in which a user associates a display direction with a conversion target language. [Figure 10] 1 is an example of handwritten data handwritten by a user. [Figure 11]13 is a diagram showing an example of five character string candidates displayed by the character recognition unit through character recognition of "honjitsunogi." FIG. [Figure 12] FIG. 13 is a diagram showing an example of a conversion target character string converted into a conversion target language and displayed when a user selects an arbitrary character string candidate. [Figure 13] 11 is a diagram illustrating an example of reconversion of a conversion destination character string. [Figure 14] FIG. 13 is a diagram showing an example of five character string candidates displayed in response to the selection of a conversion destination character string "Today's proceedings." [Figure 15] 13 is a diagram showing an example of a converted character string in a converted language that is displayed when a user selects "Today's minutes" with a pen. [Figure 16] FIG. 13 is a diagram showing an example of character string candidates that are displayed when "Today's proceedings" is selected with a pen having a pen ID=2. [Figure 17] FIG. 12 is a diagram showing an example of conversion when a user of the pen with pen ID=3 in FIG. 11 selects a character string candidate. [Figure 18] 11 is a flowchart illustrating a process in which the display device converts a selected character string candidate into a conversion target language associated with the pen; FIG. [Figure 19] 11 is a diagram illustrating an example of rotation of a converted character string; [Figure 20] 11 is a diagram showing an example of a conversion destination character string and an operation guide displayed in a display direction corresponding to a pen. [Figure 21] 13A and 13B are diagrams showing operation guides when the display directions associated with the pen ID are 90° and 270°. [Figure 22] FIG. 13 is a diagram showing another configuration example of the display device. [Diagram 23] FIG. 13 is a diagram showing another configuration example of the display device. [Figure 24] FIG. 13 is a diagram showing another configuration example of the display device. [Diagram 25] FIG. 13 is a diagram showing another configuration example of the display device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A display device and a display method performed by the display device will be described below as an example of an embodiment of the present invention with reference to the drawings. EXAMPLES

[0010] Display devices are used in workplaces and other locations where people with different languages ​​coexist. In such situations, when a person from one country wants to convey information to a person from another country by handwriting, the characters shown on the display can be converted into a language that the person from the other country can understand, making it easier to communicate.

[0011] When these people communicate using a display device, if the display device displays a plurality of candidate character strings that can be converted into other languages, the user can easily select a character string that is suitable for communication.

[0012] In addition, when these people communicate using a display device, even if the handwritten data has been converted into another language by the display device, if the conversion is not properly conveyed, the conversion may be inappropriate and the person may want to select the conversion result again. Conventionally, it has been difficult to select the conversion result again unless the speaker re-enters the handwritten data.

[0013] <Outline of the operation of the display device> The display device of the present embodiment can display a plurality of character string candidates that can be converted into another language. In addition, the display device can convert a character string that has been converted into another language back into another character string in the other language.

[0014] 1 shows an example of character string conversion displayed by the display device of this embodiment. Two users are communicating by writing and talking on either side of the display device. The pen 2500 used by the Japanese-speaking user is set with information for writing in Japanese (display language described later is Japanese) and for converting Japanese to English (conversion language described later is English).

[0015] As shown in Fig. 1(a), the user handwrites "Honjitsunogi" with the pen 2500. When the user finishes writing, the display device performs character recognition of the handwritten data into Japanese because the pen 2500 is designed for writing in Japanese. The display device also converts "Honjitsunogi" into kana-kanji or performs predictive conversion, and displays candidate character string 539 within operation guide 500.

[0016] The display device displays a plurality of candidate character strings to be converted into other languages, so that the user can select the candidate character string that he / she considers appropriate for communication.

[0017] In FIG. 1(b), since the pen 2500 is for writing in Japanese, five character string candidates 539 are displayed in Japanese for "honjitsunogi." The user selects "today's agenda" with the pen 2500. The display device converts "today's agenda" into English, which is the conversion language set for this pen 2500.

[0018] In FIG. 1(c), "Today's proceedings" is converted into the converted character string 101, "Today's proceedings." At this point, the operation guide 500 is automatically deleted. If the user feels that the conversation does not flow well with "Today's proceedings," the user can try the conversion again.

[0019] In FIG. 1(d), the user hand-draws a frame 102 surrounding the converted character string 101, "Today's proceedings," with the pen 2500. As a result, the user selects "Today's proceedings." Since the pen 2500 is for writing in Japanese, the display device displays again in Japanese the character string candidate 539 that the operation guide 500 displayed before the converted character string 101 was displayed.

[0020] 1(e) shows an operation guide 500 displayed in response to the selection of the converted character string 101. In this way, the display device displays a plurality of character string candidates to be converted into another language, so that the user can select another character string candidate that is likely to facilitate better communication without having to rewrite the character string by hand.

[0021] The user selects "Today's minutes" with the pen 2500. The display device converts "Today's minutes" into English, which is the conversion language set for the pen 2500.

[0022] 1(f), "Today's minutes" is converted to "Today's minutes", a converted character string 101. The user can select this converted character string 101 again to display the operation guide 500.

[0023] In this way, the display device of this embodiment allows the user to select a character string to be converted into a destination language from a plurality of character string candidates to be converted into other languages, facilitating smooth communication. Furthermore, even if a character string has been converted into a destination language once, the user can select a plurality of character string candidates to be converted into another language again by selecting a destination character string 101. Therefore, the user can select another character string candidate and convert the handwritten data into the destination language without having to write the data by hand multiple times. Furthermore, the display device 2 can convert the selected character string candidate into the destination language set in the pen.

[0024] <Terminology> The input means may be any means capable of specifying coordinates on a touch panel and writing by hand, such as a pen, a human finger or hand, or a rod-shaped member.

[0025] A stroke refers to a series of operations in which a user presses an input means against a display, moves it continuously, and then releases it from the display. Stroke data is information displayed on the display based on a trajectory of coordinates input by the input means. Stroke data may be appropriately interpolated. Handwritten data is data having one or more stroke data. Handwritten input refers to handwritten data being input by a user.

[0026] An object is an object that is displayed on a display based on stroke data. Object means a target, but in this embodiment it means a display target. The character string converted from handwritten data through character recognition may include not only text data, but also data displayed based on user operations, such as fixed characters such as "Done," stamps displayed as marks, shapes such as circles and stars, and straight lines.

[0027] The confirmed data is data that has been converted into a character code (font) by character recognition of handwritten data and selected by the user, or handwritten data that has been confirmed not to be converted into a character code (font).

[0028] An operation command is a command prepared for operating a handwriting input device to instruct the execution of a specific process. In this embodiment, as an example, a case will be described in which a user instructs the display device 2 to correspond to a display direction and a converted character string 101. In addition, operation commands are also used in a character string editing system, a modification system, an input / output system, and the like.

[0029] A string is one or more characters that are handled by a computer. The actual entity of a string is a character code. Characters include numbers, alphabets, and symbols. Strings are also called text data.

[0030] Conversion means to change or replace. Converting a string of characters from one language to another is sometimes called translation.

[0031] The conversion target language is the other language into which the character string candidate is converted, and the display language is the language in which the character string candidate that has been recognized is displayed.

[0032] <Example of device configuration> The overall configuration of the display device 2 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing the overall configuration of the display device 2. Fig. 2(a) shows, as an example of the display device 2, a display device 2 used as a horizontally long electronic whiteboard hung on a wall.

[0033] 2(a), a display 220 as an example of the display device 2 is provided on the upper part of the display device 2. A user U can use a pen 2500 to handwrite (input or draw) characters or the like on the display 220.

[0034] FIG. 2(b) shows a display device 2 used as a vertically long electronic whiteboard suspended on a wall.

[0035] 2(c) shows the display device 2 placed flat on a desk 230. The display device 2 is about 1 cm thick, so there is no need to adjust the height of the desk even if it is placed flat on a regular desk. In addition, the user can easily move the display device 2.

[0036] The input method of the coordinates of the pen 2500 includes an electromagnetic induction method, an active electrostatic coupling method, etc. The pen 2500 may also have functions such as pen pressure detection, tilt detection, and a hover function (displaying the cursor before the pen touches the screen).

[0037] <Hardware configuration of the device> Next, the hardware configuration of the display device 2 will be described with reference to Fig. 3. As shown in the figure, the display device 2 has the configuration of an information processing device or a computer. Fig. 3 is an example of a hardware configuration diagram of the display device 2. As shown in Fig. 3, the display device 2 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and an SSD (Solid State Drive) 204.

[0038] Of these, the CPU 201 controls the overall operation of the display device 2. The ROM 202 stores the CPU 201 and programs used to drive the CPU 201, such as an IPL (Initial Program Loader). The RAM 203 is used as a work area for the CPU 201.

[0039] The SSD 204 stores various data such as an OS and programs for the display device 2. The programs may be application programs that run on an information processing device equipped with a general-purpose OS (Windows (registered trademark), Mac OS (registered trademark), Android (registered trademark), iOS (registered trademark), etc.).

[0040] The display device 2 also includes a display controller 213, a touch sensor controller 215, a touch sensor 216, a display 220, a power switch 227, a tilt sensor 217, a serial interface 218, a speaker 219, a microphone 221, a wireless communication device 222, an infrared I / F 223, a power control circuit 224, an AC adapter 225, and a battery 226.

[0041] The display controller 213 controls and manages the screen display in order to output an output image to the display 220, etc. The touch sensor 216 detects that the pen 2500, the user's hand, etc. (the pen and the user's hand serve as input means) have touched the display 220. The touch sensor 216 also receives a pen ID.

[0042] The touch sensor controller 215 controls the processing of the touch sensor 216. The touch sensor 216 inputs and detects coordinates. For example, in the case of an optical type, the input of coordinates and the detection of coordinates are performed by two light receiving / emitting devices installed at both ends of the upper side of the display 220 emitting a plurality of infrared rays in parallel to the display 220, and receiving the light reflected by a reflecting member provided around the display 220 and returning along the same optical path as the optical path of the light emitted by the light receiving element. The touch sensor 216 outputs position information of the infrared rays emitted by the two light receiving / emitting devices that are blocked by an object to the touch sensor controller 215, and the touch sensor controller 215 specifies the coordinate position that is the contact position of the object. The touch sensor controller 215 also has a communication unit 215a and can wirelessly communicate with the pen 2500. For example, when communication is performed according to a standard such as Bluetooth (registered trademark), a commercially available pen can be used. If one or more pens 2500 are registered in advance in the communication unit 215a, the user can communicate with the display device 2 without having to perform connection settings to allow the pen 2500 to communicate with the display device 2.

[0043] The power switch 227 is a switch for switching ON / OFF the power of the display device 2. The tilt sensor 217 is a sensor for detecting the tilt angle of the display device 2. It is mainly used to detect whether the display device 2 is being used in the installation state of Fig. 2(a), Fig. 2(b), or Fig. 2(c), and can automatically change the thickness of characters, etc. according to the installation state.

[0044] The serial interface 218 is an external communication interface such as a USB. The serial interface 218 is used for inputting information from the outside. The speaker 219 is used for outputting audio, and the microphone 221 is used for inputting audio. The wireless communication device 222 communicates with a terminal carried by the user, and relays a connection to the Internet, for example. The wireless communication device 222 communicates using Wi-Fi, Bluetooth (registered trademark), or the like, but any communication standard may be used. The wireless communication device 222 forms an access point, and when the user sets an SSID (Service Set Identifier) ​​and password obtained by the user in the terminal carried by the user, the user can connect to the access point.

[0045] It is preferable that the wireless communication device 222 is provided with two access points. a. Access point → Internet b. Access point → Internal network → Internet Access point a is for users outside the company, and users cannot access the company network but can use the Internet. Access point b is for users inside the company, and users can use the company network and the Internet.

[0046] The infrared I / F 223 detects the display device 2 arranged adjacently. The infrared I / F 223 can detect only the display device 2 arranged adjacently by utilizing the linearity of infrared rays. It is preferable that one infrared I / F 223 is provided on each side, and it can detect in which direction of the display device 2 another display device 2 is arranged. This expands the screen, and it is possible to display handwritten information that was previously handwritten on the adjacent display device 2 (handwritten information on another page, with the area of ​​one display 220 being one page) and the like.

[0047] The power supply control circuit 224 controls an AC adapter 225 and a battery 226 which are power sources for the display device 2. The AC adapter 225 converts AC common to commercial power sources into DC.

[0048] If the display 220 is so-called electronic paper, it consumes little or no power to maintain the image display, and therefore can be driven by a battery 226. This makes it possible to use the display device 2 for purposes such as digital signage even in places where it is difficult to connect to a power source, such as outdoors.

[0049] The display device 2 further includes a bus line 210. The bus line 210 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 201 shown in FIG.

[0050] The touch sensor 216 is not limited to an optical type, and may be a capacitive touch panel that identifies the contact position by detecting a change in capacitance, a resistive film touch panel that identifies the contact position by a voltage change between two opposing resistive films, or an electromagnetic induction touch panel that identifies the contact position by detecting electromagnetic induction caused by a contact object touching the display unit. The touch sensor 216 may be of a type that does not require an electronic pen to detect the presence or absence of a touch with the tip of the pen. In this case, a fingertip or a pen-shaped stick can be used to perform a touch operation. The pen 2500 does not need to be a long, thin pen type.

[0051] <About the function> Next, the functions of the display device 2 will be described with reference to Fig. 4. Fig. 4 is an example of a functional block diagram in which the functions of the display device 2 are divided into blocks and described. The display device 2 has a contact position detection unit 21, a drawing data generation unit 22, a character recognition unit 23, a display control unit 24, a data recording unit 25, a network communication unit 26, an operation reception unit 27, an operation command processing unit 28, a rotation processing unit 29, a language conversion unit 30, a pen communication unit 31, and an input direction detection unit 32. Each function of the display device 2 is a function or means realized by any of the components shown in Fig. 3 operating according to an instruction from the CPU 201 in accordance with a program expanded from the SSD 204 onto the RAM 203.

[0052] The contact position detection unit 21 detects the coordinates of the position where the pen 2500 touches the touch sensor 216. The drawing data generation unit 22 acquires the coordinates where the tip of the pen 2500 touches from the contact position detection unit 21. The contact position detection unit 21 generates stroke data by connecting the sequence of coordinate points by interpolating them.

[0053] The character recognition unit 23 performs character recognition processing on one or more stroke data (handwritten data) handwritten by the user according to the display language associated with the pen 2500 used by the user, and converts the data into a character code. In parallel with the user's pen operation, the character recognition unit 23 recognizes characters (not only Japanese but also multiple languages ​​such as English), numbers, symbols (%, $, &, etc.), figures (lines, circles, triangles, etc.), etc. Various algorithms have been devised for the recognition method, but details will be omitted since known technology can be used in this embodiment.

[0054] The display control unit 24 displays the handwritten data, character strings converted from the handwritten data, and an operation menu for user operation on the display. The data recording unit 25 stores the handwritten data handwritten on the display device 2, the converted character strings, the PC screen, files, and the like in the storage unit 40. The network communication unit 26 connects to a network such as a LAN and transmits and receives data to and from other devices via the network.

[0055] The operation command processing unit 28 detects whether the character string recognized by the character recognition unit 23 corresponds to an operation command, and if the user presses an operation command, executes the operation command.

[0056] The rotation processing unit 29 rotates the entire image or the character string in response to an operation command or a user operation. In this embodiment, it is possible to rotate only the character string, or to rotate the entire image including the character string.

[0057] The language conversion unit 30 converts the language of the character string candidate selected by the user into a conversion target language associated with the pen 2500 used by the user. In other words, the language conversion unit 30 performs translation.

[0058] The pen communication section 31 controls the communication unit 215a to receive the pen ID from the pen 2500. The pen 2500 transmits the pen ID at least once when the tip of the pen 2500 is pressed against the display. The display device 2 can identify the pen 2500 used for the handwritten data.

[0059] The input direction detection unit 32 detects the display direction facing the user who inputs the handwritten data based on the handwritten data. Alternatively, the input direction detection unit 32 detects the display direction facing the user who writes by hand using the input means, according to the handwritten data handwritten by a predetermined method.

[0060] The display device 2 also has a memory unit 40 constructed in the SSD 204 or RAM 203 shown in Figure 3, and the memory unit 40 has a conversion language memory unit 41, an operation command definition data memory unit 42, an input data memory unit 43, and several conversion dictionaries 44 constructed therein.

[0061] [Table 1] Table 1 shows the correspondence between the display direction, display language, and conversion destination language set in the conversion language storage unit 41. The display direction, display language, and conversion destination language are registered in association with the pen ID in the conversion language storage unit 41. For example, the display direction of the pen 2500 with pen ID=1 is 0°, the display language is Japanese, and the conversion destination language is English. The pen ID is identification information of the pen 2500. Each pen 2500 holds a pen ID in advance. The display direction is preset as, for example, 90°, 180°, or 270° counterclockwise, with the reference display direction being 0°. The display direction indicates the location of the user operating the pen 2500. When a character string is displayed in the display direction, the character string is displayed facing the user using the pen 2500. The display language is the language used when the character recognition unit 23 recognizes characters in the handwritten data. The display language is also the language of the character string candidates 539 and the like displayed on the operation guide 500. The conversion target language is a language to which the language conversion unit 30 converts the display language.

[0062] According to the display language and conversion destination language in Table 1, a user using pen 2500 with pen ID=1 (an example of a first electronic pen) is a Japanese speaker who can communicate with English speakers. A user using pen 2500 with pen ID=2 (an example of a second electronic pen) is an English speaker who can communicate with Japanese speakers. A user using pen 2500 with pen ID=3 (an example of a third electronic pen) is a Japanese speaker who can communicate with Chinese speakers.

[0063] [Table 2] Table 2 shows a schematic diagram of the operation command definition data stored in the operation command definition data storage unit 42. The operation command definition data defines an operation command that the user instructs the display device 2 in association with a recognized character string obtained by character recognition.

[0064] For example, if the character strings "Rotate," "90," "180," and "270" are recognized from the handwritten data, the corresponding operation commands are "Rotate display 90 degrees," "Rotate display 180 degrees," and "Rotate display 270 degrees." The display device 2 displays these operation commands and accepts operations from the user. When an operation command is selected, the operation command processing unit 28 executes the content described in the processing item. For example, the operation command processing unit 28 instructs the rotation processing unit 29 to rotate characters or images.

[0065] Also, when the character string "Japanese" is recognized, the corresponding operation commands are "Set to Japanese (display language)" and "Set to Japanese (conversion destination language)." When an operation command is selected, the operation command processing unit 28 registers the pen ID and the display language (or the conversion destination language) in the conversion language storage unit 41.

[0066] [Table 3] Table 3 shows a schematic of the contents of the input data storage unit 43. The input data is the attribute of data input by the user. The input data is recorded for each object (one stroke data, one character string, one image, etc.). · DataId is the identification information of the input data. Type is the type of input data, and can be Stroke, Text, or Image. The attributes held by the input data can differ depending on the Type, but Table 3 explains the case where Type="Text". Text represents a character string, and Image is an image. ·PenID is the pen ID of the pen 2500 used to enter this string. ·Angle is the display direction of this string. · StartPoint is the coordinate of the upper left vertex of the bounding rectangle of the string. · StartTime is the time when the user started writing the string. · EndPoint is the coordinate of the lower right vertex of the bounding rectangle of the string. · EndTime is the time when the user finishes writing the string. · FontName is the font name as a string. · FontSize is the font size. ·Text is the entered text (character code). ·Language is the language of the character string. In this embodiment, the display language or the conversion language is set.

[0067] <Example of selectable candidates> Next, referring to Fig. 5, an operation guide 500 displayed when converting handwritten data will be described. Fig. 5 shows an example of the operation guide 500 and selectable candidates 530 displayed by the operation guide. The user handwrites handwritten data 504, and the operation guide 500 is displayed. The operation guide 500 displays operation command candidates 510, handwritten recognition character string candidates 506, converted character string candidates 507, and character string / predictive conversion candidates 508. The selectable candidates 530 are the operation command candidates 510, the handwritten recognition character string candidates 506, the converted character string candidates 507, and the character string / predictive conversion candidates 508. The selectable candidates 530 excluding the operation command candidates 510 are referred to as character string candidates 539.

[0068] Handwritten data 504 is the character "gi" handwritten by the user. Handwritten data rectangular area display 503 is displayed surrounding handwritten data 504. In Fig. 5, operation guide 500 is displayed when one character is input, but the timing at which operation guide 500 is displayed is the timing at which the user stops handwriting. Therefore, the number of characters in handwritten data 504 is arbitrary.

[0069] The character string candidates are arranged in descending order of probability in handwriting recognition character string candidate 506, conversion character string candidate 507, and character string / predictive conversion candidate 508. "Gi" in handwriting recognition character string candidate 506 is a candidate for the recognition result. In this example, character recognition unit 23 correctly recognizes "Gi".

[0070] Conversion character string candidate 507 is a conversion character string candidate (e.g., a phrase containing "technique") converted from the result of kana-kanji conversion of "gi" (e.g., "technique"). In this example, "Giryoshi" is an abbreviation for "Technology Mass Production Trial." Character string / predictive conversion candidate 508 is a predicted character string candidate converted from conversion character string candidate 507. In this example, "Approve the giryoshi" and "Address to send the minutes of the meeting" are displayed.

[0071] Operation command candidates 510 are predefined operation command candidates (commands for file operations, character editing, etc.) that are displayed according to the recognized characters. In the example of Fig. 5, the first character of the line "》" 511 indicates that it is an operation command candidate. In Fig. 5, "meiji", which is a character string candidate for "gi", partially matches the definition data, so operation command candidate 510 is displayed.

[0072] The operation command candidates 510 are displayed when operation command definition data including the converted character string is found, and are not necessarily displayed at all times. In this embodiment, the operation command candidates 510 related to the conversion are displayed.

[0073] The operation header 520 has buttons 501, 509, 502, and 505. Button 501 accepts an operation to switch between predictive conversion and kana conversion. Button 502 operates the page of candidate display. In the example of FIG. 5, there are three candidate display pages, and the first page is currently displayed. Button 505 accepts the deletion of the operation guide 500. When the user presses button 505, the operation acceptance unit 27 accepts it, and the display control unit 24 erases the display except for the handwritten data. Button 509 accepts the deletion of all displays. When the user presses button 509, the operation acceptance unit 27 accepts it, and the display control unit 24 erases all displays shown in FIG. 5, including the handwritten data, allowing the user to start over with handwriting.

[0074] <Registering display direction and conversion language> 6 is a diagram for explaining the display direction. In this embodiment, the direction of the perpendicular line from the center of the display to a predetermined side is set to 0°. The display directions facing the user on each side are set to 90°, 180°, and 270° counterclockwise.

[0075] There are several methods by which the display device 2 can determine which of the four sides the user who is writing handwriting is on. In one method, the input direction detection unit 32 can determine which of the four sides the user is on, based on the direction in which the user is writing handwriting.

[0076] Figure 7 is a diagram that shows the correspondence between the direction of handwriting by the user and the display direction. For example, if the coordinates of the stroke data move from left to right and from top to bottom (horizontal writing from left to right), the side that exists from top to bottom is the display direction. The start and end points of each character are shown with black circles in Figure 7, and it can be seen that the overall movement is from the top left to the bottom right.

[0077] Similarly, if the coordinates of the stroke data move from top to bottom and from right to left (vertical writing from top to bottom), the side that exists from top to bottom is the display direction. In this way, the input direction detection unit 32 can estimate which of the four sides the user is located in based on the handwritten data.

[0078] Furthermore, the correspondence between the coordinates of the handwritten data and the display direction may be generated by the manufacturer of the display device 2 through machine learning. For example, a person in charge inputs time-series handwritten data to a neural network and provides the display direction as teacher data, thereby obtaining the input direction detection unit 32 as a learning model.

[0079] 6, the display directions are set to four angles, 0°, 90°, 180°, and 270°, but the display directions may be set more precisely. For example, the display directions may be set at intervals of 45° or 30°.

[0080] As another method, there is a method in which the display device 2 judges in which direction of the four sides the user is present by using the pen 2500. In this method, first, the pen 2500 is associated with a display direction.

[0081] Fig. 8 is an example of a diagram for explaining a method for inputting angle information. Fig. 8 explains a case where a user who is positioned at a 90° angle from the display device 2 inputs angle information. Characters handwritten from a 90° angle are correctly recognized when rotated 90° clockwise, so it is recommended that a display direction of 90° be input.

[0082] 8(a) shows a state in which operation guide 500 is displayed because a user standing at an angle of 90 degrees from display device 2 has handwritten "ぎ". Since display device 2 performs character recognition of "ぎ" handwritten from an angle of 90 degrees while keeping the display direction at the initial value of 0 degrees, an unexpected selectable candidate 530 is displayed.

[0083] When inputting angle information, the user hand-draws a straight line in the operation guide 500 from the top to the bottom as seen by the user. Fig. 8(b) shows an example of this straight line 521. The angle α formed by the 0° direction and this straight line 521 in the counterclockwise direction is the display direction. In other words, the angle α formed by the straight line 522 drawn from the starting point S in the 0° direction and the straight line 521 input by the user in the counterclockwise direction is the display direction. Therefore, the display direction input by the user in Fig. 8(b) is 90°.

[0084] For detecting a straight line, for example, the coordinates from the start point S to the end point E are converted into a straight line by the least squares method, and the obtained correlation coefficient is compared with a threshold value to determine whether or not it is a straight line.

[0085] Immediately after the user starts drawing the straight line 521 (immediately after the pen 2500 touches the starting point S of the straight line 521), the display device 2 erases the operation guide 500. Also, immediately after the user finishes drawing the straight line 521 (immediately after the pen 2500 is released from the ending point E of the straight line 521), the display device 2 searches for the value of the angle α that is closest to 90 degrees, 180 degrees, 270 degrees, and 0 degrees, and determines this as the display direction. The angle α itself may be angle information. This display direction is associated with the pen 2500 that was used.

[0086] When the pen tip of the pen 2500 is pressed for handwriting or the like, the pen 2500 transmits a pen ID to the display device 2, so that the display device 2 can associate the pen 2500 with a display direction.

[0087] Fig. 8(c) shows the operation guide 500 immediately after the operation in Fig. 8(b). Since 90° is set as the display direction of the pen 2500, the handwritten data is internally rotated 90° clockwise for character recognition, and the operation guide 500 is displayed rotated 90° counterclockwise. Note that the user may manually input the angle information from a menu.

[0088] Fig. 9 is a diagram for explaining a method in which a user associates a display direction with a conversion destination language. In Fig. 9, the user is present in a display direction of 0° and a display direction of 180°. As explained in Fig. 7 or Fig. 8, the display device 2 can determine which of the four sides the user is present in based on the handwritten data. Alternatively, the display direction is already associated with the pen 2500 used by each user.

[0089] A user with a display orientation of 0° handwrites "Japanese" to call up an operation command for setting the language. As a result, operation commands 310, "Set to Japanese (display language)" and "Set to Japanese (conversion destination language)", are displayed on the operation guide 500. When the user selects the operation command "Set to Japanese (display language)", the operation command processing unit 28 sets the display language "Japanese" to the display orientation of 0° detected by the input direction detection unit 32 in the handwritten data of "Japanese" in association with the pen ID. The pen ID is transmitted from the pen 2500 at the time of handwriting. Alternatively, the operation command processing unit 28 sets the display language "Japanese" to the display orientation of 0° of the pen 2500 used by the user with a display orientation of 0° in association with the pen ID. The same applies when "Set to Japanese (conversion destination language)" is selected.

[0090] Also, a user in a 180° display direction handwrites "English" to call up a language setting operation command. As a result, operation commands 311 such as "Set to English (Display language)" and "Set to English (Destination language)" are displayed on the operation guide 500. When a user selects the "Set to English (Display language)" operation command, the operation command processing unit 28 sets the display language "English" in the 180° display direction detected by the input direction detection unit 32 in the handwritten data of "English" in association with the pen ID. Alternatively, the operation command processing unit 28 sets the display language "English" in the 180° display direction of the pen 2500 used by a user in a 180° display direction in association with the pen ID. The same applies when "Set to English (Destination language)" is selected. In this manner, the conversion language storage unit 41 as shown in Table 1 is generated.

[0091] The correspondence between the pen ID, the display direction, and the conversion destination language registered in the conversion language storage unit 41 can be cancelled by the user executing a predetermined operation command.

[0092] Furthermore, handwritten data written by a user using the pen 2500 with pen ID=1 is character-recognized in the display language associated with pen ID=1. For example, "Hello" handwritten by the user is recognized as "Hello" in Japanese. Furthermore, "Hello" is converted to the target language, English, according to the user's operation.

[0093] <Example of handwritten data conversion> Next, an example of conversion of handwritten data will be described with reference to Fig. 10 to Fig. 18. Fig. 10 shows handwritten data 504 handwritten by a user. In Fig. 10, "Honjitsu no gi" was handwritten on the display (touch panel) with a pen 2500 having a pen ID=1. The communication unit 215a of the display device 2 receives the pen ID=1 from the pen 2500.

[0094] Fig. 11 shows five character string candidates 539 displayed by the display control unit 24 after the character recognition unit 23 performs character recognition of "honjitsunogi." In Fig. 11, five character string candidates 539, "Today's agenda," "Today's minutes," "Today's agenda," "Today's ceremony," and "Today's technology," are displayed on the operation guide 500. This is because the display language set for the pen 2500 with pen ID=1 is Japanese, and the character recognition unit 23 has converted it into Japanese character string candidates 539.

[0095] Fig. 12 shows a converted character string 101 converted into a conversion target language, which is displayed when the user selects an arbitrary character string candidate 539. In Fig. 11, the user selects "Today's proceedings" with the pen 2500. Because the pen 2500 has pressed inside the operation guide, the language conversion unit 30 determines to convert the character string candidate into the conversion target language of the pen 2500. Because the conversion target language set for the pen 2500 with pen ID = 1 is English, the language conversion unit 30 converts the selected Japanese character string candidate "Today's proceedings" into English. Therefore, in Fig. 12, the display control unit 24 displays "Today's proceedings". When the character string candidate is selected, the operation guide 500 is erased.

[0096] 12, the user of pen ID=1 determines that communication with an English speaker using pen 2500 with pen ID=2 is not going well (they do not communicate well), and that the converted character string 101 displayed by the user is inappropriate. The user of pen ID=1 attempts to reconvert the converted character string 101.

[0097] 13 is a diagram illustrating the re-conversion of the converted character string 101. The user selects the converted character string 101, "Today's proceedings."

[0098] 13A shows a method for selecting the destination character string 101 by the user pressing and holding the pen 2500 with pen ID=1. When the coordinates of the pen tip of the pen 2500 are within the circumscribing rectangle of the destination character string 101 and are pressed for a certain period of time or longer, the language conversion unit 30 determines that the destination character string 101 has been selected.

[0099] 13(b) shows a method in which the user selects the destination character string 101 by surrounding it with a frame 102 using a pen 2500 with a pen ID of 1. If the rightmost, leftmost, topmost, and bottommost coordinates of the frame 102 are outside the circumscribing rectangle of the destination character string 101, the language conversion unit 30 determines that the destination character string 101 has been selected.

[0100] 13(c) shows a method in which the user selects the destination character string 101 by drawing a superimposed line 104 on the destination character string 101 with a pen 2500 having a pen ID of 1. The width direction of the destination character string 101 is taken as the X-axis, and the height direction as the Y-axis. If the overall Y coordinate of the superimposed line 104 is included in the height direction of the circumscribing rectangle of the destination character string 101 (is between the minimum and maximum Y coordinates of the circumscribing rectangle), the language conversion unit 30 determines that the destination character string 101 has been selected.

[0101] Fig. 14 shows five character string candidates 539 displayed in response to the selection of the conversion destination character string "Today's proceedings." The character recognition unit 23 stores the character string candidates in Fig. 11, and the display control unit 24 displays them again. Alternatively, the character recognition unit 23 may store handwritten data 504 of "Honjitsunogi" and perform character recognition again, thereby causing the display control unit 24 to display the five character string candidates 539 again.

[0102] 13, the pen ID of the pen 2500 used to select the converted character string 101 is 1, so the character recognition unit 23 recognizes it as Japanese, and the display control unit 24 also displays the Japanese character string candidate 539. As will be described later, if the pen ID of the pen 2500 used to select the converted character string 101 is 2, the character string candidate 539 is displayed in English.

[0103] 14, the operation guide 500 including the character string candidate 539 is displayed again, so that the user using the pen 2500 with the pen ID=1 can select the character string candidate again. For example, in FIG.

[0104] FIG. 15 shows a destination character string 101 in the destination language that is displayed when the user selects "Today's minutes" with the pen 2500. Because the pen 2500 has pressed inside the operation guide in FIG. 14, the language conversion unit 30 determines to convert the character string candidate into the destination language of the pen 2500. Because the destination language set for the pen 2500 with pen ID=1 is English, the language conversion unit 30 converts the selected Japanese character string candidate "Today's minutes" into English. Therefore, in FIG. 15, the display control unit 24 displays the destination character string 101, "Today's minutes." When the character string candidate 539 is selected, the operation guide 500 is erased.

[0105] In this way, the display device 2 of this embodiment allows the user to select a character string to be converted into the conversion target language from a plurality of character string candidates to be converted into other languages, facilitating smooth communication. Even if a character string has been converted into the conversion target language once, the user can select the conversion target character string 101 to display a plurality of character string candidates to be converted into another language again. Therefore, the user can select another character string candidate and convert the handwritten data into the conversion target language without having to handwrite the data multiple times.

[0106] <Reconversion by pen 2500 of pen ID=2> 13, when a situation occurs in which communication does not go well, the user of the pen 2500 with pen ID=2 may select "Today's proceedings." Since the pen ID of the pen 2500 used to select the conversion destination character string 101 is 2, the language conversion unit 30 displays the recognition result in English.

[0107] 16 shows character string candidates 539 that are displayed when "Today's proceedings" is selected by the pen 2500 with pen ID=2. The language conversion unit 30 converts five character string candidates, "Today's proceedings," "Today's minutes," "Today's agenda," "Today's ceremony," and "Today's technology," into English, which is the display language of the pen 2500 with pen ID=2.

[0108] Since the user of pen 2500 with pen ID = 2 is assumed to be an English speaker, this user can select a string candidate from string candidates 539 that can be displayed in a language that the user of pen 2500 with pen ID = 1 can understand. For example, if the user of pen 2500 with pen ID = 2 selects "Today's minutes" on pen 2500, since the target language for conversion of pen 2500 with pen ID = 2 is Japanese, the language conversion unit 30 converts "Today's minutes" to "本日の議事録". Therefore, in FIG. 16, Japanese becomes the target string 101 after conversion.

[0109] In this way, since English speakers can directly select string candidates on their own pen 2500, communication becomes easier.

[0110] <Conversion by pen 2500 with pen ID = 3> According to Table 1, for pen 2500 with pen ID = 3, the display language is Japanese and the target language for conversion is Chinese. Therefore, in FIG. 11, when the user of pen 2500 with pen ID = 3 selects a string candidate, the language conversion unit 30 converts the selected string candidate to Chinese.

[0111] FIG. 17 is an example of conversion when the user of pen 2500 with pen ID = 3 selects a string candidate in FIG. 11. Suppose the user selects "本日の議事" on pen 2500 with pen ID = 3. Since the pen 2500 has pressed inside the operation guide, the language conversion unit 30 determines to convert the string candidate "本日の議事" to the target language of pen 2500. Since the target language set for pen 2500 with pen ID = 3 is Chinese, the language conversion unit 30 converts the selected Japanese string candidate to Chinese.

[0112] In FIG. 17, the display control unit 24 displays the target string 101 converted to "今天的程序" in Chinese. By selecting the string candidate 539, the operation guide 500 is erased.

[0113] In this way, the display device 2 can convert the character string candidate according to the conversion destination language associated with the pen 2500 used by the user.

[0114] <Conversion process to the conversion destination language associated with the pen> FIG. 18 is an example of a flowchart illustrating a process in which the display device 2 converts a selected character string candidate into a conversion target language associated with the pen 2500. In the process shown in FIG.

[0115] First, input of handwritten data is accepted (S1) according to the coordinates of the pen 2500 detected by the contact position detection unit 21. The drawing data generation unit 22 holds the coordinate points and generates handwritten data.

[0116] When a certain time has elapsed since the user removed the pen 2500 from the display, the character recognition unit 23 performs character recognition on the handwritten data (S2). The character recognition unit 23 acquires the display language from the conversion language storage unit 41 in Table 1 based on the pen ID received from the pen 2500 by the pen communication unit 31. The character recognition unit 23 performs character recognition on the handwritten data in this display language.

[0117] The display control unit 24 displays the operation guide 500 (S3). The display control unit 24 displays at least the character string candidate 539, and if the converted character string corresponds to an operation command, it also displays the operation command candidate.

[0118] When the user selects character string candidate 539 with pen 2500, contact position detection unit 21 accepts the selection of character string candidate 539 (S4). Contact position detection unit 21 detects the coordinates of the selection by pen 2500. Language conversion unit 30 determines that the character string candidate should be converted into the conversion destination language of pen 2500 because pen 2500 has pressed within the operation guide. Language conversion unit 30 converts the character string candidate into the conversion destination language associated with the pen ID received by pen communication unit 31 from pen 2500 (S5). Display control unit 24 displays conversion destination character string 101, and erases operation guide 500 in response to the selection of the character string candidate.

[0119] Next, the language conversion unit 30 judges whether or not the converted character string 101 has been selected by the pen 2500 (S6). If not, the processing in FIG.

[0120] When the converted character string 101 is selected by the pen 2500, the display control unit 24 displays the operation guide 500 again (S7). The character recognition unit 23 acquires the display language from the converted language storage unit 41 in Table 1 based on the pen ID received by the pen communication unit 31 from the pen 2500. When the character recognition unit 23 has already converted the handwritten data into the character string candidate 539 in this display language, the display control unit 24 displays the stored character string candidate 539. When the character recognition unit 23 has not already converted the handwritten data into the character string candidate 539 in this display language, the character recognition unit 23 converts the handwritten data 504 into the character string candidate 539 in the display language, and the display control unit 24 displays the character string candidate 539.

[0121] Thereafter, the process returns to step S4, and the user can repeatedly select character string candidates 539 that will result in good communication.

[0122] <About rotating the converted string> As shown in Table 1, the conversion language storage unit 41 registers a display direction associated with the pen ID. When a character string is displayed in the display language in this display direction, it is easy to read because it faces the user of the pen 2500. Therefore, when the display device 2 displays the converted character string 101 converted into the converted language, the rotation processing unit 29 may rotate the converted character string 101 to the display direction of the pen 2500 having the converted language as one of its display languages.

[0123] For example, the conversion destination language of the pen 2500 with pen ID=1 is English, but the display language of the pen 2500 with pen ID=2 is English. Therefore, the rotation processing unit 29 rotates the conversion destination character string 101 converted into English by 180°, which is the display orientation of the pen ID=2, and displays it.

[0124] FIG. 19 is a diagram for explaining the rotation of the converted character string 101. It is assumed that the display direction is determined as shown in the figure. FIG. 19(a) is in the same state as FIG. 11. When the user selects the character string candidate 539 with the pen 2500 with pen ID=1, the character string candidate 539 is converted to English. Since the pen 2500 having English as its display language is set in the conversion language storage unit 41 and the display direction of the pen is 180°, the rotation processing unit 29 rotates the converted character string 101 in English to a display direction of 180°.

[0125] 19(b) shows the converted character string 101 rotated to a display orientation of 180°. Therefore, the converted character string 101 is easy to read for a user present at a display orientation of 180°.

[0126] Next, assume that communication does not go well, and in the state of Fig. 19(b), the user selects the converted character string 101 with the pen 2500 of pen ID = 1. In this case, since the display direction associated with pen ID = 1 is 0°, the display control unit 24 displays the operation guide 500 in a display direction of 0°.

[0127] 20A shows the converted character string 101 and the operation guide 500 displayed at a display orientation of 0°. Therefore, the user can easily read the character string candidate 539 with the pen 2500 having the pen ID=1.

[0128] Also, suppose that communication does not go well, and in the state of Fig. 19(b), the user selects the converted character string 101 with the pen 2500 of pen ID = 2. In this case, since the display direction associated with pen ID = 1 is 180°, the display control unit 24 displays the operation guide 500 in a display direction of 180°.

[0129] 20B shows the converted character string 101 and the operation guide 500 displayed in a display direction of 180°. Therefore, the user can easily read the character string candidates with the pen 2500 having the pen ID=2.

[0130] Figure 20 shows cases where the display direction associated with the pen ID is 0° and 180°, but as shown in Figure 21, the display device 2 can similarly display the operation guide 500 when the display direction associated with the pen ID is 90° and 270°.

[0131] Fig. 21(a) shows a converted character string 101 and an operation guide 500 displayed in a display direction of 90°. Fig. 21(b) shows a converted character string 101 and an operation guide 500 displayed in a display direction of 270°. Figs. 21(a) and 21(b) both show a case in which a user selects a converted character string 101 from the state shown in Fig. 19(b) using a pen 2500 associated with display directions of 90° and 270°. The character string candidate 539 in Fig. 21 is in English, but can be displayed in any language associated with the pen 2500, such as Chinese or French.

[0132] In this way, the display device 2 can rotate the converted character string 101 and the operation guide 500 while converting the character string, thereby facilitating communication.

[0133] <Major Effects> As described above, the display device of this embodiment allows the user to select a character string to be converted into a destination language from a plurality of character string candidates to be converted into other languages, facilitating smooth communication. Furthermore, even if a character string has been converted into a destination language once, the user can select a plurality of character string candidates to be converted into another language again by selecting the destination character string 101. Therefore, the user can select another character string candidate and convert the handwritten data into the destination language without having to write the handwritten data by hand multiple times. Furthermore, the display device 2 can convert the selected character string candidate into the destination language set in the pen. EXAMPLES

[0134] In the following embodiment, another configuration example of the display device 2 will be described.

[0135] <<Another configuration example 1 of the display device>> Although the display device 2 of the present embodiment has been described as having a large touch panel, the display device 2 is not limited to having a touch panel.

[0136] Fig. 22 is a diagram showing another example of the configuration of the display device 2. In Fig. 22, a projector 411 is installed on the upper side of a normal whiteboard 413. This projector 411 corresponds to the display device 2. The normal whiteboard 413 is not a flat panel display integrated with a touch panel, but a whiteboard on which a user directly writes by hand with a marker. The whiteboard may be a blackboard, and may be any flat surface large enough to project an image.

[0137] The projector 411 has an ultra-short focal length optical system and can project an image with little distortion from a distance of about 10 cm onto the whiteboard 413. This image may be transmitted from a PC connected wirelessly or via a wire, or may be stored in the projector 411.

[0138] A user writes handwriting on the whiteboard 413 using a dedicated electronic pen 2501. The electronic pen 2501 has a light-emitting part, for example at the tip, that is switched on and emits light when the user presses it against the whiteboard 413 to write. The wavelength of the light is near-infrared or infrared, so it is invisible to the user's eyes. The projector 411 has a camera that captures the light-emitting part, analyzes the image, and identifies the direction of the electronic pen 2501. The electronic pen 2501 also emits sound waves along with the light emission, and the projector 411 calculates the distance based on the arrival time of the sound waves. The projector 411 can identify the position of the electronic pen 2501 based on the direction and distance. Handwritten data is drawn (projected) at the position of the electronic pen 2501.

[0139] Since the projector 411 projects the menu 430, when the user presses a button with the electronic pen 2501, the projector 411 identifies the position of the electronic pen 2501 and the pressed button by the ON signal of the switch. For example, when the save button 431 is pressed, the handwritten data (coordinate point sequence) handwritten by the user is saved by the projector 411. The projector 411 saves the handwritten information in a predetermined server 412 or USB memory 2600 or the like. The handwritten information is saved for each page. Since the information is saved as coordinates rather than as image data, the user can re-edit it. However, in this embodiment, the menu 430 does not need to be displayed because the operation command can be called up by handwriting. EXAMPLES

[0140] <<Another configuration example 2 of the display device>> 23 is a diagram showing another example of the configuration of the display device 2. In the example of FIG.

[0141] The terminal device 600 is connected by wire to the image projection device 700A and the pen operation detection device 810. The image projection device 700A projects image data input by the terminal device 600 onto the screen 800.

[0142] The pen operation detection device 810 communicates with the electronic pen 820 and detects the operation of the electronic pen 820 in the vicinity of the screen 800. Specifically, the electronic pen 820 detects coordinate information indicating a point pointed to by the electronic pen 820 on the screen 800 (the detection method may be the same as that in FIG. 22 ), and transmits the coordinate information to the terminal device 600.

[0143] The terminal device 600 generates image data of handwritten data input by the electronic pen 820 based on the coordinate information received from the pen operation detection device 810, and causes the image projection device 700A to draw an image of the handwritten data on the screen 800.

[0144] Furthermore, the terminal device 600 generates superimposed image data indicating a superimposed image obtained by combining the background image projected by the image projection device 700A and the image of the handwritten data input by the electronic pen 820. EXAMPLES

[0145] <<Another configuration example 3 of the display device>> Fig. 24 is a diagram showing a configuration example of the display device 2. In the example of Fig. 24, the display device 2 includes a terminal device 600, a display 800A, and a pen motion detection device 810A.

[0146] The pen action detection device 810 is disposed near the display 800A, detects coordinate information indicating a point indicated by the electronic pen 820A on the display 800A (the detection method may be the same as that in FIG. 22), and transmits the coordinate information to the terminal device 600. In the example of FIG. 24, the electronic pen 820A may be charged by the terminal device 600 via a USB connector.

[0147] Based on the coordinate information received from the pen operation detection device 810A, the terminal device 600 generates image data of the handwritten data input by the electronic pen 820A, and displays the image data on the display 800A. EXAMPLES

[0148] <<Another Configuration Example 4 of the Display Device>> Fig. 25 is a diagram showing a configuration example of the display device 2. In the example of Fig. 25, the display device 2 includes a terminal device 600 and an image projection device 700A.

[0149] The terminal device 600 performs wireless communication (Bluetooth (registered trademark) or the like) with the electronic pen 820B to receive coordinate information of a point indicated by the electronic pen 820B on the screen 800. The coordinate information may be obtained by the electronic pen 820B reading minute position information formed on the screen 800, or the coordinate information may be received from the screen 800.

[0150] Then, the terminal device 600 generates image data of the handwritten data input by the electronic pen 820B based on the received coordinate information, and causes the image projection device 700A to project an image of the handwritten data.

[0151] Furthermore, the terminal device 600 generates superimposed image data indicating a superimposed image obtained by combining the background image projected by the image projection device 700A and the image of the handwritten data input by the electronic pen 820B.

[0152] As described above, each of the above-described embodiments can be applied to various system configurations.

[0153] <Other application examples> The above describes the best mode for carrying out the present invention using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0154] The character string is stored as character codes, and the handwritten data is stored as coordinate point data in the display device 2. The data can also be stored in various storage media or in a storage device on a network, and later downloaded from the display device 2 for reuse. The display device 2 to be reused may be any display device, or may be a general information processing device. Therefore, the user can reproduce the handwritten content on a different display device 2 and continue a meeting, etc.

[0155] For example, in the present embodiment, an electronic whiteboard has been described as an example, but the electronic whiteboard may be called an electronic whiteboard, an electronic information board, or the like. In addition, the present embodiment can be suitably applied to any information processing device having a touch panel. The information processing device equipped with a touch panel may be, for example, a PJ (Projector), an output device such as digital signage, a HUD (Head Up Display) device, an industrial machine, an imaging device, a sound collecting device, a medical device, a network home appliance, a notebook PC (Personal Computer), a mobile phone, a smartphone, a tablet terminal, a game machine, a PDA (Personal Digital Assistant), a digital camera, a wearable PC, or a desktop PC.

[0156] In this embodiment, the coordinates of the pen are detected by detecting the coordinates of the pen tip with a touch panel, but the display device 2 may detect the coordinates of the pen tip with ultrasonic waves. The pen also emits ultrasonic waves together with light, and the display device 2 calculates the distance based on the arrival time of the ultrasonic waves. The position of the pen can be identified based on the direction and distance. The projector draws (projects) the trajectory of the pen as stroke data.

[0157] Moreover, the configuration example in FIG. 4 and the like is divided according to main functions in order to facilitate understanding of the processing by the display device 2. The present invention is not limited by the manner in which the processing units are divided or the names of the processing units. The processing by the display device 2 can be divided into more processing units according to the processing contents. Also, it can be divided so that one processing unit includes more processes.

[0158] Furthermore, part of the processing performed by the display device 2 may be performed by a server connected to the display device 2 via a network.

[0159] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above.

[0160] The contact position detection unit 21 is an example of a receiving means. The storage unit 40 is an example of a storage means. The character recognition unit 23 is an example of a character recognition means. The display control unit 24 is an example of a display control means. The language conversion unit 30 is an example of a conversion means. [Explanation of symbols]

[0161] 2 Display device [Prior art documents] [Patent documents]

[0162] [Patent Document 1] Patent No. 3599927

Claims

1. a receiving means for receiving input of handwritten data by an electronic pen; character recognition means for converting the handwritten data into a plurality of character strings; a display control means for displaying the plurality of character strings converted by the character recognition means; a storage means for storing a conversion destination language and a display language in association with the identification information of the electronic pen; the display control means converts the character string selected by the electronic pen from among the plurality of character strings into a plurality of character strings in the conversion target language associated with identification information of the electronic pen, and displays the converted character string; The display device according to claim 1, wherein the character recognition means converts the handwritten data into the display language associated with identification information of the electronic pen used to input the handwritten data.

2. When the display control means receives a selection of a destination character string converted into the destination language by the electronic pen, The display device according to claim 1 , characterized in that the display control means re-displays the converted plurality of character strings in the display language associated with the electronic pen used to select the converted character string.

3. The display device according to claim 2, characterized in that the display control means converts the character string selected by the electronic pen from the plurality of displayed character strings into a conversion target language associated with the electronic pen used for selection and displays the converted character string.

4. In the storage means, different display languages ​​are associated with the first electronic pen and the second electronic pen, respectively; The display device according to claim 2, characterized in that the display control means displays the converted multiple character strings in different display languages ​​when the converted character string is selected with the first electronic pen and when it is selected with the second electronic pen.

5. In the storage means, the first electronic pen and the third electronic pen are associated with different conversion destination languages, respectively; The display device according to claim 1, characterized in that when a character string is selected from the plurality of character strings with the first electronic pen and when a character string is selected from the plurality of character strings with the third electronic pen, the display control means converts the selected character string into a different target language and displays it.

6. In the storage means, a display direction is associated with identification information of the electronic pen, The display device according to any one of claims 2 to 4, characterized in that the display control means displays the converted character string in a display direction corresponding to the electronic pen having the converted language as one of its display languages.

7. When the selection of the conversion destination character string displayed in the display direction is accepted by the electronic pen, The display device according to claim 6, characterized in that the display control means displays the converted multiple character strings in the display language associated with the electronic pen used to select the converted character string and in the display direction associated with the electronic pen used to select the converted character string.

8. A step of accepting handwritten data input by an electronic pen by an accepting means; a character recognition means for converting the handwritten data into a plurality of character strings; a step of displaying the plurality of character strings converted by the character recognition means by a display control means; a step of displaying the character string selected by the electronic pen from among the plurality of character strings, by the display control means, into a plurality of character strings in a conversion target language associated with identification information of the electronic pen; a step of converting the handwritten data into a display language associated with identification information of the electronic pen used to input the handwritten data by the character recognition means; A display method having the following structure:

9. The display device, a receiving means for receiving input of handwritten data by an electronic pen; character recognition means for converting the handwritten data into a plurality of character strings; a display control means for displaying the plurality of character strings converted by the character recognition means; the display control means converts the character string selected by the electronic pen from among the plurality of character strings into a plurality of character strings in a conversion target language associated with identification information of the electronic pen, and displays the converted character string; the character recognition means converts the handwritten data into a display language associated with identification information of the electronic pen used to input the handwritten data; program.

Images