Input device, input method, and program

Abstract

To increase the degree of freedom in display of a character-recognized character string on the basis of a handwriting input.SOLUTION: An input device comprises: handwriting input means for receiving a handwriting input; setting reception means for receiving, from a user, a setting of setting information about whether or not to display a character-recognized character string on the basis of the handwriting input as a character string with an inclination adjustment; and display control means which displays the character string with an inclination adjustment nation or without an inclination adjustment on a display screen based on the setting information.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an input device, an input method, and a program. [Background technology]

[0002] For example, an electronic whiteboard or application (hereinafter referred to as an input device) that accepts handwritten input from a user accepts input of handwritten characters or handwritten character strings consisting of multiple handwritten characters by handwritten input of straight lines and curves (hereinafter referred to as strokes). A typical input device recognizes a handwritten character or character string as a character or character string based on the similarity between pre-stored character sample information and the input stroke. A user can input a character or character string that is to be recognized as a character at an angle, such as an upward slant to the right.

[0003] For example, technology that can recognize characters (character strings) handwritten in any direction (e.g., vertical, horizontal, and diagonal) without using a keyboard or providing a writing frame (writing frame) has been known for some time (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional input devices have a problem in that the degree of freedom in displaying a character string that has been character-recognized from a handwritten character string is not high. For example, when a character string that has been character-recognized from a handwritten character string is displayed in a conventional input device, the inclination of the displayed character string is fixed. Note that Patent Document 1 does not describe such a problem.

[0005] An object of the embodiment of the present invention is to improve the degree of freedom in displaying character strings that have been character-recognized based on handwritten input. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, claim 1 of the present application provides an input device comprising: handwriting input means for accepting handwritten input; setting receiving means for accepting from a user setting information regarding whether a character string recognized based on the handwritten input is to be displayed as a tilt-adjusted character string; and display control means for displaying the character string on a display screen as a tilt-adjusted character string or a non-tilt-adjusted character string based on the setting information. Effect of the Invention

[0007] According to an embodiment of the present invention, it is possible to improve the degree of freedom in displaying character strings that have been character-recognized based on handwritten input. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an overall configuration of an example of an electronic whiteboard according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating a hardware configuration of an example of a touch panel according to the embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a control unit according to the embodiment. [Figure 4] FIG. 2 is a functional block diagram of an example of an electronic whiteboard according to the present embodiment. [Diagram 5] FIG. 11 is an image diagram of an example of a setting screen according to the embodiment. [Figure 6] FIG. 4 is a diagram illustrating an example of setting information. [Figure 7] FIG. 11 is a sequence diagram of an example of a process of the electronic whiteboard according to the embodiment. [Figure 8] 11A and 11B are diagrams illustrating an example of a process for grouping strokes for each character or each character string. [Figure 9] 1 is an example of strokes grouped by character or by character string. [Figure 10] FIG. 11 is an image diagram of an example of processing on an electronic whiteboard according to the embodiment; [Figure 11] FIG. 13 is a display image diagram of an example of a character string that has been character-recognized. [Figure 12]FIG. 13 is a display image diagram of an example of a character string that has been character-recognized. [Figure 13] FIG. 13 is a display image diagram of an example of a character string that has been character-recognized. [Figure 14] FIG. 13 is a display image diagram of an example of a character string that has been character-recognized. [Figure 15] FIG. 13 is a display image diagram of an example of a character string that has been character-recognized. [Figure 16] FIG. 4 is a diagram illustrating an example of setting information. [Figure 17] 11 is a flowchart illustrating an example of a process for determining a display direction of a character string that has been character-recognized. [Figure 18] FIG. 13 is a display image diagram of an example of a character string that has been character-recognized. [Figure 19] 13 is a display image diagram of an example of a submenu that accepts a rotation angle designation of a character string from a user. FIG. [Figure 20] FIG. 13 is an image diagram of an example of a handwritten character string input by handwriting in vertical writing. [Figure 21] FIG. 13 is a diagram illustrating another example of the configuration of the input device. [Figure 22] FIG. 13 is a diagram illustrating another example of the configuration of the input device. [Figure 23] FIG. 13 is a diagram illustrating another example of the configuration of the input device. [Figure 24] FIG. 13 is a diagram illustrating another example of the configuration of the input device. [Diagram 25] FIG. 11 is an image diagram of an example of a setting screen according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that, although an electronic whiteboard 1, which is an example of an input device that accepts handwritten input from a user, will be described below, the present invention is not limited to the electronic whiteboard 1.

[0010] [First embodiment] <Hardware configuration of the electronic whiteboard> The overall configuration of an electronic whiteboard 1 according to an embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is an overall configuration diagram of an example of an electronic whiteboard according to the present embodiment. The electronic whiteboard 1 has a display unit 10, a touch panel 11, and a control unit 12. The display unit 10 is a display, and for example, an LCD (liquid crystal display) is used. Note that the display is an example of a device that displays a display screen, and for example, a projector capable of displaying (projecting) a display screen may be used.

[0011] Touch panel 11 has an infrared LED (light emitting diode) and a phototransistor arranged opposite each other, and detects position information by regarding a part where the phototransistor cannot detect light, i.e., a part where light is blocked, as a touched position. Touch panel 11 has a hardware configuration such as that shown in FIG. 2, for example.

[0012] Fig. 2 is a hardware configuration diagram of an example of a touch panel according to this embodiment. In the touch panel 11, the LEDs and phototransistors in one row are arranged at equal intervals, and each LED and phototransistor are arranged facing each other. Note that Fig. 2 shows an example of the touch panel 11 in which 20 LEDs and 20 phototransistors are arranged horizontally and 15 LEDs and 15 phototransistors are arranged vertically. For example, in the case of a touch panel 11 having a size of 40 inches or more, more LEDs and phototransistors are actually required.

[0013] The touch panel 11 is not limited to an optical type, and various detection means may be used, such as a capacitive touch panel that identifies the touch position by detecting a change in capacitance, a resistive film touch panel that identifies the touch position by a voltage change between two opposing resistive films, or an electromagnetic induction touch panel that identifies the touch position by detecting electromagnetic induction caused by a contact object touching the display unit. The touch panel 11 may or may not require an electronic pen to detect whether or not the tip of the pen has been touched. In this case, a fingertip or a pen-shaped stick can be used to perform the touch operation.

[0014] The control unit 12 has a hardware configuration as shown in Fig. 3, for example. Fig. 3 is a hardware configuration diagram of an example of the control unit according to this embodiment. The control unit 12 is composed of a CPU 20, a main memory 21, a clock 22, a bus controller 23, a ROM (Read Only Memory) 24, a PCI (Peripheral Component Interconnect) bridge 25, a cache memory 26, a hard disk 27, an HD (Hard Disk) controller 28, a display controller 29, a LAN controller 30, a LAN I / F (Interface) 31, a touch panel I / F 32, an RTC (Real Time Clock) 33, a CPU bus 34, a PCI bus 35, and an X bus 36 (internal bus).

[0015] The CPU 20 executes and processes the control processing program stored in the ROM 24, and the OS (Operating System) and various application programs read from the hard disk 27 to the main memory 21. The main memory 21 is composed of a DRAM (Dynamic Random Access Memory) and is used as a work area for the CPU 20, etc. The clock 22 is composed of a crystal oscillator and a frequency divider circuit, and generates a clock for controlling the operation timing of the CPU 20 and the bus controller 23. The bus controller 23 controls data transfers on the CPU bus 34 and the X bus 36.

[0016] Programs for starting up the system when the power is turned on and for controlling various devices are written in advance to the ROM 24. The PCI bridge 25 transfers data between the PCI bus 35 and the CPU 20 using a cache memory 26. The cache memory 26 is made up of DRAM and is used by the PCI bridge 25.

[0017] The hard disk 27 stores system software, various application programs, data saved by the user, etc. The HD controller 28 has, for example, an IDE (Integrated Device Electronics) interface as an interface with the hard disk 27, and performs high-speed data transfer with the hard disk 27.

[0018] The display controller 29 performs D / A (Digital / Analog) conversion of character data, graphic data, etc., and controls the display of this data on the display unit 10. The LAN controller 30 executes a communication protocol conforming to, for example, the IEEE (Institute of Electrical and Electronics Engineers) 802.3 standard, and controls communication with other devices connected to the LAN via the LAN I / F 31.

[0019] The touch panel I / F 32 has a port for the touch panel 11, and is controlled by a touch panel driver (control program). The RTC 33 is a date clock, and is backed up by a battery.

[0020] In this embodiment, unless otherwise specified, the CPU 20 uses the main memory 21 as a work area to execute processes according to a program.

[0021] <Terminology> The input means may be any means capable of handwriting by specifying coordinates on the touch panel 11. Examples include a pen, a human finger or hand, and a rod-shaped member. Also, gaze input may be possible. Handwritten data is data that displays, as a trajectory, a sequence of coordinate points on the touch panel 11 where the user has continuously moved the input means. Also, a series of operations in which the user presses the input means against the touch panel 11, moves it continuously, and then releases it from the touch panel 11 is called a stroke. Data handwritten by strokes is called stroke data. Handwritten data has one or more stroke data. Handwritten input refers to handwritten data being input by the user.

[0022] <Functional configuration of the electronic whiteboard> Next, functions of the electronic whiteboard 1 will be described with reference to Fig. 4. Fig. 4 is a functional block diagram of an example of an electronic whiteboard according to the present embodiment. The electronic whiteboard 1 shown in Fig. 4 includes a handwriting input unit 40, a stroke division unit 41, a data storage unit 42, a character centroid calculation unit 43, a character string inclination calculation unit 44, a character recognition / conversion unit 45, a display control unit 46, a network communication unit 47, and a setting reception unit 48.

[0023] Each function of the electronic whiteboard 1 is a function or means realized by any of the components shown in Figures 1 to 3 operating in accordance with instructions from the CPU 20 in accordance with a program expanded on the main memory 21.

[0024] The handwriting input unit 40 is realized by instructions from the CPU 20 shown in FIG. 3, and the touch panel 11 and touch panel I / F 32, etc., shown in FIG. 3, and obtains coordinate data of a portion touched on the touch panel 11 by the user's hand or the like (a pen or the user's hand serves as an input means), and accepts handwritten input by the user. The handwriting input unit 40 functions as a contact position detection unit. The handwriting input unit 40 obtains a coordinate data string from handwritten input by the user. The handwriting input unit 40 may obtain the coordinate data string as discrete values. Coordinate data between discrete values ​​is interpolated. The handwriting input unit 40 accepts stroke input from the coordinate data string obtained when the user wrote.

[0025] The stroke division unit 41 is realized by the CPU 20 shown in Fig. 3 using the main memory 21 as a work area to execute processing according to a program, and divides (groups) input strokes into strokes for each character and strokes for each character string. The method of grouping strokes for each character and each character string will be described later.

[0026] The data storage unit 42 is realized by an instruction from the CPU 20 shown in FIG. 3, the hard disk 27, the HD controller 28, etc., and stores various data used by the electronic whiteboard 1. The data storage unit 42 in FIG. 4 has a stroke storage unit 51, a character string storage unit 52, and a setting information storage unit 53. The stroke storage unit 51 is realized by an instruction from the CPU 20 shown in FIG. 3, the hard disk 27, the HD controller 28, etc., and stores stroke data input by the handwriting input unit 40. The character string storage unit 52 is realized by an instruction from the CPU 20 shown in FIG. 3, the hard disk 27, the HD controller 28, etc., and stores character string data obtained by character recognition from a character string input by a user with one or more strokes (hereinafter referred to as a handwritten character string). Character recognition refers to conversion into text data (data composed of some character code) that can be processed by a computer. A character code refers to a byte representation assigned to each character for the purpose of using the character on a computer.

[0027] The setting information storage unit 53 is realized by commands from the CPU 20 shown in Fig. 3, the hard disk 27 and the HD controller 28 shown in Fig. 3, and stores various setting information of the interactive whiteboard 1. The setting information stored in the setting information storage unit 53 also includes setting information for the "character string inclination adjustment function" described later.

[0028] The character centroid calculation unit 43 is realized by the CPU 20 shown in FIG. 3 using the main memory 21 as a work area and executing a process according to a program, and calculates the centroid coordinates of each character from the strokes of each grouped character, as described below. The character string inclination calculation unit 44 is realized by the CPU 20 shown in FIG. 3 using the main memory 21 as a work area and executing a process according to a program, and calculates the inclination (slant) of a character string from the centroid coordinates of each character, as described below. The character recognition and conversion unit 45 is realized by the CPU 20 shown in FIG. 3 using the main memory 21 as a work area and executing a process according to a program, and performs character recognition by comparing the strokes of each grouped character and each character string with dictionary data such as a handwriting recognition dictionary, and converts the strokes into a character string represented by character information that can be displayed as a font such as text data. Note that various algorithms have been devised for the method of character recognition of a handwritten character string, but details will be omitted since publicly known technology can be used.

[0029] The display control unit 46 is realized by an instruction from the CPU 20 shown in FIG. 3 and the display controller 29 shown in FIG. 3, and performs control for displaying strokes, converted character strings, etc. on the display (display unit 10). The network communication unit 47 is realized by an instruction from the CPU 20 shown in FIG. 3, the LAN controller 30 shown in FIG. 3, the LAN I / F 31, etc., and connects to a network such as a LAN to transmit and receive data. The setting reception unit 48 is realized by an instruction from the CPU 20 shown in FIG. 3, the touch panel 11 shown in FIG. 3, the touch panel I / F 32, etc., and performs control related to the setting screen 1000 shown in FIG. 5 that receives various settings of the electronic whiteboard 1 from the user, and stores the setting information of the various settings received from the setting screen in the setting information storage unit 53.

[0030] FIG. 5 is an image diagram of an example of a setting screen according to the present embodiment. The setting screen 1000 in FIG. 5 accepts various setting information settings for the electronic whiteboard 1 from the user. For example, the user can enable the character string tilt adjustment function described below by checking a check box 1002 with an explanatory text "display characters handwritten diagonally horizontally," and can disable the character string tilt adjustment function by unchecking the check box. The check box in the setting screen 1000 in FIG. 5 is an example. For example, the tilt adjustment function can be enabled ("Yes") or disabled ("No") by selecting "Yes" or "No" from a small round button-shaped input element such as a radio button.

[0031] The "enabled" or "disabled" setting of the "character string tilt adjustment function" received from the setting screen 1000 in Fig. 5 is stored in the setting information storage unit 53, for example, as the setting information in Fig. 6. Fig. 6 is a configuration diagram of an example of setting information. In Fig. 6, the "enabled" or "disabled" setting of the "character string tilt adjustment function" is represented by "ON" or "OFF".

[0032] <Processing> Fig. 7 is a sequence diagram of an example of processing of the electronic whiteboard according to the present embodiment. Fig. 7 describes processing for performing character recognition on a handwritten character string inputted diagonally and converting the character string into a character string that can be displayed as a font.

[0033] In step S10, the handwriting input unit 40 of the electronic whiteboard 1 accepts stroke input by the user from the coordinate data of the portion touched on the touch panel 11. The handwriting input unit 40 stores information about the strokes that have been accepted in the stroke storage unit 51 of the data storage unit 42. Then, the process proceeds to step S12, where the handwriting input unit 40 notifies the stroke division unit 41 of the information about the strokes that have been accepted.

[0034] The stroke division unit 41 requests the display control unit 46 to display the strokes notified from the handwriting input unit 40, and causes the strokes to be displayed on the display unit 10. The processes of steps S10 to S14 are repeated every time a stroke is input by the user.

[0035] The stroke dividing unit 41 groups the strokes input in step S12 into strokes for each character, for example, by a method that will be described with reference to Fig. 8. Fig. 8 is an explanatory diagram of an example of processing for grouping strokes for each character or each character string. Fig. 8 shows an example of a method for grouping strokes for each character or each character string based on the stroke interval.

[0036] 8 shows a time series of a "stroke input" state and a "no stroke input" state. When the time Δt during which strokes are not input is equal to or exceeds the character discrimination threshold, the stroke division unit 41 groups the strokes up to that time as one stroke group of a character. Also, when the time Δt during which strokes are not input is equal to or exceeds the character string discrimination threshold, the stroke division unit 41 groups the stroke groups of the character up to that time as one stroke group of a character string.

[0037] For example, in the example of Figure 8, since the time Δt between "stroke 4" and "stroke 5" is equal to or exceeds the character discrimination threshold, "stroke 1" through "stroke 4" immediately preceding them are grouped as a stroke group of a single character.

[0038] Also, in the example of Figure 8, since the time Δt between "stroke 7" and "stroke 8" is equal to or exceeds the character discrimination threshold, the strokes immediately preceding them, "stroke 5" through "stroke 7", are grouped as a stroke group of a single character.

[0039] Furthermore, in the example of Figure 8, since the time Δt between "stroke 7" and "stroke 8" is equal to or exceeds the character string discrimination threshold, the stroke group of characters from "stroke 1" to "stroke 4" just before that, and the stroke group of characters from "stroke 5" to "stroke 7" are grouped together as a stroke group of a single character string.

[0040] In the example of Fig. 8, strokes are grouped into character stroke groups and character string stroke groups as shown in Fig. 9. Fig. 9 shows an example of strokes grouped by character or by character string.

[0041] Returning to FIG. 7, if the time Δt during which no strokes are input is equal to or exceeds the character identification threshold, the stroke division unit 41 proceeds to step S16 and issues a notification to the stroke storage unit 51 to store the strokes up to that point as a stroke group of one character, as described using FIG. 8.

[0042] Proceeding to step S18, character centroid calculation unit 43 calculates the centroid coordinates of each character from the strokes grouped as a stroke group of one character. For example, character centroid calculation unit 43 may determine the centroid coordinates of the character by using the centroid of all stroke points constituting one character, or may determine the centroid coordinates of the character by using the centroid coordinates of each stroke grouped as a stroke group of one character.

[0043] The processes of steps S10 to S18 are repeated until the time Δt during which no strokes are input becomes equal to or exceeds the character string identification threshold by the stroke division unit 41. When the time Δt during which no strokes are input becomes equal to or exceeds the character string identification threshold, the process proceeds to step S20, and the stroke division unit 41 notifies the stroke storage unit 51 to store the stroke groups of the characters up to that point as a stroke group of one character string, as described with reference to FIG.

[0044] In step S22, the character recognition / conversion unit 45 detects that a stroke group of one character string has been stored in the stroke storage unit 51, and starts the following process of character recognition of the strokes included in the stroke group of the character string and converting them into a character string.

[0045] Proceeding to step S24, the character string tilt calculation unit 44 calculates the rotation angle of the handwritten character string (tilt of the handwritten character string), for example, as shown in FIG. 10(a). FIG. 10 is an image diagram of an example of processing of the electronic whiteboard according to this embodiment. The character string tilt calculation unit 44 estimates a center line 102 of the handwritten character string with respect to the center coordinates 100 of each handwritten character calculated in step S18. The center line 102 of the handwritten character string can be found, for example, by the least squares method with respect to the center coordinates 100 of each handwritten character. The character string tilt calculation unit 44 calculates an acute angle α° (angle 106) formed by the center line 102 of the handwritten character string and a line 104 horizontal to the display unit 10 as the rotation angle of the handwritten character string (tilt of the character string before adjustment).

[0046] Proceeding to step S26, character recognition / conversion unit 45 performs affine transformation on each handwritten character string (or each character) by the acute angle α° calculated in step S24, for example, as shown in FIG. 10(b), in order to adjust the inclination of handwritten character string 108 in the direction of horizontal line 104 (horizontal direction). Here, adjusting the inclination of handwritten character string 108 in the direction of horizontal line 104 (horizontal direction) refers to a process of rotating (straightening) the inclination of handwritten character string 108 to match horizontal line 104 as a reference, for example, in the case of FIG. 10(b). Through this process, the inclination of handwritten characters included in handwritten character string 108 is adjusted in the horizontal direction (the inclination of handwritten characters is rotated and straightened to match the horizontal direction as a reference).

[0047] Proceeding to step S28, the character recognition and conversion unit 45 performs character recognition on the horizontally adjusted handwritten character string 108 and converts it into a character string 110 such as text data. Proceeding to step S30, the character recognition and conversion unit 45 requests the display control unit 46 to update the display of the handwritten character string 108.

[0048] Proceeding to step S32, the display control unit 46 refers to the setting information, for example, shown in FIG. 6, stored in the data storage unit 42, and checks whether the setting of the "character string inclination adjustment function" is "enabled" or "disabled". If the character string inclination adjustment function is "disabled", the display control unit 46 rearranges the characters included in the character string 110 converted in step S28 so that the coordinates of the center of gravity of the handwritten character calculated in step S18 are at the center. Thereafter, the display control unit 46 performs affine transformation on the rearranged character string 112, for example, as shown in FIG. 10(d), by an acute angle of -α° for each character string (or character). By this process, the inclination of the character string 112 is adjusted in the same direction as the inclination of the handwritten character string 108. Here, adjusting the inclination of the character string 112 in the same direction as the inclination of the handwritten character string 108 means, for example, in the case of FIG. 10(d), rotating (adjusting) the inclination of the character string 112 to match the center line 102 of the handwritten character string that is used as a reference. The inclination of the characters included in the character string 112 is adjusted in the direction of the center line 102 (the inclination of the character string is rotated and adjusted in accordance with the reference center line).

[0049] Therefore, when the character string inclination adjustment function is "disabled", in step S36 the display control unit 46 can display the character-recognized character string 112 with the same inclination as the handwritten character string 108 input by the user, as shown in Fig. 11(a). Fig. 11 is a display image diagram of an example of a character string that has been character-recognized.

[0050] If the character string inclination adjustment function is "enabled", the display control unit 46 rearranges the characters included in the character string 110 converted in step S28 so that the barycentric coordinates of the handwritten characters calculated in step S18 come to the center. However, the display control unit 46 does not perform the affine transformation of step S34 on the rearranged character string 112.

[0051] Therefore, when the string inclination adjustment function is “enabled,” in step S36 the display control unit 46 can adjust the character-recognized string 112 to a horizontal direction different from the inclination of the handwritten string 108 input by the user and display it, as shown in FIG. 11(b).

[0052] The display image in Fig. 11 is an example, and the display control unit 46 may display images such as those shown in Fig. 12 to Fig. 15. Fig. 12 to Fig. 15 are display image diagrams of examples of character strings that have been character-recognized.

[0053] The display image in Fig. 12 is an example in which the handwritten character string in Fig. 12(a) inputted in a diagonal direction is character-recognized and a horizontal character string is displayed as in Fig. 12(f). The display control unit 46 may, for example, display the process (change in the character string) in which the character string converted by character recognition is adjusted from the character string in Fig. 12(a) to the character string in Fig. 12(f) (change in character string) by animation or the like. Fig. 12 shows, as an example, a step-by-step display of the change from the character string inclination to the character string recognized by character recognition, with the character string blinking.

[0054] The display image of Fig. 13 may display the center line 102 shown in Fig. 10(a) for the handwritten character string of Fig. 13(a) that has been input in an oblique direction. For example, when a handwritten character string including a certain number or more of handwritten characters (e.g., three characters) is input, if the handwritten character string is oblique, the display control unit 46 may visually notify the user that the handwritten character string is oblique by displaying the center line 102 as shown in Figs. 13(b) to 13(f). After adjustment of the inclination of the character string is completed as shown in Fig. 13(g), the display control unit 46 erases the center line 102.

[0055] In the display image of FIG. 13, as in FIGS. 13(d) to 13(f), the inclination of the character string is adjusted after it is converted into a character string such as text data by character recognition. However, it is also possible to adjust the inclination of the handwritten character string as in the display image of FIG. 14, for example, and then switch to displaying the character string as text data or the like after the inclination adjustment is complete, as in FIG. 14(f).

[0056] Furthermore, as in the display image of FIG. 15, by displaying not only the center line 102 but also a horizontal line 104 and an angle 106, the user may be visually notified that the handwritten character string is oblique.

[0057] In this way, according to the first embodiment, it is possible to switch by settings whether a character string obtained by character recognition from a handwritten character string handwritten in any direction is displayed in the same direction as the handwritten character string, or is displayed by adjusting the direction to the horizontal direction. Therefore, according to the first embodiment, it is possible to improve the degree of freedom in displaying a character string obtained by character recognition from a handwritten character string.

[0058] [Second embodiment] When multiple people input handwritten character strings to the electronic whiteboard 1, the display direction of the character string recognized from the handwritten character string desired by the users may differ depending on the cultural background of the users (= the character type to be input). The character type is defined as hiragana, katakana, kanji, alphabet, numbers, etc., but is not limited to these. For example, a user who inputs a handwritten character string of alphabets may want to display the character string recognized from the handwritten character string input diagonally in the same diagonal direction as the handwritten character string. Also, a user who inputs a handwritten character string of hiragana may want to adjust the character string horizontally and display the character string recognized from the handwritten character string input diagonally.

[0059] Therefore, in the second embodiment, the "character string slant adjustment function" can be set for each character type. In the second embodiment, the setting of the "character string slant adjustment function" for each character type is stored in the setting information storage unit 53, for example, as the setting information in FIG. 16. FIG. 16 is a configuration diagram of an example of setting information. In FIG. 16, the setting value of the "character string slant adjustment function" is set in association with each character type.

[0060] For example, in the setting information of FIG. 16, if the character string is alphabetic only, the "character string slant adjustment function" is disabled, and the display direction of the character-recognized character string is the same as the display direction of the handwritten character string. In addition, in the setting information of FIG. 16, if the character string is hiragana only, the "character string slant adjustment function" is enabled, and the display direction of the character-recognized character string is horizontal. In addition, in the setting information of FIG. 16, if the character string is alphabetic + other character types, the "character string slant adjustment function" is enabled, and the display direction of the character-recognized character string is horizontal.

[0061] 17 is a flowchart of an example of a process for determining the display direction of a character string that has been character-recognized. In step S50, the display control unit 46 recognizes the character type of the character string that has been character-recognized, such as alphabet only, hiragana only, or alphabet plus other character types. In step S52, the display control unit 46 determines whether the display direction of the handwritten character string is oblique.

[0062] If the display direction of the handwritten string is not diagonal, in other words, if the display direction of the handwritten string is horizontal, the display direction of the handwritten string will be horizontal, which is the same as the direction of the adjusted string, so the display control unit 46 proceeds to step S56 and displays the string in the same horizontal direction as the handwritten string.

[0063] Furthermore, if the display direction of the handwritten character string is oblique, the display direction of the handwritten character string differs from the direction of the adjusted character string (horizontal direction), so the display control unit 46 proceeds to step S54 and determines whether the character type recognized in step S50 is "alphabetical only."

[0064] If the character type recognized in step S50 is "alphabetical characters only", the display control unit 46 proceeds to step S56, where it displays the character-recognized character string in the same direction as the display direction of the handwritten character string. If the character type recognized in step S50 is not "alphabetical characters only", the display control unit 46 proceeds to step S58, where it adjusts the display direction of the character-recognized character string horizontally and displays it.

[0065] By processing the flowchart of FIG. 17, when the character type in FIG. 18(a) is "alphabet only" and the display control unit 46 performs character recognition of character string 124 from handwritten character string 120 that was manually input in an oblique direction, it can display character string 124 in the same direction (same inclination) as handwritten character string 120, as shown in FIG. 18(b).

[0066] Furthermore, by processing the flowchart of FIG. 17, when the character type in FIG. 18(a) is “Hiragana only” and character string 126 is character-recognized from handwritten character string 122 that was manually input diagonally, the display control unit 46 can display character string 126 adjusted in a horizontal direction different from handwritten character string 120, as shown in FIG. 18(b).

[0067] In this way, according to the second embodiment, it is possible to set for each character type whether a character string obtained by character recognition from a handwritten character string input by hand in any direction is displayed in the same direction as the handwritten character string, or is displayed by adjusting the direction to the horizontal direction. Therefore, according to the second embodiment, it is possible to improve the degree of freedom in displaying a character string obtained by character recognition from a handwritten character string.

[0068] [Other embodiments] In addition, as in the first or second embodiment, the character string 132 after character recognition may be displayed with the inclination of the handwritten character string unchanged or adjusted horizontally, and a submenu 130 that accepts a designation of a rotation angle of the character string 132 from the user may be displayed by an operation such as a long press on the character string 132, as shown in Fig. 19. The user can perform an operation to adjust the inclination of the character string 132 after character recognition by designating the rotation angle of the character string 132 from the submenu 130. The display control unit 46 adjusts the inclination of the character string after character recognition and displays it according to the operation to adjust the inclination of the character string 132 after character recognition received from the user.

[0069] In addition, in this embodiment, the accuracy of character recognition is improved by performing character recognition after adjusting the inclination of the handwritten character string in the horizontal direction, but in the case of a character string written vertically as in Fig. 20(a), for example, the accuracy of character recognition may not be improved. Fig. 20 is an image diagram of an example of a handwritten character string input by handwriting in vertical writing.

[0070] 20(b), an acute angle (angle 116) formed by center line 102 of the character string and line 114 oriented vertically with respect to display unit 10 is set as the rotation angle of the handwritten character string. Character recognition / conversion unit 45 may compare the determination rate of character recognition when angle 106 in FIG. 20(a) is set as the rotation angle with when angle 116 in FIG. 20(b) is set as the rotation angle, thereby determining whether the character string is written vertically or horizontally, and switching the direction in which the inclination of the recognized character string is adjusted.

[0071] In addition, in the case of a character string written vertically as in Fig. 20(a), the setting screen 1000 can enable the character string slant adjustment function by checking a check box 1002 with a description such as "display characters handwritten diagonally vertically" as shown in Fig. 25, and can disable the character string slant adjustment function by unchecking the check box. If enabled, the character string slant adjustment function in the setting screen 1000 of Fig. 25 vertically adjusts the slant of a character string written vertically by hand as in Fig. 20(b) after character recognition.

[0072] <Another example of input device configuration 1> Although the input device of this embodiment has been described as having a large touch panel, the input device is not limited to having a touch panel.

[0073] Fig. 21 is a diagram showing another example of the configuration of an input device. In Fig. 21, a projector 411 is installed on the upper side of a normal whiteboard 413. This projector 411 corresponds to the input device. The normal whiteboard 413 is not a flat panel display integrated with a touch panel, but a whiteboard on which a user directly writes by hand with a marker. The whiteboard may be a blackboard, and may be any flat surface large enough to project an image.

[0074] Projector 411 has an ultra-short focal length optical system, and can project an image with little distortion from a distance of about 10 cm onto whiteboard 413. This image may be transmitted from a PC connected wirelessly or via a wire, or may be stored in projector 411.

[0075] A user writes by hand on the whiteboard 413 using a dedicated electronic pen 2501. The electronic pen 2501 has a light-emitting part, for example at the tip, that is switched on and emits light when the user presses the electronic pen 2501 against the whiteboard 413 to write by hand. The wavelength of the light is near infrared or infrared, so it is invisible to the user's eyes. The projector 411 has a camera that captures the light-emitting part, analyzes the image, and identifies the direction of the electronic pen 2501.

[0076] The electronic pen 2501 also emits sound waves along with the light emission, and the projector 411 calculates the distance based on the arrival time of the sound waves. The position of the electronic pen 2501 can be specified based on the direction and distance. A stroke is drawn (projected) at the position of the electronic pen 2501.

[0077] The projector 411 projects the menu 430, and when the user presses a button with the electronic pen 2501, the projector 411 identifies the pressed button by the position of the electronic pen 2501 and an ON signal of a switch. For example, when the save button 431 is pressed, the projector 411 saves a stroke (a set of coordinates) handwritten by the user.

[0078] The projector 411 stores the handwritten information in a predetermined server 412 or USB memory 2600. The handwritten information is stored for each page. Since the information is stored as coordinates rather than as image data, the user can re-edit the information. However, in this embodiment, the menu 430 does not need to be displayed because operation commands can be called up by handwriting.

[0079] <Another example of input device configuration 2> Fig. 22 is a diagram showing another example of the configuration of the input device. In the example of Fig. 22, the input device includes a terminal device 600, an image projection device 700A, and a pen operation detection device 810.

[0080] The terminal device 600 is connected by wire to the image projection device 700A and the pen operation detection device 810. The image projection device 700A projects image data input by the terminal device 600 onto the screen 800.

[0081] The pen operation detection device 810 communicates with the electronic pen 820 and detects the operation of the electronic pen 820 in the vicinity of the screen 800. Specifically, the electronic pen 820 detects coordinate information indicating a point indicated by the electronic pen 820 on the screen 800 and transmits the coordinate information to the terminal device 600.

[0082] The terminal device 600 generates image data of a stroke image input by the electronic pen 820 based on the coordinate information received from the pen operation detection device 810, and causes the image projection device 700A to draw the stroke image on the screen 800.

[0083] Furthermore, the terminal device 600 generates superimposed image data indicating a superimposed image obtained by combining the background image projected by the image projection device 700A and the stroke image input by the electronic pen 820.

[0084] <Another example 3 of input device configuration> Fig. 23 is a diagram showing a configuration example of an input device. In the example of Fig. 23, the input device includes a terminal device 600, a display 800A, and a pen operation detection device 810.

[0085] The pen operation detection device 810 is disposed near the display 800A, detects coordinate information indicating a point indicated by the electronic pen 820A on the display 800A, and transmits the coordinate information to the terminal device 600. In the example of Fig. 34, the electronic pen 820A may be charged by the terminal device 600 via a USB connector.

[0086] Based on the coordinate information received from the pen operation detection device 810, the terminal device 600 generates image data of a stroke image input by the electronic pen 820A, and displays the image data on the display 800A.

[0087] <Another example 4 of input device configuration> Fig. 24 is a diagram showing a configuration example of an input device. In the example of Fig. 24, the input device includes a terminal device 600 and an image projection device 700A.

[0088] The terminal device 600 performs wireless communication (Bluetooth (registered trademark) or the like) with the electronic pen 820B and receives coordinate information of a point indicated by the electronic pen 820B on the screen 800. Based on the received coordinate information, the terminal device 600 generates image data of a stroke image input by the electronic pen 820B and causes the image projection device 700A to project the stroke image.

[0089] Furthermore, the terminal device 600 generates superimposed image data indicating a superimposed image obtained by combining the background image projected by the image projection device 700A and the stroke image input by the electronic pen 820.

[0090] As described above, each of the above-described embodiments can be applied to various system configurations.

[0091] <Other application examples> The above describes the best mode for carrying out the present invention using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0092] For example, although the present embodiment has been described as an example of an electronic whiteboard, the present invention can be suitably applied to any information processing device having a touch panel. A device having the same function as an electronic whiteboard is also called an electronic whiteboard, an electronic information board, an interactive board, etc. Examples of information processing devices equipped with a touch panel include output devices such as PJ (Projector), digital signage, HUD (Head Up Display) devices, industrial machinery, imaging devices, sound collection devices, medical equipment, network home appliances, notebook PCs (Personal Computers), mobile phones, smartphones, tablet terminals, game consoles, PDAs (Personal Digital Assistants), digital cameras, wearable PCs, desktop PCs, etc.

[0093] In this embodiment, the coordinates of the pen are detected by detecting the coordinates of the pen tip with a touch panel, but the coordinates of the pen tip may be detected by ultrasonic waves. The pen emits light and also emits ultrasonic waves, and the input device calculates the distance based on the arrival time of the ultrasonic waves. The input device can identify the position of the pen based on the direction and distance. In this case, the projector draws (projects) the trajectory of the pen as a stroke.

[0094] In addition, the configuration examples in Figs. 1 to 4 are divided according to main functions to facilitate understanding of the processing by the electronic whiteboard 1. The method of dividing the processing units and their names do not limit the present invention. The processing of the electronic whiteboard 1 can be divided into more processing units according to the processing contents. Also, it can be divided so that one processing unit includes more processes.

[0095] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above.

[0096] The handwriting input unit 40 is an example of a handwriting input means in the claims. The display control unit 46 is an example of a display control means. The setting reception unit 48 is an example of a setting reception means. [Explanation of symbols]

[0097] 1. Electronic whiteboard 10 Display 11 Touch Panel 12 Control section 40 Handwriting input section 41 Stroke division section 42 Data Storage Section 43 Character centroid calculation part 44 String slope calculation part 45 Character Recognition and Conversion Section 46 Display control unit 47 Network Communication Department 48 Settings reception section 51 Stroke storage section 52 String storage section 53 Setting information storage section 100 Centroid coordinates 102 Center line 104 Horizontal Line 106 angle 108 Handwritten Character String 110, 112 String 1000 Setting screen [Prior art documents] [Patent documents]

[0098] [Patent Document 1] JP 2007-188512 A

Claims

1. a handwriting input means for accepting handwritten input; a character recognition means for performing character recognition on the handwritten character string inputted by handwriting immediately before the time when the handwritten input is not performed reaches or exceeds a threshold value; a display control means for adjusting an inclination of the handwritten input character string in a horizontal direction and displaying the character string that has been character-recognized in a horizontal direction on a display unit, the display control means displaying on the display unit a change in inclination of the character string from the inclination before the adjustment to the inclination after the adjustment; 13. An input device comprising:

2. The display control means displays on the display unit a handwritten character string handwritten input before character recognition is performed during the process of adjusting the inclination of the character string, and displays the character string after the inclination adjustment is completed.

2. The input device according to claim 1, wherein:

3. The display unit displays a character string with or without tilt adjustment based on a setting of whether or not to adjust the tilt of the character string that has been recognized.

3. The input device according to claim 1 or 2,

4. The present invention further includes a setting receiving means for receiving, from a user, setting information regarding whether or not a character string recognized based on the handwritten input is to be displayed as a character string with an adjusted inclination, The setting receiving means receives a setting of the setting information, from a setting screen on which the input device can be set, as to whether to adjust the inclination of the character string before displaying the character string on the display unit, or to display the character string without adjusting the inclination of the character string on the display unit.

4. An input device according to claim 1, wherein:

5. The present invention further includes a setting receiving means for receiving, from a user, setting information regarding whether or not a character string recognized based on the handwritten input is to be displayed as a character string with an adjusted inclination, the setting receiving means receives a setting of the setting information from a user in association with a character type of the character string; The display control means displays on the display unit a character string with or without tilt adjustment according to a character type of the character string.

4. An input device according to claim 1, wherein:

6. a handwriting input step in which a handwriting input means accepts handwritten input; a character recognition step in which, when the time during which no handwritten input has been made becomes equal to or greater than a threshold, a character recognition means performs character recognition on the handwritten character string that has been handwritten input up until that time; a display control step of displaying the character string that has been handwritten and inputted horizontally on a display unit by a display control means, the display control step displaying on the display unit a change in the inclination of the character string from the inclination before the adjustment to the inclination after the adjustment; 13. An input method comprising:

7. A program to be executed by an input device, a handwriting input step in which a handwriting input means accepts handwritten input; a character recognition step in which, when the time during which no handwritten input has been made becomes equal to or greater than a threshold, a character recognition means performs character recognition on the handwritten character string that has been handwritten input up until that time; a display control step of displaying the character string that has been handwritten and inputted horizontally on a display unit by a display control means, the display control step displaying on the display unit a change in the inclination of the character string from the inclination before the adjustment to the inclination after the adjustment; A program comprising:

Images