Input device, input method, and program

Abstract

To increase the degree of freedom in display of a character-recognized character string on the basis of a handwriting input.SOLUTION: An input device comprises: handwriting input means for receiving a handwriting input; setting reception means for receiving, from a user, a setting of setting information about whether or not to display a character-recognized character string on the basis of the handwriting input as a character string with an inclination adjustment; and display control means which displays the character string with an inclination adjustment nation or without an inclination adjustment on a display screen based on the setting information.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an input device, an input method, and a program. [Background technology]

[0002] For example, an electronic whiteboard or application (hereinafter referred to as an input device) that accepts handwritten input from a user accepts input of handwritten characters or handwritten character strings consisting of multiple handwritten characters by hand-drawn straight lines and curves (hereinafter referred to as strokes). A typical input device recognizes a handwritten character or character string as a character or character string based on the similarity between pre-stored character sample information and the input strokes. A user can input a character or character string that they want to be recognized as a character by slanting the input, such as by sloping upward to the right.

[0003] For example, technology that can recognize characters (strings of characters) handwritten in any direction (e.g., vertical, horizontal, and diagonal) without using a keyboard or providing an entry frame (writing frame) has been known for some time (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional input devices have a problem in that they do not have a high degree of freedom in displaying character strings that have been character-recognized from handwritten character strings. For example, when conventional input devices display character strings that have been character-recognized from handwritten character strings, the inclination of the displayed character strings is fixed. Note that Patent Document 1 does not address this problem.

[0005] An object of the embodiment of the present invention is to improve the degree of freedom in displaying character strings that have been character-recognized based on handwritten input. [Means for solving the problem]

[0006] In order to achieve the above object, claim 1 of the present application provides a handwriting input means for accepting handwritten input, When the time during which no handwriting input has been made reaches or exceeds a threshold, a character recognition means performs character recognition on the handwritten character string that has been handwritten input immediately before that time, and a display control means adjusts the inclination of the handwritten input character string in a horizontal direction and displays the recognized character string horizontally on a display unit, and the change in inclination of the character string from the inclination before the adjustment to the inclination after the adjustment is displayed on the display unit. and a display control means for displaying the input device on the display screen. [Effects of the Invention]

[0007] According to the embodiment of the present invention, it is possible to improve the degree of freedom in displaying character strings that have been character-recognized based on handwritten input. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an overall configuration of an example of an electronic whiteboard according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a touch panel according to the present embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a control unit according to the present embodiment. [Figure 4] FIG. 2 is a functional block diagram of an example of an electronic whiteboard according to the present embodiment. [Figure 5] FIG. 10 is an image diagram of an example of a setting screen according to the embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of setting information. [Figure 7] FIG. 10 is a sequence diagram of an example of processing of the electronic whiteboard according to the embodiment; [Figure 8] 10A and 10B are diagrams illustrating an example of processing for grouping strokes for each character or each character string. [Figure 9] 10 shows an example of strokes grouped by character or by character string. [Figure 10] FIG. 10 is an image diagram of an example of processing on an electronic whiteboard according to the present embodiment. [Figure 11] FIG. 10 is a display image diagram of an example of a character string that has been character-recognized. [Figure 12] FIG. 10 is a display image diagram of an example of a character string that has been character-recognized. [Figure 13] FIG. 10 is a display image diagram of an example of a character string that has been character-recognized. [Figure 14]FIG. 10 is a display image diagram of an example of a character string that has been character-recognized. [Figure 15] FIG. 10 is a display image diagram of an example of a character string that has been character-recognized. [Figure 16] FIG. 10 is a diagram illustrating an example of setting information. [Figure 17] 10 is a flowchart illustrating an example of a process for determining the display direction of a character string that has been character-recognized. [Figure 18] FIG. 10 is a display image diagram of an example of a character string that has been character-recognized. [Figure 19] FIG. 10 is a display image diagram of an example of a submenu that accepts a user's specification of a rotation angle of a character string. [Figure 20] FIG. 10 is an image diagram of an example of a handwritten character string input by hand in vertical writing. [Figure 21] FIG. 10 is a diagram illustrating another example of the configuration of the input device. [Figure 22] FIG. 10 is a diagram illustrating another example of the configuration of the input device. [Figure 23] FIG. 10 is a diagram illustrating another example of the configuration of the input device. [Figure 24] FIG. 10 is a diagram illustrating another example of the configuration of the input device. [Figure 25] FIG. 10 is an image diagram of an example of a setting screen according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that, although an electronic whiteboard 1 will be described below as an example of an input device that accepts handwritten input from a user, the present invention is not limited to the electronic whiteboard 1.

[0010] [First embodiment] <Hardware configuration of the electronic whiteboard> The overall configuration of an electronic whiteboard 1 according to an embodiment of the present invention will be described with reference to Figure 1. Figure 1 is an overall configuration diagram of an example of an electronic whiteboard according to this embodiment. The electronic whiteboard 1 has a display unit 10, a touch panel 11, and a control unit 12. The display unit 10 is a display, and for example, an LCD (liquid crystal display) is used. The display is an example of a device that displays a display screen, and for example, a projector that can display (project) a display screen may be used.

[0011] The touch panel 11 has an infrared light emitting diode (LED) and a phototransistor arranged opposite each other, and detects position information by taking the area where the phototransistor cannot detect light, i.e., the area where light is blocked, as the touched position. The touch panel 11 has a hardware configuration such as that shown in FIG. 2, for example.

[0012] Fig. 2 is a hardware configuration diagram of an example of a touch panel according to this embodiment. In touch panel 11, LEDs and phototransistors are arranged at equal intervals in a row, and each LED and phototransistor is arranged facing each other. Note that Fig. 2 shows an example of touch panel 11 in which 20 LEDs and 20 phototransistors are arranged horizontally and 15 phototransistors are arranged vertically. For example, a touch panel 11 with a size of 40 inches or more actually requires many more LEDs and phototransistors.

[0013] Note that touch panel 11 is not limited to an optical type, and various detection means may be used, such as a capacitive touch panel that identifies the touch position by detecting changes in capacitance, a resistive film touch panel that identifies the touch position by voltage changes across two opposing resistive films, or an electromagnetic induction touch panel that identifies the touch position by detecting electromagnetic induction caused by an object touching the display unit. Touch panel 11 may or may not require an electronic pen to detect whether or not the pen tip is touching. In this case, a fingertip or a pen-shaped stick can be used to perform touch operations.

[0014] The control unit 12 has a hardware configuration as shown in Fig. 3, for example. Fig. 3 is a hardware configuration diagram of an example of a control unit according to this embodiment. The control unit 12 is made up of a CPU 20, a main memory 21, a clock 22, a bus controller 23, a ROM (Read Only Memory) 24, a PCI (Peripheral Component Interconnect) bridge 25, a cache memory 26, a hard disk 27, an HD (Hard Disk) controller 28, a display controller 29, a LAN controller 30, a LAN I / F (Interface) 31, a touch panel I / F 32, an RTC (Real Time Clock) 33, a CPU bus 34, a PCI bus 35, and an X bus 36 (internal bus).

[0015] The CPU 20 executes and processes control processing programs stored in the ROM 24, and the OS (Operating System) and various application programs read from the hard disk 27 to the main memory 21. The main memory 21 is made up of DRAM (Dynamic Random Access Memory) and is used as a work area for the CPU 20. The clock 22 is made up of a crystal oscillator and a frequency divider circuit, and generates a clock for controlling the operation timing of the CPU 20 and the bus controller 23. The bus controller 23 controls data transfers via the CPU bus 34 and the X bus 36.

[0016] Programs for starting up the system when the power is turned on and for controlling various devices are written in advance to the ROM 24. The PCI bridge 25 uses a cache memory 26 to transfer data between the PCI bus 35 and the CPU 20. The cache memory 26 is made up of DRAM and is used by the PCI bridge 25.

[0017] The hard disk 27 stores system software, various application programs, data saved by the user, etc. The HD controller 28 has, for example, an IDE (Integrated Device Electronics) interface as an interface with the hard disk 27, and performs high-speed data transfer with the hard disk 27.

[0018] The display controller 29 performs D / A (Digital / Analog) conversion of character data, graphic data, etc., and controls the display of this data on the display unit 10. The LAN controller 30 executes a communication protocol that complies with the IEEE (Institute of Electrical and Electronics Engineers) 802.3 standard, for example, and controls communication with other devices connected to the LAN via the LAN I / F 31.

[0019] The touch panel I / F 32 has a port for the touch panel 11 and is controlled by a touch panel driver (control program). The RTC 33 is a date and time clock and is backed up by a battery.

[0020] In this embodiment, unless otherwise specified, the CPU 20 uses the main memory 21 as a work area to execute processing according to a program.

[0021] <Terminology> The input means may be any means that allows handwriting by specifying coordinates on the touch panel 11. Examples include a pen, a human finger or hand, or a rod-shaped object. Eye-gaze input may also be possible. Handwritten data is data that displays a sequence of coordinate points as a trajectory on the touch panel 11 where the user has continuously moved the input means. A stroke refers to a series of operations in which the user presses the input means against the touch panel 11, moves it continuously, and then releases it from the touch panel 11. Data handwritten using strokes is called stroke data. Handwritten data has one or more strokes. Handwritten input refers to handwritten data being input by the user.

[0022] <Functional configuration of the electronic whiteboard> Next, functions of the electronic whiteboard 1 will be described with reference to Fig. 4. Fig. 4 is a functional block diagram of an example of an electronic whiteboard according to this embodiment. The electronic whiteboard 1 shown in Fig. 4 includes a handwriting input unit 40, a stroke division unit 41, a data storage unit 42, a character centroid calculation unit 43, a character string tilt calculation unit 44, a character recognition / conversion unit 45, a display control unit 46, a network communication unit 47, and a setting reception unit 48.

[0023] Each function of the electronic whiteboard 1 is a function or means realized by any of the components shown in Figures 1 to 3 operating in accordance with instructions from the CPU 20 in accordance with a program deployed on the main memory 21.

[0024] The handwriting input unit 40 is realized by commands from the CPU 20 shown in FIG. 3, the touch panel 11, the touch panel I / F 32, etc. shown in FIG. 3, and calculates coordinate data of the part of the touch panel 11 that is touched by the user's hand or the like (a pen or the user's hand serves as an input means), and accepts handwritten input by the user. The handwriting input unit 40 functions as a contact position detection unit. The handwriting input unit 40 calculates a coordinate data string from handwritten input by the user. The handwriting input unit 40 may calculate the coordinate data string as discrete values. Coordinate data between discrete values ​​is interpolated. The handwriting input unit 40 accepts stroke input from the coordinate data string calculated when the user writes.

[0025] The stroke dividing unit 41 is realized by the CPU 20 shown in Fig. 3 using the main memory 21 as a work area and executing processing according to a program, and divides (groups) input strokes into strokes for each character and strokes for each character string. The method of grouping strokes for each character and strokes for each character string will be described later.

[0026] The data storage unit 42 is implemented by instructions from the CPU 20 shown in FIG. 3, the hard disk 27, the HD controller 28, and other components, and stores various data used by the interactive whiteboard 1. The data storage unit 42 shown in FIG. 4 includes a stroke storage unit 51, a character string storage unit 52, and a setting information storage unit 53. The stroke storage unit 51 is implemented by instructions from the CPU 20 shown in FIG. 3, the hard disk 27, the HD controller 28, and other components, and stores stroke data input by the handwriting input unit 40. The character string storage unit 52 is implemented by instructions from the CPU 20 shown in FIG. 3, the hard disk 27, the HD controller 28, and other components, and stores character string data obtained by character recognition from a character string input by a user using one or more strokes (hereinafter referred to as a handwritten character string). Character recognition refers to conversion into computer-processable text data (data composed of some character code). A character code refers to the byte representation assigned to each character for the purpose of using the character on a computer.

[0027] The setting information storage unit 53 is realized by commands from the CPU 20 shown in Fig. 3, the hard disk 27 and the HD controller 28 shown in Fig. 3, etc., and stores various setting information for the interactive whiteboard 1. The setting information stored by the setting information storage unit 53 also includes setting information for the "character string tilt adjustment function" described below.

[0028] The character centroid calculation unit 43 is implemented by the CPU 20 shown in FIG. 3 using the main memory 21 as a work area and executing a program to calculate the centroid coordinates of each character from the strokes of each grouped character, as described below. The character string tilt calculation unit 44 is implemented by the CPU 20 shown in FIG. 3 using the main memory 21 as a work area and executing a program to calculate the tilt (inclination) of the character string from the centroid coordinates of each character, as described below. The character recognition and conversion unit 45 is implemented by the CPU 20 shown in FIG. 3 using the main memory 21 as a work area and executing a program to recognize characters by comparing the strokes of each grouped character and each character string with dictionary data such as a handwriting recognition dictionary, and converts the strokes into a character string represented by character information that can be displayed as a font such as text data. Note that various algorithms have been devised for character recognition of handwritten character strings, but since publicly known technology can be used, details will be omitted.

[0029] The display control unit 46 is realized by instructions from the CPU 20 shown in FIG. 3 and the display controller 29 shown in FIG. 3, etc., and performs control for displaying strokes, converted character strings, etc. on the display (display unit 10). The network communication unit 47 is realized by instructions from the CPU 20 shown in FIG. 3, the LAN controller 30 and LAN I / F 31 shown in FIG. 3, etc., and connects to a network such as a LAN to send and receive data. The setting reception unit 48 is realized by instructions from the CPU 20 shown in FIG. 3 and the touch panel 11 and touch panel I / F 32 shown in FIG. 3, etc., and performs control related to the setting screen 1000 shown in FIG. 5, which receives various settings for the interactive whiteboard 1 from the user, and stores the setting information for the various settings received from the setting screen in the setting information storage unit 53.

[0030] FIG. 5 is an image diagram of an example of a setting screen according to this embodiment. The setting screen 1000 in FIG. 5 accepts various setting information settings for the interactive whiteboard 1 from the user. For example, the user can enable the character string tilt adjustment function (described below) by checking a checkbox 1002 with the description "Display characters handwritten diagonally horizontally," and can disable the character string tilt adjustment function by unchecking the checkbox. Note that the checkbox in the setting screen 1000 in FIG. 5 is an example. For example, the tilt adjustment function can be enabled ("Yes") or disabled ("No") by selecting "Yes" or "No" from a small, round button-shaped input element, such as a radio button.

[0031] The "enabled" or "disabled" setting of the "character string tilt adjustment function" received from the setting screen 1000 in Fig. 5 is saved in the setting information saving unit 53, for example, as the setting information in Fig. 6. Fig. 6 is a diagram showing an example of setting information. In Fig. 6, the "enabled" or "disabled" setting of the "character string tilt adjustment function" is represented by "ON" or "OFF."

[0032] <Processing> Fig. 7 is a sequence diagram of an example of processing of the electronic whiteboard according to this embodiment. Fig. 7 describes processing for character recognition of a handwritten character string input diagonally and converting it into a character string that can be displayed as a font.

[0033] In step S10, the handwriting input unit 40 of the interactive whiteboard 1 accepts stroke input by the user from the coordinate data of the portion touched on the touch panel 11. The handwriting input unit 40 stores information about the accepted input stroke in the stroke storage unit 51 of the data storage unit 42. Then, the process proceeds to step S12, where the handwriting input unit 40 notifies the stroke division unit 41 of the information about the accepted input stroke.

[0034] The stroke division unit 41 requests the display control unit 46 to display the strokes notified by the handwriting input unit 40, and causes the strokes to be displayed on the display unit 10. The processes of steps S10 to S14 are repeated every time a stroke is input by the user.

[0035] The stroke dividing unit 41 groups the strokes input in step S12 into strokes for each character, for example, using a method that will be explained using Fig. 8. Fig. 8 is an explanatory diagram of an example of processing for grouping strokes for each character or each character string. Fig. 8 shows an example of a method for grouping strokes for each character or each character string based on the stroke spacing.

[0036] 8 shows a time series of a "stroke input" state and a "no stroke input" state. When the time Δt during which no strokes are input is equal to or exceeds the character discrimination threshold, the stroke segmentation unit 41 groups the strokes up to that time as a stroke group of one character. Also, when the time Δt during which no strokes are input is equal to or exceeds the character string discrimination threshold, the stroke segmentation unit 41 groups the stroke groups of the character up to that time as a stroke group of one character string.

[0037] For example, in the example of Figure 8, the time Δt between "stroke 4" and "stroke 5" is equal to or exceeds the character discrimination threshold, so "stroke 1" to "stroke 4" just before that are grouped as a stroke group of one character.

[0038] Also, in the example of Figure 8, since the time Δt between "stroke 7" and "stroke 8" is equal to or exceeds the character discrimination threshold, "stroke 5" to "stroke 7" immediately before that are grouped as a stroke group of one character.

[0039] Furthermore, in the example of Figure 8, since the time Δt between "stroke 7" and "stroke 8" is equal to or exceeds the character string discrimination threshold, the stroke group of characters from "stroke 1" to "stroke 4" immediately before that, and the stroke group of characters from "stroke 5" to "stroke 7" are grouped together as a stroke group of a single character string.

[0040] In the example of Fig. 8, strokes are grouped into character stroke groups and character string stroke groups as shown in Fig. 9. Fig. 9 shows an example of strokes grouped by character or by character string.

[0041] Returning to FIG. 7, if the time Δt during which no strokes are input is equal to or exceeds the character identification threshold, the stroke division unit 41 proceeds to step S16 and notifies the stroke storage unit 51 to store the strokes up to that point as a stroke group of one character, as explained using FIG. 8.

[0042] Proceeding to step S18, the character centroid calculation unit 43 calculates the centroid coordinates of each character from the strokes grouped as a stroke group of one character. For example, the character centroid calculation unit 43 may use the centroid of the points of all the strokes that make up one character as the centroid coordinate of the character, or may use the centroid calculated from the centroid coordinates of each stroke grouped as a stroke group of one character as the centroid coordinate of the character.

[0043] The processing of steps S10 to S18 is repeated until the time Δt during which no strokes are input becomes equal to or exceeds the character string identification threshold value. When the time Δt during which no strokes are input becomes equal to or exceeds the character string identification threshold value, the process proceeds to step S20, and the stroke segmentation unit 41 notifies the stroke storage unit 51 to store the stroke groups of the characters up to that point as a stroke group of one character string, as described with reference to FIG.

[0044] In step S22, the character recognition / conversion unit 45 detects that a stroke group of one character string has been stored in the stroke storage unit 51, and starts the following process of character recognition of the strokes included in the stroke group of the character string and converting them into a character string.

[0045] Proceeding to step S24, the character string tilt calculation unit 44 calculates the rotation angle of the handwritten character string (tilt of the handwritten character string), for example, as shown in FIG. 10(a). FIG. 10 is an image diagram of an example of processing by the electronic whiteboard according to this embodiment. The character string tilt calculation unit 44 estimates a center line 102 of the handwritten character string based on the centroid coordinates 100 of each handwritten character calculated in step S18. The center line 102 of the handwritten character string can be found using, for example, the least squares method with respect to the centroid coordinates 100 of each handwritten character. The character string tilt calculation unit 44 calculates the acute angle α° (angle 106) formed by the center line 102 of the handwritten character string and a line 104 horizontal to the display unit 10 as the rotation angle of the handwritten character string (tilt of the character string before adjustment).

[0046] Proceeding to step S26, the character recognition / conversion unit 45 performs affine transformation on each handwritten character string (or each character) by the acute angle α° calculated in step S24, for example, as shown in FIG. 10(b), in order to adjust the tilt of the handwritten character string 108 in the direction of the horizontal line 104 (horizontal direction). Here, adjusting the tilt of the handwritten character string 108 in the direction of the horizontal line 104 (horizontal direction) means, for example, in the case of FIG. 10(b), a process of rotating (straightening) the tilt of the handwritten character string 108 to match the horizontal line 104 that serves as a reference. Through this process, the tilt of the handwritten characters included in the handwritten character string 108 is adjusted in the horizontal direction (the tilt of the handwritten characters is rotated and straightened to match the horizontal direction that serves as a reference).

[0047] Proceeding to step S28, the character recognition and conversion unit 45 performs character recognition on the horizontally adjusted handwritten character string 108 and converts it into a character string 110 such as text data. Proceeding to step S30, the character recognition and conversion unit 45 requests the display control unit 46 to update the display of the handwritten character string 108.

[0048] In step S32, the display control unit 46 refers to the setting information stored in the data storage unit 42, for example, as shown in FIG. 6, and checks whether the setting for the "character string tilt adjustment function" is "enabled" or "disabled." If the character string tilt adjustment function is "disabled," the display control unit 46 rearranges the characters included in the character string 110 converted in step S28 so that the coordinates of the center of gravity of the handwritten characters calculated in step S18 are at the center. The display control unit 46 then performs affine transformation on the rearranged character string 112, for example, by an acute angle of -α°, for each character, as shown in FIG. 10(d). This process adjusts the tilt of the character string 112 to the same direction as the tilt of the handwritten character string 108. Here, adjusting the tilt of the character string 112 to the same direction as the tilt of the handwritten character string 108 refers to, for example, in the case of FIG. 10(d), rotating (straightening) the tilt of the character string 112 to align it with the center line 102 of the handwritten character string that serves as a reference. The inclination of the characters included in the character string 112 is adjusted in the direction of the center line 102 (the inclination of the character string is rotated and adjusted in accordance with the center line that is used as a reference).

[0049] Therefore, when the character string tilt adjustment function is "disabled," in step S36, the display control unit 46 can display the character-recognized character string 112 with the same tilt as the handwritten character string 108 input by the user, as shown in Fig. 11(a). Fig. 11 is a display image diagram of an example of a character string that has been recognized.

[0050] If the character string tilt adjustment function is "enabled," the display control unit 46 rearranges the characters included in the character string 110 converted in step S28 so that the coordinates of the center of gravity of the handwritten characters calculated in step S18 are at the center. However, the display control unit 46 does not perform the affine transformation of step S34 on the rearranged character string 112.

[0051] Therefore, if the string inclination adjustment function is "enabled," in step S36 the display control unit 46 can adjust the recognized string 112 to a horizontal direction different from the inclination of the handwritten string 108 entered by the user and display it, as shown in FIG. 11(b).

[0052] The display image in Figure 11 is an example, and the display control unit 46 may display images such as those shown in Figures 12 to 15. Figures 12 to 15 are display image diagrams of examples of character strings that have been character-recognized.

[0053] The display image in Fig. 12 is an example in which the handwritten character string in Fig. 12(a) that was input diagonally is character-recognized and a horizontal character string is displayed as in Fig. 12(f). For example, the display control unit 46 may display, by animation or the like, the process in which the tilt of the character string converted by character recognition is adjusted from the tilt of the handwritten character string in Fig. 12(a) to the tilt of the character string in Fig. 12(f). Fig. 12 shows, as an example, an example in which the change from the tilt of the handwritten character string to the tilt of the character string after character recognition is displayed in stages while the character string blinks.

[0054] Furthermore, the display image of Fig. 13 may be configured to display the center line 102 shown in Fig. 10(a) for the handwritten character string of Fig. 13(a) that has been input diagonally. For example, when a handwritten character string containing a certain number or more of handwritten characters (e.g., three characters) is input, if the handwritten character string is diagonal, the display control unit 46 may visually notify the user that the handwritten character string is diagonal by displaying the center line 102 as shown in Figs. 13(b) to 13(f). After adjustment of the character string inclination is completed as shown in Fig. 13(g), the display control unit 46 erases the center line 102.

[0055] In the display image of FIG. 13, as in FIGS. 13(d) to 13(f), the tilt of the character string is adjusted after it is converted into a character string such as text data by character recognition. However, as in the display image of FIG. 14, for example, the tilt may be adjusted while the character string is still handwritten, and after the tilt adjustment is complete, the display may be switched to the character string such as text data, as in FIG. 14(f).

[0056] Furthermore, as in the display image of FIG. 15, not only the center line 102 but also a horizontal line 104 and an angle 106 may be displayed to visually notify the user that the handwritten character string is slanted.

[0057] As described above, according to the first embodiment, it is possible to switch, by setting, whether a character string obtained by character recognition from a handwritten character string input by hand in any direction is displayed in the original direction of the handwritten character string or is displayed by adjusting the direction to the horizontal direction. Therefore, according to the first embodiment, it is possible to improve the degree of freedom in displaying a character string obtained by character recognition from a handwritten character string.

[0058] [Second embodiment] When multiple users input handwritten strings on the interactive whiteboard 1, the display orientation of the string recognized from the handwritten string desired by the users may differ depending on the cultural background of the users (i.e., the type of characters input). Character types are defined as hiragana, katakana, kanji, alphabets, and numbers, but are not limited to these. For example, a user who inputs a handwritten string of alphabets may want the string recognized from the diagonally input handwritten string to be displayed in the same diagonal orientation as the handwritten string. Also, a user who inputs a handwritten string of hiragana may want the string recognized from the diagonally input handwritten string to be displayed horizontally.

[0059] Therefore, in the second embodiment, the "character string tilt adjustment function" can be set for each character type. In the second embodiment, the setting of the "character string tilt adjustment function" for each character type is saved in the setting information saving unit 53, for example, as the setting information in FIG. 16. FIG. 16 is a diagram showing an example of the configuration of the setting information. In FIG. 16, the setting value of the "character string tilt adjustment function" is set in association with each character type.

[0060] For example, in the setting information of FIG. 16, if the character string is alphabetic only, the "character string tilt adjustment function" is disabled, and the display direction of the character-recognized character string is the same as the display direction of the handwritten character string. Also, in the setting information of FIG. 16, if the character string is hiragana only, the "character string tilt adjustment function" is enabled, and the display direction of the character-recognized character string is horizontal. Furthermore, in the setting information of FIG. 16, if the character string is alphabetic + other character types, the "character string tilt adjustment function" is enabled, and the display direction of the character-recognized character string is horizontal.

[0061] 17 is a flowchart of an example of a process for determining the display direction of a character string that has undergone character recognition. In step S50, the display control unit 46 recognizes the character type of the character string that has undergone character recognition, such as alphabets only, hiragana only, or alphabets plus other character types. In step S52, the display control unit 46 determines whether the display direction of the handwritten character string is oblique.

[0062] If the display direction of the handwritten character string is not diagonal, in other words, if the display direction of the handwritten character string is horizontal, the display direction of the handwritten character string will be horizontal, which is the same as the direction of the string after adjustment, so the display control unit 46 proceeds to step S56 and displays the character string in the same horizontal direction as the handwritten character string.

[0063] Furthermore, if the display direction of the handwritten character string is diagonal, the display direction of the handwritten character string differs from the direction of the adjusted character string (horizontal), so the display control unit 46 proceeds to step S54 and determines whether the character type recognized in step S50 is "alphabet only."

[0064] If the character type recognized in step S50 is "alphabetical characters only," the display control unit 46 proceeds to step S56, where it displays the character-recognized character string in the same direction as the display direction of the handwritten character string. If the character type recognized in step S50 is not "alphabetical characters only," the display control unit 46 proceeds to step S58, where it adjusts the display direction of the character-recognized character string to the horizontal direction and displays it.

[0065] By processing the flowchart of FIG. 17, when the character type in FIG. 18(a) is "alphabet only" and character string 124 is recognized from handwritten character string 120 that was manually input diagonally, display control unit 46 can display character string 124 in the same direction (same inclination) as handwritten character string 120, as shown in FIG. 18(b).

[0066] Furthermore, by processing the flowchart of FIG. 17, when the character type in FIG. 18(a) is "Hiragana only" and character string 126 is recognized from handwritten character string 122 that was manually input diagonally, display control unit 46 can display character string 126 adjusted horizontally, different from handwritten character string 120, as shown in FIG. 18(b).

[0067] As described above, according to the second embodiment, it is possible to set for each character type whether a character string obtained by character recognition from a handwritten character string input in any direction is displayed in the original direction of the handwritten character string or adjusted to the horizontal direction. Therefore, according to the second embodiment, it is possible to improve the degree of freedom in displaying a character string obtained by character recognition from a handwritten character string.

[0068] [Other embodiments] 19 , a submenu 130 may be displayed that accepts a user's specification of a rotation angle of the character string 132, for example, by performing an operation such as a long press on the character string 132 after character recognition, when the character string 132 after character recognition is displayed with the inclination of the handwritten character string unchanged or adjusted horizontally. The user can perform an operation to adjust the inclination of the character string 132 after character recognition by specifying the rotation angle of the character string 132 from the submenu 130. The display control unit 46 adjusts the inclination of the character string after character recognition and displays it in accordance with the operation to adjust the inclination of the character string 132 after character recognition received from the user.

[0069] In addition, in this embodiment, the accuracy of character recognition is improved by performing character recognition after adjusting the inclination of the handwritten character string in the horizontal direction, but in the case of a character string written vertically as in Figure 20(a), for example, the accuracy of character recognition may not be improved. Figure 20 is an image diagram of an example of a handwritten character string input by hand in vertical writing.

[0070] Therefore, as shown in Figure 20(b), the acute angle (angle 116) formed by the center line 102 of the character string and a line 114 that is vertical to the display unit 10 is set as the rotation angle of the handwritten character string. The character recognition / conversion unit 45 may compare the character recognition determination rates when angle 106 in Figure 20(a) is set as the rotation angle with when angle 116 in Figure 20(b) is set as the rotation angle, thereby determining whether the character string is written vertically or horizontally, and switching the direction in which the tilt of the recognized character string is adjusted.

[0071] Furthermore, in the case of a character string written vertically as in Fig. 20(a), the setting screen 1000 allows the user to enable the character string tilt adjustment function by checking a checkbox 1002 with a description such as "Display characters handwritten diagonally vertically" as shown in Fig. 25, and disable the character string tilt adjustment function by unchecking the checkbox. If the character string tilt adjustment function in the setting screen 1000 of Fig. 25 is enabled, the tilt of the character string written vertically by hand as in Fig. 20(b) is adjusted vertically after character recognition.

[0072] <Another example of input device configuration 1> Although the input device of this embodiment has been described as having a large touch panel, the input device is not limited to having a touch panel.

[0073] Fig. 21 is a diagram showing another example of the configuration of an input device. In Fig. 21, a projector 411 is installed on the upper side of a normal whiteboard 413. This projector 411 corresponds to the input device. The normal whiteboard 413 is not a flat panel display integrated with a touch panel, but a whiteboard on which a user directly writes by hand with a marker. The whiteboard may also be a blackboard, as long as it is flat and large enough to project an image.

[0074] The projector 411 has an ultra-short focus optical system and can project an image with little distortion from a distance of about 10 cm onto the whiteboard 413. This image may be transmitted from a PC connected wirelessly or via a wired connection, or may be stored in the projector 411.

[0075] The user writes by hand on the whiteboard 413 using a dedicated electronic pen 2501. The electronic pen 2501 has a light-emitting part, for example at the tip, that is switched on and emits light when the user presses it against the whiteboard 413 to write. The wavelength of the light is near-infrared or infrared, so it is invisible to the user's eyes. The projector 411 has a camera that captures an image of the light-emitting part, analyzes the image, and identifies the direction of the electronic pen 2501.

[0076] The electronic pen 2501 emits light and also emits sound waves, and the projector 411 calculates the distance based on the time it takes for the sound waves to reach the projector. The position of the electronic pen 2501 can be identified based on the direction and distance. A stroke is drawn (projected) at the position of the electronic pen 2501.

[0077] The projector 411 projects the menu 430, and when the user presses a button with the electronic pen 2501, the projector 411 identifies the pressed button based on the position of the electronic pen 2501 and the ON signal of the switch. For example, when the save button 431 is pressed, the projector 411 saves the strokes (a set of coordinates) handwritten by the user.

[0078] The projector 411 stores the handwritten information in a predetermined server 412 or USB memory 2600. The handwritten information is stored for each page. Since the information is stored as coordinates rather than as image data, the user can re-edit the information. However, in this embodiment, the menu 430 does not need to be displayed because operation commands can be called up by handwriting.

[0079] <Another example of input device configuration 2> 22 is a diagram showing another example of the configuration of the input device, which includes a terminal device 600, an image projection device 700A, and a pen motion detection device 810 as the input device.

[0080] The terminal device 600 is connected by wire to the image projection device 700A and the pen motion detection device 810. The image projection device 700A projects image data input by the terminal device 600 onto the screen 800.

[0081] The pen operation detection device 810 communicates with the electronic pen 820 and detects the operation of the electronic pen 820 in the vicinity of the screen 800. Specifically, the electronic pen 820 detects coordinate information indicating a point pointed by the electronic pen 820 on the screen 800 and transmits the coordinate information to the terminal device 600.

[0082] The terminal device 600 generates image data of a stroke image input by the electronic pen 820 based on the coordinate information received from the pen operation detection device 810, and causes the image projection device 700A to draw the stroke image on the screen 800.

[0083] Furthermore, the terminal device 600 generates superimposed image data that indicates a superimposed image obtained by combining the background image projected by the image projection device 700A and the stroke image input by the electronic pen 820.

[0084] <Another example 3 of input device configuration> 23 is a diagram showing an example of the configuration of an input device, which includes a terminal device 600, a display 800A, and a pen operation detection device 810.

[0085] The pen operation detection device 810 is placed near the display 800A, detects coordinate information indicating a point indicated by the electronic pen 820A on the display 800A, and transmits the coordinate information to the terminal device 600. In the example of Fig. 34, the electronic pen 820A may be charged by the terminal device 600 via a USB connector.

[0086] Based on the coordinate information received from the pen operation detection device 810, the terminal device 600 generates image data of the stroke image input by the electronic pen 820A, and displays the image data on the display 800A.

[0087] <Another example 4 of input device configuration> Fig. 24 is a diagram showing an example of the configuration of an input device. In the example of Fig. 24, the input device includes a terminal device 600 and an image projection device 700A.

[0088] The terminal device 600 performs wireless communication (Bluetooth (registered trademark) or the like) with the electronic pen 820B and receives coordinate information of a point indicated by the electronic pen 820B on the screen 800. Based on the received coordinate information, the terminal device 600 generates image data of a stroke image input by the electronic pen 820B and causes the image projection device 700A to project the stroke image.

[0089] Furthermore, the terminal device 600 generates superimposed image data that indicates a superimposed image obtained by combining the background image projected by the image projection device 700A and the stroke image input by the electronic pen 820.

[0090] As described above, the above-described embodiments can be applied to various system configurations.

[0091] <Other application examples> The best mode for carrying out the present invention has been described above using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0092] For example, although an electronic whiteboard has been described as an example in the present embodiment, the present invention can be suitably applied to any information processing device having a touch panel. Furthermore, devices having similar functions to an electronic whiteboard are also referred to as electronic whiteboards, electronic information boards, interactive boards, etc. Examples of information processing devices equipped with a touch panel include output devices such as PJs (Projectors), digital signage, HUDs (Head Up Display) devices, industrial machinery, imaging devices, sound collection devices, medical equipment, network appliances, notebook PCs (Personal Computers), mobile phones, smartphones, tablet devices, game consoles, PDAs (Personal Digital Assistants), digital cameras, wearable PCs, and desktop PCs.

[0093] In this embodiment, the coordinates of the pen tip are detected by detecting them on a touch panel, but the coordinates of the pen tip may also be detected by ultrasonic waves. The pen emits light and also emits ultrasonic waves, and the input device calculates the distance based on the time it takes for the ultrasonic waves to reach the object. The input device can identify the position of the pen based on the direction and distance. In this case, the projector draws (projects) the trajectory of the pen as a stroke.

[0094] Furthermore, the configuration examples in Figures 1 to 4 are divided according to main functions to facilitate understanding of the processing by the electronic whiteboard 1. The method of dividing the processing units and their names do not limit the present invention. The processing of the electronic whiteboard 1 can be divided into even more processing units depending on the processing content. Furthermore, it can also be divided so that one processing unit includes even more processing.

[0095] Furthermore, each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions.

[0096] The handwriting input unit 40 is an example of a handwriting input means in the claims. The display control unit 46 is an example of a display control means. The setting receiving unit 48 is an example of a setting receiving means. [Explanation of symbols]

[0097] 1. Electronic whiteboard 10 Display section 11 Touch Panel 12 Control Unit 40 Handwriting input section 41 Stroke division section 42 Data Storage Unit 43 Character centroid calculation part 44 String slope calculation part 45 Character Recognition and Conversion Unit 46 Display control unit 47 Network Communications Department 48 Settings Reception Section 51 Stroke storage section 52 String storage section 53 Setting information storage section 100 Centroid coordinates 102 Center line 104 Horizontal Lines 106 angle 108 Handwritten String 110, 112 string 1000 Settings Screen [Prior art documents] [Patent documents]

[0098] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-188512

Claims

1. handwriting input means for accepting handwriting input; a character recognition means for, when the time period during which no handwritten input has been made reaches or exceeds a threshold, performing character recognition on the handwritten character string that has been handwritten input up until that time period; a display control means for adjusting the inclination of the handwritten input character string in a horizontal direction and displaying the character string that has been recognized in a horizontal direction on a display unit, the display control means causing a change in inclination of the character string from the inclination before the adjustment to the inclination after the adjustment to be displayed on the display unit; An input device comprising:

2. The display control means displays the process of adjusting the inclination of the character string on the display unit using the handwritten character string input by hand before character recognition, and displays the character string after the inclination adjustment is completed.

2. The input device according to claim 1, wherein:

3. The display unit displays a character string with or without tilt adjustment based on a setting of whether or not to adjust the tilt of the recognized character string.

3. The input device according to claim 1 or 2, wherein:

4. The present invention further comprises a setting receiving means for receiving from a user setting information as to whether or not a character string recognized based on the handwritten input is to be displayed as a character string with an adjusted tilt; The setting receiving means receives, from a setting screen on which the input device can be set, setting information as to whether to adjust the inclination of the character string before displaying it on the display unit, or to display it on the display unit without adjusting the inclination of the character string.

4. The input device according to claim 1, wherein:

5. The present invention further comprises a setting receiving means for receiving from a user setting information as to whether or not a character string recognized based on the handwritten input is to be displayed as a character string with an adjusted tilt; the setting receiving means receives a setting of the setting information from a user in association with a character type of the character string; The display control means controls the display unit to display a character string with or without tilt adjustment according to the character type of the character string.

4. The input device according to claim 1, wherein:

6. a handwriting input step in which handwriting input means accepts handwriting input; a character recognition step in which, when the time during which no handwritten input has been made becomes equal to or greater than a threshold, the character recognition means performs character recognition on the handwritten character string that has been handwritten input immediately before that time; a display control step in which a display control means adjusts the inclination of the handwritten input character string in a horizontal direction and causes the recognized character string to be displayed horizontally on a display unit, the display control step displaying a change in inclination of the character string from the inclination before the adjustment to the inclination after the adjustment on the display unit; An input method comprising:

7. A program to be executed by an input device, a handwriting input step in which handwriting input means accepts handwriting input; a character recognition step in which, when the time during which no handwritten input has been made becomes equal to or greater than a threshold, the character recognition means performs character recognition on the handwritten character string that has been handwritten input immediately before that time; a display control step in which a display control means adjusts the inclination of the handwritten input character string in a horizontal direction and causes the recognized character string to be displayed horizontally on a display unit, the display control step displaying a change in inclination of the character string from the inclination before the adjustment to the inclination after the adjustment on the display unit; A program comprising:

Images