Display device, display method, program, and display system

Abstract

To suppress inconvenience caused by forced conversion into a predetermined character type.SOLUTION: A display device according to the present invention which performs character recognition on a handwritten input using a dictionary and converts the handwritten input into a predetermined character type, comprises: a handwriting recognition control unit that calculates accuracy information relating to a recognition result for each handwritten character; a handwriting recognition dictionary control unit that uses a first dictionary to recognize the handwritten character and convert it into a first character string of a first character type when the accuracy information exceeds a predetermined threshold, and uses a second dictionary including a second character type different from the first character type predetermined in the first dictionary to recognize the handwritten character and convert it into a second character string of the second character type when the accuracy information is less than or equal to the predetermined threshold; and a display control unit that displays, in a predetermined input field in which forced conversion is performed, the first character string of the first character type recognized using the first dictionary and the second character string of the second character type recognized using the second dictionary.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a display device, a display method, a program, and a display system. [Background technology]

[0002] 2. Description of the Related Art Display devices are known that utilize handwriting recognition technology to recognize characters in handwritten data, convert the data into a character string, and display the converted character string on a display.

[0003] A technology for translating a character string obtained by character recognition is known (see, for example, Patent Document 1). Patent Document 1 discloses a technology for character recognition (recognizing data handwritten by a user as Japanese) and translating the recognized Japanese character string into another language (English). Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional technologies have a problem in that when a user's handwritten input is forcibly converted to a predetermined character type, characters of other character types cannot be input. For example, among input fields pre-prepared on a display device, there is an input field in which only numbers are to be input, and handwritten input is forcibly converted to numbers. When numbers are handwritten in this input field, the display device only needs to convert them to the number that is closest to the handwritten data, so character recognition is more accurate than when non-numeric characters such as hiragana are also used as recognition candidates. However, there are cases in which a user wants to handwrite other character types (hiragana, kanji, alphabet, etc.) in addition to numbers in this input field. In this case, numbers are correctly recognized and converted, but character types other than numbers are forcibly converted to numbers, and are not converted to the character string intended by the user.

[0005] In view of the above-mentioned problems, the present invention provides a technique for suppressing the inconvenience of content handwritten by a user being forcibly converted into a predetermined character type. [Means for solving the problem]

[0006] In view of the above-described problems, the present invention provides a display device that performs character recognition on a handwritten input using a dictionary and converts the input into a predetermined character type, the display device including a handwritten input unit that receives handwritten input including a plurality of handwritten characters from a user and determines whether the received handwritten input has been entered into a predetermined input field for forced conversion; a handwritten recognition control unit that, when the received handwritten input has been entered into the predetermined input field for forced conversion, performs character recognition on each handwritten character included in the handwritten input using a first dictionary and calculates accuracy information related to the recognition result for each handwritten character; and, when the accuracy information exceeds a predetermined threshold, calculates accuracy information related to the recognition result for each handwritten character. The system includes a handwriting recognition dictionary control unit that uses a first dictionary to perform character recognition on the handwritten characters and converts them into a first character string of a first character type, and if the accuracy information is less than or equal to a predetermined threshold, uses a second dictionary including a second character type different from the first character type predetermined in the first dictionary to perform character recognition on the handwritten characters and converts them into a second character string of the second character type; and a display control unit that displays the first character string of the first character type character-recognized using the first dictionary and the second character string of the second character type character-recognized using the second dictionary in the predetermined input field to be forcibly converted. [Effects of the Invention]

[0007] The present invention can prevent the inconvenience of content handwritten by a user being forcibly converted into a predetermined character type. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 10 is a diagram showing an example of a tabular input form displayed on a display device. [Figure 2] FIG. 10 is a diagram for schematically explaining the number of recognition points. [Figure 3] The recognition score is shown when the display device recognizes the handwritten data 325 "dai". [Figure 4] FIG. 1 is a diagram showing an example of the arrangement of display devices. [Figure 5] FIG. 2 is a diagram illustrating an example of a hardware configuration of a display device. [Figure 6] FIG. 2 is an example of a functional block diagram illustrating functions of the display device in blocks. [Figure 7] 10A and 10B are diagrams illustrating an example of input field definition data for each input field of an input form. [Figure 8] 10A and 10B are diagrams illustrating an example of an operation guide and selectable candidates displayed by the operation guide. [Figure 9] 10 is a flowchart illustrating an example of a conversion method when characters other than numbers are handwritten in a forced conversion input field. [Figure 10] 10 is a diagram showing an example of conversion of handwritten data into a character string by the process of FIG. 9. FIG. [Figure 11] FIG. 10 is an example of a flowchart illustrating a method for converting characters other than numbers handwritten in a forced conversion input field into a character string by allowing the user to select from a plurality of character string candidates. [Figure 12] 12A to 12C are diagrams illustrating an example of conversion of handwritten data into a character string by the process of FIG. 11. [Figure 13] 10 is an example of a flowchart illustrating a method for selecting a dictionary based on the difference in recognition scores of different dictionaries to convert characters other than numbers when the characters are handwritten in a forced conversion input field. [Figure 14] 14 is a diagram showing an example of conversion of handwritten data into a character string by the process of FIG. 13. FIG. [Figure 15] FIG. 10 is an example of a flowchart illustrating a method for selecting a dictionary and reconverting characters when a character other than a number is handwritten in a forced conversion input field by a user performing a correction operation. [Figure 16] 16 is a diagram illustrating an example of conversion of handwritten data into a character string by the process of FIG. 15. FIG. [Figure 17] 10 is a flowchart illustrating an example of a conversion method when characters other than numbers are handwritten in a forced conversion input field in which forced conversion has been canceled by a mode switching gesture. [Figure 18] FIG. 18 is a diagram showing an example of conversion of handwritten data into a character string by the process of FIG. 17. [Figure 19]FIG. 2 is an example of a functional block diagram of a display device. [Figure 20] 10 is a flowchart illustrating an example of changing conditions based on the result of character recognition of handwritten data in a forced conversion input field. [Figure 21] FIG. 10 is a diagram showing an example of a determination result screen. [Figure 22] FIG. 1 is a diagram illustrating an example of a system configuration of a display system. [Figure 23] FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing system. [Figure 24] FIG. 2 is an example of a functional block diagram illustrating functions of the display system in blocks. [Figure 25] 10 is an example of a sequence diagram illustrating a process in which a display device and an information processing system communicate with each other to display a recognition result of handwritten data. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A display device and a display method performed by the display device will be described below as an example of an embodiment of the present invention with reference to the accompanying drawings.

[0010] <When only strings of a specific character type are converted> Referring to FIG. 1, a case where handwritten data is converted only into numbers as a predetermined character type will be described. FIG. 1 shows an example of a tabular input form 300 displayed on a display device. Each cell in the input form 300 is an input field. As an example, a display device is used in a workplace or on-site where a high level of urgency and flexible response are required. The input form 300 in FIG. 1(a) has a column with "Name" as the column header and a column with "Age" as the column header. Since only numbers are handwritten in the column with "Age" as the column header, this column is pre-designated as a forced conversion input field. A forced conversion input field (an example of a predetermined input field that is subjected to forced conversion) is an input field in which the display device 2 forcibly converts handwritten data into a character string of one or more limited character types without displaying character string candidates. A character type refers to the type of character. Examples of character types include, but are not limited to, numbers, hiragana, katakana, kanji, symbols, and alphabets. A character string is an object converted by character recognition of handwritten data. A character string can be represented by a character code. A character string is one or more characters, and characters include hiragana, katakana, kanji, alphabets, numbers, and symbols.

[0011] The display device performs character recognition of handwritten data 321 in the forced conversion input field 310 using only the dictionary 309 (recognition engine) for numbers. In other words, the display device converts handwritten data entered in any forced conversion input field 310 in a column with "age" as the column heading to numbers only. In FIG. 1(b), the handwritten data 321 is correctly converted into the character string "19" 322. In this way, limiting the types of characters to be converted improves the accuracy of character recognition. The high accuracy also saves the user the task of selecting a final character string from multiple character string candidates.

[0012] However, in other words, the user cannot enter characters other than numbers into a column with "Age" as the column heading. Therefore, even if a user needs to enter a character string other than numbers, the user cannot enter a character string other than the pre-set character type (numbers).

[0013] For example, as shown in Fig. 1(c), there may be a case where a user wants to input "in their 60s" in the forced conversion input field 301 of the column with "Age" as the column heading. The user inputs the handwritten data 302 of "in their 60s" into the forced conversion input field 301. The display device converts the handwritten "s" to the nearest digit, and the user cannot input the desired character. In Fig. 1(d), "in their 60s" is incorrectly converted to the character string 304 of "608".

[0014] <Overview of the recognition method of this embodiment> Therefore, in this embodiment, focusing on the fact that the display device has multiple dictionaries, the recognition scores (probabilities) calculated by each dictionary during character recognition are utilized. The recognition score represents the likelihood of converting to that character and is internally treated as a probability.

[0015] Fig. 2 is a diagram schematically explaining the recognition score. In Fig. 2(a), the user handwrites the handwritten data 323 of "in their 60s" in the forced conversion input field 301 of the column with "Age" as the column heading. The display device preferentially uses the first dictionary D1 initially set in the forced conversion input field 301. The first dictionary D1 is a dictionary that converts handwritten data to numbers. When the display device recognizes the handwritten data 323 of "6", for example, it calculates that the recognition score for being "6" is 70 points. Similarly, as shown in Fig. 2(b), when the display device recognizes the handwritten data 324 of "0", for example, it calculates that the recognition score for being "0" is 90 points. Thus, the display device calculates the recognition score each time it performs character recognition. Although Fig. 2 shows recognition of one character at a time, the display device recognizes the stroke data handwritten until the pen is lifted up collectively.

[0016] Since the first dictionary D1 only corresponds to numbers, when the display device uses the first dictionary D1 to recognize characters other than numbers, none of the numbers from 0 to 9 is a correct recognition result, so the recognition score is considered to be low. Fig. 3 shows the recognition score when the display device recognizes the handwritten data 325 of "s". In Fig. 3(a), when the display device recognizes "s", it recognizes it as "8", but the recognition score is 15 points.

[0017] When the recognition score using the first dictionary D1 initially set in the forced conversion input field 301 is below the threshold, the display device uses the second dictionary D2 that can recognize character types other than numbers. The second dictionary D2 can recognize not only numbers but also hiragana, katakana, Chinese characters, symbols, and alphabets. The second dictionary D2 does not need to be able to convert all of these, and it is sufficient if it can convert two or more of them. Also, the second dictionary D2 only needs to include character types different from the first dictionary D1. In FIG. 3(b), the display device correctly recognizes the handwritten data 325 of the handwritten character "代" as "代" and calculates a recognition score of 70 points.

[0018] In this way, when the recognition score by the first dictionary D1 is low, the display device recognizes using the second dictionary D2, so that when numbers are handwritten in the forced conversion input field 301, they can be recognized with high accuracy, and even if character types other than numbers that could not be input previously are input, they can be recognized by character recognition.

[0019] <Regarding terms> A series of operations where the user presses a pen on the touch panel and then continuously moves it and then releases it from the touch panel is called a stroke. Stroke data is information displayed on the touch panel based on the trajectory of coordinates input by the input means. The stroke data may be interpolated as appropriate.

[0020] An object displayed on the display based on the stroke data is called an object. Although an object means a target, in this embodiment, it means a display target, etc.

[0021] Handwritten data is an object composed of one or more stroke data and refers to a display target handwritten by the user.

[0022] The character string converted from handwritten data through character recognition may include not only text data, but also data displayed based on user operations, such as stamps displayed as marks, shapes such as circles and stars, and straight lines. Characters include numbers, alphabets, and symbols. Characters are also called text data.

[0023] An operation command is a command that instructs the execution of a specific process and is prepared for operating a display device. Examples of operation commands include editing, modification, and input / output operation commands.

[0024] Forced conversion refers to the process of converting handwritten data into a string of characters without displaying an operation guide. When no operation guide is displayed, it does not matter what character type is converted into. Furthermore, the character type recognized by forced conversion is not limited to one, and handwritten data may be converted into different character types (for example, numbers and letters).

[0025] An input field is an area on a screen displayed on a display where handwritten data or character strings are entered. The input field may be surrounded by a border, or may only have a ruled line. The input field may also be a range enclosed in parentheses. Multiple input fields may be arranged in a one-dimensional or two-dimensional table format.

[0026] The accuracy information is the likelihood of converting handwritten data into a character string. The accuracy information is, for example, the probability or likelihood of converting into the character string. The accuracy information is, for example, 0 to 100, but may be 0 to 1.0, etc. In this embodiment, the accuracy information will be described using the term recognition score.

[0027] A dictionary is information referenced for character recognition of handwritten data. In AI, a dictionary is sometimes called a model or a recognition engine. A dictionary is also a conversion table or correspondence table that converts handwritten data into character strings.

[0028] In this embodiment, even if a threshold is exemplified as a comparison target, the threshold is not limited to the exemplified value. Therefore, in this embodiment, for all thresholds, the terms "less than threshold" and "below threshold" have the same meaning, and the terms "over threshold" and "above threshold" have the same meaning. For example, when the threshold is 11, "below threshold" has the same meaning as "below threshold" when the threshold is 10. Furthermore, when the threshold is 10, "over threshold" has the same meaning as "above threshold" when the threshold is 11.

[0029] [First embodiment] <Configuration example> The overall configuration of the display device 2 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the arrangement of the display device 2. Fig. 4(a) shows, as an example of the display device 2, a display device 2 used as a horizontally long electronic whiteboard hung on a wall.

[0030] 4(a), a display 220 as an example of the display device 2 is provided on top of the display device 2. A user U can write characters or the like by hand on the display 220 using a pen 2500.

[0031] FIG. 4(b) shows a display device 2 used as a vertically long electronic whiteboard hung on a wall.

[0032] 4(c) shows the display device 2 placed flat on a desk 230. The display device 2 is about 1 cm thick, so there is no need to adjust the height of the desk even when it is placed flat on a regular desk. In addition, the user can easily move the display device 2.

[0033] The pen 2500 can input coordinates using electromagnetic induction, active electrostatic coupling, etc. The pen 2500 may also have functions such as pressure detection, tilt detection, and a hover function (displaying the cursor before the pen touches the screen).

[0034] <Device hardware configuration> Next, the hardware configuration of the display device 2 will be described with reference to Fig. 5. As shown in the figure, the display device 2 has the configuration of an information processing device or a computer. Fig. 5 is an example of a hardware configuration diagram of the display device 2. As shown in Fig. 5, the display device 2 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and an SSD (Solid State Drive) 204.

[0035] Of these, the CPU 201 controls the overall operation of the display device 2. The ROM 202 stores the CPU 201 and programs used to drive the CPU 201, such as an IPL (Initial Program Loader). The RAM 203 is used as a work area for the CPU 201.

[0036] The SSD 204 stores various data such as an OS and programs for the display device 2. This program may be an application program that runs on an information processing device equipped with a general-purpose OS (Windows (registered trademark), Mac OS (registered trademark), Android (registered trademark), iOS (registered trademark), etc.). In this case, the device is normally used as a general-purpose information processing device, but when a user runs the application program, the user can write by hand, just like on a dedicated device for the display device 2.

[0037] The display device 2 also includes a display controller 213, a touch sensor controller 215, a touch sensor 216, a display 220, a power switch 227, a tilt sensor 217, a serial interface 218, a speaker 219, a microphone 221, a wireless communication device 222, an infrared I / F 223, a power control circuit 224, an AC adapter 225, and a battery 226.

[0038] The display controller 213 controls and manages the screen display in order to output an output image to the display 220, etc. The touch sensor 216 detects that the pen 2500, the user's hand, etc. (the pen and the user's hand are input means) has come into contact with the display 220. The touch sensor 216 also receives a pen ID.

[0039] The touch sensor controller 215 controls the processing of the touch sensor 216. The touch sensor 216 inputs and detects coordinates. The method for inputting and detecting coordinates will be described below. For example, in the case of an optical type, two light receiving and emitting devices installed at both ends of the upper side of the display 220 emit a plurality of infrared rays parallel to the display 220. The two light receiving and emitting devices receive light that is reflected by a reflecting member installed around the periphery of the display 220 and returns along the same optical path as the light emitted by the light receiving element.

[0040] The touch sensor 216 outputs position information of the infrared rays emitted by the two light receiving and emitting devices that are blocked by the object to the touch sensor controller 215, and the touch sensor controller 215 identifies the coordinate position of the contact position of the object. The touch sensor controller 215 also has a communication unit 215a and can communicate wirelessly with the pen 2500. For example, when communication is based on a standard such as Bluetooth (registered trademark), a commercially available pen can be used. If one or more pens 2500 are registered in the communication unit 215a in advance, the user can communicate with the display device 2 without having to perform connection settings to connect the pen 2500 to the display device 2.

[0041] The power switch 227 is a switch for switching ON / OFF the power of the display device 2. The tilt sensor 217 is a sensor for detecting the tilt angle of the display device 2. It is mainly used to detect whether the display device 2 is being used in the installation state shown in FIG. 4(a), FIG. 4(b), or FIG. 4(c), and can automatically change the orientation of characters, etc. depending on the installation state.

[0042] The serial interface 218 is a communication interface with the outside, such as a USB. The serial interface 218 is used for inputting information from the outside, etc. The speaker 219 is used for outputting audio, and the microphone 221 is used for inputting audio. The wireless communication device 222 communicates with a terminal carried by the user, and relays a connection to the Internet, for example. The wireless communication device 222 communicates using Wi-Fi, Bluetooth (registered trademark), or the like, but any communication standard is acceptable. The wireless communication device 222 forms an access point, and when the user sets the SSID (Service Set Identifier) ​​and password they have obtained in the terminal they carry, they can connect to the access point.

[0043] Preferably, the wireless communication device 222 is provided with two access points. a. Access point → Internet b. Access point → Internal network → Internet Access point a is for users outside the company, and users cannot access the company network but can use the Internet. Access point b is for users inside the company, and users can use the company network and the Internet.

[0044] The infrared I / F 223 detects the display device 2 arranged adjacently. The infrared I / F 223 can detect only the display device 2 arranged adjacently by taking advantage of the rectilinearity of infrared rays. It is preferable to provide one infrared I / F 223 on each side, and it can detect in which direction another display device 2 is arranged relative to the display device 2. This expands the screen, making it possible to display handwritten information that was previously handwritten on the adjacent display device 2 (handwritten information on another page, with the area of ​​one display 220 being one page), etc.

[0045] The power supply control circuit 224 controls an AC adapter 225 and a battery 226, which are power sources for the display device 2. The AC adapter 225 converts AC common to commercial power sources into DC.

[0046] If the display 220 is so-called electronic paper, it consumes little or no power to maintain the image display, and therefore can be driven by a battery 226. This makes it possible to use the display device 2 for purposes such as digital signage even in places where it is difficult to connect to a power source, such as outdoors.

[0047] The display device 2 further includes a bus line 210. The bus line 210 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 201 shown in FIG.

[0048] The touch sensor 216 is not limited to an optical type, and may be a capacitive touch panel that identifies the contact position by detecting changes in capacitance. The touch sensor 216 may also be a resistive film type touch panel that identifies the contact position by detecting voltage changes across two opposing resistive films. The touch sensor 216 may also be an electromagnetic induction type touch panel that identifies the contact position by detecting electromagnetic induction generated when a contacting object touches the display unit, or various other detection means may be used. The touch sensor 216 may be of a type that does not require an electronic pen to detect whether or not the pen tip is touching. In this case, a fingertip or a pen-shaped stick can be used to perform a touch operation. The pen 2500 does not have to be a long, thin pen-shaped panel.

[0049] <About the device functions> Next, functions of the display device 2 will be described with reference to FIG. 6. FIG. 6 is an example of a functional block diagram showing functions of the display device 2 in blocks. The display device 2 includes a handwriting input unit 21, a display unit 22, a handwriting input display control unit 23, a candidate display timer control unit 24, a handwriting input storage unit 25, a handwriting recognition control unit 26, a handwriting recognition dictionary unit 27, a character string conversion control unit 28, a character string conversion dictionary unit 29, a predictive conversion control unit 30, a predictive conversion dictionary unit 31, an operation command recognition control unit 32, an operation command definition unit 33, a handwriting recognition dictionary control unit 34, a recognition point determination unit 35, and an input field definition unit 36. Each function of the display device 2 is realized by any of the components shown in FIG. 5 operating in response to an instruction from the CPU 201 in accordance with a program loaded from the SSD 204 onto the RAM 203.

[0050] The handwriting input unit 21 is realized by a touch sensor 216 or the like, and accepts handwritten input from the user. The handwriting input unit 21 converts the user's pen input d1 into pen operation data d2 (pen up, pen down, or pen coordinate data) and transmits it to the handwriting input display control unit 23. The pen coordinate data is periodically transmitted as discrete values, and coordinates between discrete values ​​are interpolated. As will be described later, the handwriting input unit 21 accepts handwritten input including multiple handwritten characters from the user, and determines whether the accepted handwritten input has been entered into a predetermined input field (for example, a forced conversion input field).

[0051] The display unit 22 is realized by a display 220 or the like, and displays handwritten objects and operation menus. The display unit 22 converts the drawing data d3 written to the video memory by the handwriting input display control unit 23 into data according to the characteristics of the display 220, and transmits the data to the display 220.

[0052] The handwriting input display control unit 23 performs overall control over handwriting input and display. The handwriting input display control unit 23 processes the pen operation data d2 from the handwriting input unit 21 and transmits it to the display unit 22 for display.

[0053] The candidate display timer control unit 24 is a timer for controlling the display of selectable candidates. The candidate display timer control unit 24 starts or stops the timer to generate the timing for starting the display of selectable candidates and the timing for erasing the display. The candidate display timer control unit 24 receives a timer start request d4 (which may also be a timer stop request) from the handwriting input display control unit 23, and transmits a timeout event d5 to the handwriting input display control unit 23.

[0054] The handwriting input saving unit 25 has a storage function for saving user data (handwritten objects / character string objects). The handwriting input saving unit 25 receives user data d6-1 from the handwriting input display control unit 23 and saves it in the handwriting input saving unit 25, and receives an acquisition request d6-2 from the handwriting input display control unit 23 and transmits the user data d7 saved in the handwriting input saving unit 25. The handwriting input saving unit 25 transmits position information d36 of the confirmed object to the operation command recognition control unit 32.

[0055] The handwriting recognition control unit 26 is a recognition engine that performs online handwriting recognition. Unlike a typical OCR (Optical Character Reader), it recognizes characters (not only Japanese but also multiple languages ​​such as English), numbers, symbols (%, $, &, etc.), shapes (lines, circles, triangles, etc.) in parallel with the user's pen operation. Various algorithms have been devised for recognition methods, but this embodiment uses known techniques and will not be described in detail.

[0056] The handwriting recognition control unit 26 receives the pen operation data d8-1 from the handwriting input display control unit 23, performs handwriting recognition, and stores handwriting-recognized character string candidates. The handwriting recognition control unit 26 also stores language string candidates converted from the handwriting-recognized character string candidates d12 using the handwriting recognition dictionary unit 27. Separately, when an acquisition request d8-2 is received from the handwriting input display control unit 23, the handwriting recognition control unit 26 transmits the stored handwriting-recognized character string candidates and language string candidates d9 to the handwriting input display control unit 23. As will be described later, when the accepted handwritten input is entered in a predetermined input field (for example, a forced conversion input field), the handwriting recognition control unit 26 performs character recognition on each handwritten character included in the handwritten input using a dictionary, and calculates accuracy information regarding the recognition result for each handwritten character.

[0057] The handwriting recognition dictionary unit 27 is dictionary data for language conversion in handwriting recognition. In this embodiment, the handwriting recognition dictionary unit 27 has a first dictionary and a second dictionary. The first dictionary only supports numbers (can convert only numbers). The second dictionary can recognize not only numbers, but also hiragana, katakana, kanji, symbols, and the alphabet. The second dictionary does not need to be able to convert into all of these, but only needs to be able to convert into two or more of these. The priority of the first and second dictionaries is set in the input field definition data. The handwriting recognition dictionary unit 27 receives handwriting recognition character string candidates d12 from the handwriting recognition control unit 26, converts them into linguistically likely language character string candidates d13, and transmits them to the handwriting recognition control unit 26. For example, in the case of Japanese, hiragana is converted into kanji or katakana.

[0058] The character string conversion control unit 28 controls the conversion of conversion character string candidates into character strings. A conversion character string is a character string that is likely to be generated by including a handwriting recognition character string or a language character string. The character string conversion control unit 28 receives the handwriting recognition character string and language character string candidates d11 from the handwriting recognition control unit 26, converts them into conversion character string candidates using the character string conversion dictionary unit 29, and stores them. When the character string conversion control unit 28 separately receives an acquisition request d14 from the handwriting input display control unit 23, it transmits the stored conversion character string candidates d15 to the handwriting input display control unit 23.

[0059] The character string conversion dictionary unit 29 stores dictionary data for character string conversion. The character string conversion dictionary unit 29 receives handwriting-recognized character strings and language character string candidates d17 from the character string conversion control unit 28, and transmits converted character string candidates d18 to the character string conversion control unit 28.

[0060] The predictive conversion control unit 30 receives the handwriting-recognized character string and language character string candidate d10 from the handwriting recognition control unit 26 and receives the converted character string candidate d16 from the character string conversion control unit 28, and converts each of them into a predicted character string candidate using the predictive conversion dictionary unit 31. A predicted character string candidate is a character string that is likely to be generated by including the handwriting-recognized character string, language string, or converted character string. If an acquisition request d19 is separately received from the handwriting input display control unit 23, the predictive conversion control unit 30 transmits the predicted character string candidate d20 to the handwriting input display control unit 23.

[0061] The predictive conversion dictionary unit 31 stores dictionary data for predictive conversion. The predictive conversion dictionary unit 31 receives handwritten recognition character strings, language character string candidates, and conversion character string candidates d21 from the predictive conversion control unit 30, and transmits predicted character string candidates d22 to the predictive conversion control unit 30.

[0062] The operation command recognition control unit 32 receives the handwriting recognition character string and language character string candidates d30 from the handwriting recognition control unit 26, receives the converted character string candidates d28 from the character string conversion control unit 28, and receives the predicted character string candidates d29 from the predictive conversion control unit 30. Then, the operation command recognition control unit 32 transmits operation command conversion requests d26 for each of the characters to the operation command definition unit 33, and receives operation command candidates d27 from the operation command definition unit 33. The operation command recognition control unit 32 holds the operation command candidates d27.

[0063] If the operation command conversion request d26 partially matches a predefined character string, the operation command definition unit 33 sends an operation command candidate d27 to the operation command recognition control unit 32.

[0064] Furthermore, the operation command recognition control unit 32 receives pen operation data d24-1 from the handwriting input display control unit 23, and transmits a position information acquisition request d23 of a confirmed object that was previously input and confirmed to the handwriting input storage unit 25. The operation command recognition control unit 32 stores the confirmed object designated by the pen operation data as a selected object (including position information). The operation command recognition control unit 32 identifies a selected object that satisfies the position of the pen operation data d24-1 and a predetermined criterion. When the operation command recognition control unit 32 separately receives an acquisition request d24-2 from the handwriting input display control unit 23, it transmits a selected object d25 identified as a candidate for the stored operation command to the handwriting input display control unit 23.

[0065] When converting handwritten data into a restricted character type, the handwriting recognition control unit 26 transmits a recognition score indicating the accuracy of the recognition result to the recognition score determination unit 35. The recognition score determination unit 35 acquires the recognition score d41 when characters are recognized using each dictionary from the handwriting recognition control unit 26. The recognition score determination unit 35 transmits a dictionary switching command d42 to the handwriting recognition control unit 26, indicating that the recognition score is lower than the threshold. The handwriting recognition control unit 26 transmits a switch request d43 to the handwriting recognition dictionary control unit 34 to switch to the second dictionary. The handwriting recognition dictionary control unit 34 transmits a switch request d45 to the handwriting recognition dictionary unit 27 to switch from the first dictionary to the second dictionary. The handwriting recognition dictionary control unit 34 transmits a signal d44 to the handwriting recognition control unit 26 indicating that the switch to the second dictionary has been completed. The handwriting recognition control unit 26 again causes the handwriting recognition dictionary unit 27 to recognize the handwritten data. If the accuracy information exceeds a predetermined threshold, the handwriting recognition dictionary control unit 34 uses a first dictionary to perform character recognition on the handwritten characters and converts them into a first string of characters of a first character type, and if the accuracy information is less than or equal to the predetermined threshold, the handwriting recognition dictionary control unit 34 uses a second dictionary including a second character type different from the first character type predetermined in the first dictionary to perform character recognition on the handwritten characters and converts them into a second string of characters of the second character type.

[0066] The handwriting input display control unit 23 transmits the converted character string with the highest final recognition score to the display unit 22, whereby character strings other than those of the limited character types are displayed on the display 220. The handwriting input display control unit 23 displays a first character string of a first character type that has been recognized using the first dictionary, and a second character string of a second character type that has been recognized using the second dictionary, in a predetermined input field (for example, a forced conversion input field).

[0067] The input field definition unit 36 ​​holds definition data relating to the input field, such as the dictionary to be used, whether it is a forced conversion input field, etc. The input field definition unit 36 ​​transmits the input field definition data d46 to the handwriting input display control unit 23.

[0068] <Examples of characters that will be forcibly converted> Fig. 7 is an example of input field definition data for each input field in input form 300. The input field definition data is information about each input field in input form 300. The input field definition data in Fig. 7 is stored for each input field. The input field ID is the identification information for the input field. The coordinates item indicates the coordinates of the input field on the display of the display device 2 (for example, the x and y coordinates of the upper left vertex). The size item is the length and width of the input field. The font item specifies the font of the text displayed in the input field. The font size item specifies the font of the text displayed in the input field. The font color item specifies the color of the text displayed in the input field. The font alignment item specifies the alignment of the text displayed in the input field horizontally (left, center, right) and vertically (top, center, bottom). The input field margin item specifies the amount of space between the text and the input field frame in large, medium, or small. The specified dictionary item specifies the conversion dictionary used to convert handwritten data entered in the input field. The user can improve recognition accuracy by registering dictionaries for each input field in advance. The first dictionary is the dictionary used with priority, and the second dictionary is the dictionary used mainly when the recognition score is below the threshold. The displayable characters item is the maximum number of characters that the input field can display. The string item is the string entered in the input field. The operation guide item defines whether to display an operation guide before displaying the text in the input field. An input field without this item hidden is a forced conversion input field.

[0069] <Example of selectable candidates> Next, referring to Fig. 8, an explanation will be given of the operation guide 500 that is displayed when converting handwritten data. The operation guide 500 is not displayed when handwriting is done in a forced conversion input field. Fig. 8 shows an example of the operation guide 500 and the selectable candidates 530 that the operation guide 500 displays. The operation guide 500 is displayed when the user inputs handwritten data 504 by handwriting (by lifting the pen). The operation guide 500 is displayed when the operation guide 500 is set to be displayed in the input field.

[0070] The operation guide 500 displays operation command candidates 510, handwriting-recognized character string candidates 506, conversion character string candidates 507, and character string / predictive conversion candidates 508. Note that the selectable candidates 530 are the operation command candidates 510, handwriting-recognized character string candidates 506, conversion character string candidates 507, and character string / predictive conversion candidates 508. Note that the selectable candidates 530 excluding the operation command candidates 510 are referred to as character string candidates 539.

[0071] Handwritten data 504 is a character string "file" handwritten by the user. A handwritten data rectangular area display 503 is displayed surrounding handwritten data 504. In FIG. 8, operation guide 500 is displayed after four characters have been entered, but operation guide 500 is displayed when the user stops writing. Therefore, the number of characters in handwritten data 504 can be any number.

[0072] The handwriting recognition character string candidates 506, conversion character string candidates 507, and character string / predictive conversion candidates 508 are arranged in descending order of probability (for convenience of drawing, only one of each is shown in FIG. 8). The handwriting recognition character string candidate 506, "file", is a candidate for the recognition result. In this example, the handwriting recognition control unit 26 correctly recognizes "file".

[0073] Conversion character string candidate 507 is a conversion character string candidate (e.g., a phrase containing "file") converted from the kana-kanji conversion result of "file" (e.g., "file"). Character string / predictive conversion candidate 508 is a predicted character string candidate converted from handwriting recognition character string candidate 506 or conversion character string candidate 507. In this example, "file server" is displayed, which is predicted from "file".

[0074] Operation command candidates 510 are predefined operation command candidates that are displayed according to the recognized characters. In the example of Fig. 8, the first character of the line, "■" 511, indicates that it is a candidate operation command. In Fig. 8, there are two pieces of command information that partially match the handwriting recognition character string candidate 506, "file," so two operation command candidates 510 are displayed. Operation command candidates 510 are displayed when command information containing the converted character string is found, so they may not always be displayed.

[0075] The operation header 520 has buttons 501, 509, 502, and 505. Button 501 accepts an operation to switch between predictive conversion and kana conversion. Button 502 operates the page of candidate display. In the example of FIG. 8, there are three candidate display pages, and the first page is currently displayed. Button 505 accepts an operation to erase the operation guide 500. When the user presses button 505, the handwriting input unit 21 accepts the operation, and the handwriting input display control unit 23 erases the display except for the handwritten data. Button 509 accepts an operation to erase all displays. When the user presses button 509, the handwriting input unit 21 accepts the operation, and the handwriting input display control unit 23 erases all displays shown in FIG. 8, including the handwritten data, allowing the user to start handwriting again from the beginning.

[0076] 8 is an example, and operation command candidates 510 may be displayed below character string candidates 539, or may be displayed in descending order of past operation history. The user may be able to set the display order.

[0077] <Action or Processing> 9 is a flowchart illustrating a conversion method when characters other than numbers are handwritten in the forced conversion input field. The process in FIG. 9 is started, for example, when handwriting input unit 21 receives handwritten data.

[0078] The handwriting input display control unit 23 accepts input into the forced conversion input field. The handwriting input unit 21 determines whether the handwriting input from the user is input into a predetermined input field (for example, a forced conversion input field) (S11). The handwriting input display control unit 23 determines whether the input is a character (S12). Character input refers to the input of stroke data that is the target of character recognition. For example, a long press with the pen 2500 or a gesture to select a displayed character string is not a character input.

[0079] If the determination in step S12 is Yes, the process proceeds to step S13, and if No, the process returns to step S11.

[0080] In step S13, the handwriting input display control unit 23 starts analyzing the stroke data (character recognition) and stores the stroke data (S13).

[0081] Next, the handwriting input display control unit 23 passes the stroke data to the handwriting recognition control unit 26 (S14).

[0082] The handwriting recognition control unit 26 performs character recognition using the first dictionary set in the handwriting recognition dictionary unit 27 (S15). The handwriting recognition control unit 26 transmits the recognition score obtained by the character recognition using the first dictionary to the recognition score determination unit 35.

[0083] The recognition score determination unit 35 determines whether the recognition score is equal to or greater than the threshold (or exceeds the threshold) (S16).

[0084] If the determination in step S16 is No, the process proceeds to step S17, and if the determination is Yes, the process proceeds to step S19.

[0085] In step S17, the recognition score determination unit 35 notifies the handwriting recognition control unit 26 that the recognition score is lower than the threshold and sends a dictionary switching command d42. This causes the handwriting recognition control unit 26 to send a switching request d43 to the handwriting recognition dictionary control unit 34. The handwriting recognition dictionary control unit 34 sends a switching request d45 to the handwriting recognition dictionary unit 27 to switch to the second dictionary. The handwriting recognition dictionary control unit 34 switches the first dictionary set in the handwriting recognition dictionary unit 27 to the second dictionary. The handwriting recognition control unit 26 performs character recognition using the second dictionary set in the handwriting recognition dictionary unit 27 (S17). The second dictionary is a general-purpose dictionary, so it can convert characters into not only numbers, but also hiragana, katakana, kanji, symbols, and the alphabet. The handwriting recognition control unit 26 obtains the recognition scores for each character type.

[0086] Next, handwriting recognition control unit 26 returns the character string with the highest recognition score as the recognition result to handwriting input display control unit 23. Handwriting input display control unit 23 passes this character string to display unit 22, which then displays the character string of the character type with the highest recognition score in the forced conversion input field (S18).

[0087] In step S19, handwriting recognition control unit 26 returns the character string with the highest recognition score among the character strings converted using the first dictionary to handwriting input display control unit 23. Handwriting input display control unit 23 passes this character string to display unit 22, which then displays the character string with the highest recognition score converted using the first dictionary in the forced conversion input field (S19).

[0088] FIG. 10 shows an example of conversion of handwritten data into a character string by the process of FIG. 9. In FIG. 10(a), handwritten data 302 "60s" is entered into a forced conversion input field 301. The handwriting recognition control unit 26 performed character recognition of the handwritten data 302 "60s" using the first dictionary, but the number of recognition points was below the threshold. Therefore, the handwriting recognition dictionary control unit 34 switches from the first dictionary to the second dictionary. The handwriting recognition control unit 26 performs character recognition of the handwritten data 302 "60s" using the second dictionary. As a result, as shown in FIG. 10(b), the data is correctly converted into a character string 303 "60s." Note that "60" is an example of a first character string, and "dai" is an example of a second character string.

[0089] 9 and 10, the display device 2 performs recognition using the second dictionary when the recognition score using the first dictionary is low, thereby enabling highly accurate recognition when numbers are handwritten in the forced conversion input field 301. The display device 2 can now also accommodate input of character types other than numbers, which were previously not possible to input.

[0090] <<User selection from string candidates with high recognition scores>> If multiple character string candidates 539 with high recognition scores are displayed when switching to the second dictionary, the user can reliably select the character string of the character type that he or she intends.

[0091] 11 is a flowchart illustrating a method for a user to select from a plurality of character string candidates when characters other than numbers are handwritten in the forced conversion input field. The explanation of FIG. 11 may mainly focus on the differences from FIG. 9. The processing of steps S21 to S27 may be the same as steps S11 to S17 in FIG.

[0092] In step S28, the handwriting recognition control unit 26 executes the following when the character is recognized by the second dictionary: The handwriting input display control unit 23 returns the multiple character strings with the highest recognition scores as the recognition results. The second dictionary calculates multiple recognition results together with the recognition scores in character recognition using this dictionary. The handwriting input display control unit 23 passes these multiple character strings to the display unit 22, which then displays multiple character string candidates 539 with high recognition scores on the display of the operation guide 500 (S28). The handwriting input display control unit 23 may select, for example, the top two from each of different character types as the multiple character string candidates 539 with high recognition scores. This makes it easier to display the character type that the user expects.

[0093] When the user selects character string candidate 539, handwriting input unit 21 accepts the pen coordinates and notifies handwriting input display control unit 23. Handwriting input display control unit 23 causes display unit 22 to display the character string identified by the pen coordinates from among the multiple character string candidates (S29).

[0094] The process of step S30 may be the same as step S19 in Fig. 9. That is, handwriting recognition control unit 26 returns the character string converted with the highest recognition score using the first dictionary to handwriting input display control unit 23. Handwriting input display control unit 23 passes this character string to display unit 22, which then displays the character string converted with the highest recognition score using the first dictionary in the forced conversion input field (S30).

[0095] 12A and 12B are diagrams illustrating an example of conversion of handwritten data into a character string by the process of FIG. 11. In FIG. 12A, handwritten data 302 "60s" is entered into a forced conversion input field 301. Since the recognition score of "60s" using the first dictionary is less than the threshold, handwriting recognition control unit 26 performs character recognition of "60s" using the second dictionary. As shown in FIG. 12B, a plurality of character string candidates 539 with high recognition scores using the second dictionary are displayed in operation guide 500.

[0096] For example, when the user selects "60s" from the character string candidate 539, the display unit 22 displays the character string 303 "60s" in the forced conversion input field 301, as shown in FIG. 12(c).

[0097] 11 and 12, when the recognition score from the first dictionary is low, the display device 2 uses the second dictionary for recognition and displays multiple character string candidates 539, allowing the user to select the character string they want to convert. This makes it possible to input character types other than numbers that could not be input before. Furthermore, numbers handwritten in the forced conversion input field 301 can be recognized with high accuracy.

[0098] <<Switching dictionaries by comparing recognition scores when recognizing characters using multiple dictionaries>> The recognition score determination unit 35 may compare the recognition scores obtained when characters are recognized using multiple dictionaries, rather than comparing the recognition score obtained using the first dictionary with a threshold value. Since the handwriting input display control unit 23 recognizes characters using multiple dictionaries in advance, it is possible to quickly display a character string recognized using the second dictionary.

[0099] 13 is a flowchart illustrating a method for selecting a dictionary based on the difference in recognition points between different dictionaries when characters other than numbers are handwritten in a forced conversion input field. The explanation of FIG. 13 may mainly focus on the differences from FIG. 9. The processing of steps S31 to S34 may be the same as steps S11 to S14 in FIG. 9.

[0100] In step S35, the handwriting recognition control unit 26 passes the recognition score P1 (an example of the first accuracy information) obtained by character recognition using the first dictionary and the recognition score P2 (an example of the second accuracy information) obtained by character recognition using the second dictionary to the recognition score determination unit 35 (S35). Such processing is possible by the handwriting recognition control unit 26 sending a switching request d43 from the first dictionary to the second dictionary to the handwriting recognition dictionary control unit 34 after character recognition by the first dictionary.

[0101] The handwriting recognition control unit 26 sends the recognition score P1 and the recognition score P2 to the recognition score determination unit 35. The recognition score determination unit 35 determines whether P1 < P2 is satisfied (S36). If the determination in step S36 is No, the process proceeds to step S37, and if Yes, the process proceeds to step S39.

[0102] In step S39, the recognition score determination unit 35 determines whether "P2 - P1" is greater than or equal to a threshold value (an example of the second threshold value) (S39). This determination is a determination that even if P1 < P2, if the difference between the two is not large, it is considered that P1 < P2 does not hold (the first dictionary is prioritized). If the determination in step S39 is No, the process proceeds to step S37, and if Yes, the process proceeds to step S40. The threshold value in step S39 is considered to be smaller than the threshold values in FIGS. 9 and 11.

[0103] In step S37, the recognition score determination unit 35 notifies the handwriting recognition control unit 26 that the recognition score by the first dictionary is the highest (No in S36), or that the recognition score by the second dictionary is the highest but there is no significant difference from the first dictionary (No in S39). The handwriting recognition control unit 26 adopts the character string with the highest recognition result by the first dictionary and returns it to the handwriting input display control unit 23 (S37).

[0104] Since the handwriting input display control unit 23 passes this character string to the display unit 22, the display unit 22 displays the character string of the character type with the highest recognition score in the forced conversion input field (S38).

[0105] In step S40, the recognition score determination unit 35 notifies the handwriting recognition control unit 26 that the recognition score using the second dictionary is the highest and is significantly different from the recognition score using the first dictionary (Yes in S39). The handwriting recognition control unit 26 adopts the character string that is the recognition result using the second dictionary and returns it to the handwriting input display control unit 23 (S40).

[0106] The handwritten input display control unit 23 passes this character string to the display unit 22, which then displays the character string of the character type with the highest recognition score according to the second dictionary in the forced conversion input field (S41).

[0107] FIG. 14 shows an example of conversion in which handwritten data is converted into a character string by the process of FIG. 13. In FIG. 14(a), handwritten data 302 "60s" is entered into a forced conversion input field 301. The handwriting recognition control unit 26 performs character recognition using both the first dictionary and the second dictionary, and adopts the character string recognized by the dictionary with the higher recognition score. This causes the second dictionary to be selected. Therefore, the display device 2 was able to correctly convert the character string into "60s" 303, as shown in FIG. 14(b).

[0108] 13 and 14, the display device 2 recognizes characters using both the first dictionary and the second dictionary, and if the recognition score using the second dictionary is higher than the recognition score using the first dictionary, it displays the character string recognized using the second dictionary, making it possible to input character types other than numbers that could not be input before. The display device 2 can recognize numbers handwritten in the forced conversion input field 301 with high accuracy.

[0109] <<When a user corrects a mistranslation in the first dictionary, retranslate it using the second dictionary>> If the character string recognized by the first dictionary is incorrectly converted, the display device 2 may accept a correction operation from the user and reconvert the character string using the second dictionary.

[0110] 15 is a flowchart illustrating a method for selecting a dictionary and reconverting characters when the user performs a correction operation when characters other than numbers are handwritten in the forced conversion input field. The explanation of FIG. 15 may mainly focus on the differences from FIG. 9. The processing of steps S51 to S55 may be the same as steps S11 to S15 in FIG. 9.

[0111] In step S56, handwriting recognition control unit 26 passes the recognition result (character string) with the highest recognition score obtained in character recognition using the first dictionary to handwriting input display control unit 23, and handwriting input display control unit 23 displays the character string on display unit 22 (S56). Display unit 22 displays the character string in the forced conversion input field.

[0112] The user looks at the displayed character string and determines whether it has been converted incorrectly. If it has been converted incorrectly, the user inputs a reconversion gesture in the forced conversion input field. The handwriting input unit 21 determines whether a reconversion gesture has been detected (S57). A reconversion gesture is a user operation that instructs reconversion of handwritten data. The reconversion gesture can be any operation, but examples include long pressing on the character string, circling it, crossing it out, tapping on the forced conversion input field, etc.

[0113] If the determination in step S57 is Yes, the process proceeds to step S58, and if No, the process in FIG. 15 ends.

[0114] In step S58, the handwriting input unit 21 determines that the reconversion gesture is a reconversion operation to correct an erroneous conversion, and the handwriting input display control unit 23 acquires from the handwriting input unit 21 a notification that a reconversion operation has been performed. The handwriting input display control unit 23 requests the handwriting recognition control unit 26 to reconvert the stored handwritten data. The handwriting recognition control unit 26 sends a request d43 to switch to the second dictionary to the handwriting recognition dictionary control unit 34. The handwriting recognition dictionary control unit 34 switches the first dictionary set in the handwriting recognition dictionary unit 27 to the second dictionary. As a result, the handwriting recognition control unit 26 reconverts the handwritten data using the second dictionary (character recognition) (S58). The handwriting input display control unit 23 may select, for example, the top two from each of different character types as the multiple character string candidates 539 with high recognition scores. This makes it easier to display the character type the user expects.

[0115] The handwriting recognition control unit 26 returns a plurality of character strings having high recognition scores in character recognition using the second dictionary as recognition results to the handwriting input display control unit 23. The handwriting input display control unit 23 passes these character strings to the display unit 22, and the display unit 22 displays on the display an operation guide 500 including a plurality of character string candidates 539 having high recognition scores (S59).

[0116] When the user selects a character string candidate 539 from the operation guide 500, the handwriting input unit 21 receives the pen coordinates and notifies the handwriting input display control unit 23. The handwriting input display control unit 23 causes the display unit 22 to display the character string identified by the pen coordinates from among the multiple character string candidates 539 (S60).

[0117] Figure 16 is a diagram illustrating an example of conversion of handwritten data into a character string by the process of Figure 15. In Figure 16(a), handwritten data 302 "60s" is entered into a forced conversion input field 301. The display unit 22 displays a character string 304 "608", which is an incorrect recognition result by the first dictionary, in the forced conversion input field 301 (Figure 16(b)).

[0118] Next, as shown in FIG. 16(c), the user presses and holds the pen 2500 on the forced conversion input field 301 to reconvert (reconversion gesture). The reconversion gesture is accepted for each recognition unit (in this case, the entire "60s"). When the handwriting input display control unit 23 receives from the handwriting input unit 21 a notification that a reconversion gesture has been detected, the handwriting recognition control unit 26 sends a dictionary switching request d43 to the handwriting recognition dictionary control unit 34. The handwriting recognition control unit 26 reconverts the handwritten data 302 using the second dictionary, and the display unit 22 displays character string candidates 539 with the highest recognition scores on the operation guide 500. In FIG. 16(d), multiple character string candidates 539 with high recognition scores using the second dictionary are displayed. When the user selects, for example, "60s" from the character string candidates 539, the display unit 22 displays the character string 303 "60s" in the forced conversion input field 301, as shown in FIG. 16(e).

[0119] 15 and 16, after conversion using the first dictionary, the display device 2 accepts a reconversion gesture by the user if the conversion is incorrect, and switches from the first dictionary to the second dictionary, thereby enabling input of character types other than numbers that could not be input before. Numbers handwritten in the forced conversion input field 301 can be recognized with high accuracy.

[0120] <<Cancel forced conversion by mode switching gesture>> The user can also change the forced conversion input field to a normal input field before handwriting. The normal input field is an input field in which characters are recognized by the second dictionary and the operation guide 500 is displayed.

[0121] 17 is a flowchart illustrating a conversion method when characters other than numbers are handwritten in a forced conversion input field in which forced conversion has been canceled by a mode switching gesture. The process in FIG. 17 is started, for example, when handwriting input unit 21 accepts handwritten data.

[0122] The handwriting input display control unit 23 accepts input into the forced conversion input field (S71). The handwriting input display control unit 23 determines whether the input is a character (S72). Character input refers to the input of stroke data that is the target of character recognition. For example, a long press or a gesture to select a displayed character string is not a character input.

[0123] If the determination in step S72 is Yes, the process proceeds to step S73, and if the determination is No, the process proceeds to step S78.

[0124] In step S73, the handwriting input display control unit 23 notifies the handwriting recognition control unit 26 of the character recognition using the first dictionary (S73).

[0125] In step S74, the handwriting input display control unit 23 starts analyzing the stroke data (character recognition) and stores the stroke data (S74).

[0126] Next, the handwriting input display control unit 23 passes the stroke data to the handwriting recognition control unit 26 (S75).

[0127] The handwriting recognition control unit 26 performs character recognition using the first dictionary set in the handwriting recognition dictionary unit 27. The handwriting recognition control unit 26 determines the character string with the highest recognition score among the recognition scores obtained in the character recognition using the first dictionary (S76).

[0128] The handwriting recognition control unit 26 returns the character string with the highest recognition score as the recognition result to the handwriting input display control unit 23. The handwriting input display control unit 23 passes this character string to the display unit 22, which then displays the character string of the character type with the highest recognition score in the forced conversion input field (S77).

[0129] Next, in step S78, the handwriting input unit 21 determines whether a mode switching gesture for switching dictionaries has been input (S78). A mode switching gesture is a user operation for canceling forced conversion set in the forced conversion input field. Canceling the forced conversion causes the second dictionary to be used, and the operation guide 500 is displayed. The mode switching gesture may be any operation, but examples include double tapping or long pressing the forced conversion input field.

[0130] If the determination in step S78 is Yes, the process proceeds to step S79, and if No, the process returns to step S71.

[0131] In step S79, the handwriting input display control unit 23 acquires information that a mode switching gesture has been input from the handwriting input unit 21 (S79). The handwriting input display control unit 23 notifies the handwriting recognition control unit 26 that forced conversion in the forced conversion input field has been canceled. As a result, the handwriting recognition control unit 26 transmits a request d43 for switching to the second dictionary to the handwriting recognition dictionary control unit 34. Note that the handwriting input display control unit 23 determines that the operation guide 500 should be displayed because forced conversion is not required.

[0132] The handwriting recognition dictionary control unit 34 transmits a request d45 to switch to the second dictionary to the handwriting recognition dictionary unit 27 (S80).

[0133] The handwriting input display control unit 23 starts analyzing the stroke data (character recognition) and stores the stroke data (S81).

[0134] Next, the handwriting input display control unit 23 passes the stroke data to the handwriting recognition control unit 26 (S82).

[0135] The handwriting recognition control unit 26 performs character recognition using the second dictionary set in the handwriting recognition dictionary unit 27. The handwriting recognition control unit 26 returns the top several character strings with the highest recognition scores in the second dictionary as the recognition results to the handwriting input display control unit 23 (S83).

[0136] The handwriting input display control unit 23 passes this character string to the display unit 22, which then displays on the display an operation guide 500 including a plurality of character string candidates 539 with high recognition scores (S84).

[0137] When the user selects character string candidate 539 from operation guide 500, handwriting input unit 21 receives the pen coordinates and notifies handwriting input display control unit 23. Handwriting input display control unit 23 causes display unit 22 to display the character string identified by the pen coordinates from among the multiple character string candidates (S85).

[0138] Fig. 18 shows an example of conversion in which handwritten data is converted into a character string by the process of Fig. 17. In Fig. 18(a), the user has input a mode switching gesture in the forced conversion input field 301, so a mode switching command 311 saying "Cancel forced conversion" is displayed in the operation guide 500. When the user presses the mode switching command 311, the display device 2 cancels the forced conversion. As a result, the display device 2 displays the operation guide 500 for the handwritten data 302 input in the forced conversion input field 301. The display device 2 also switches the dictionary set in the forced conversion input field 301 to the second dictionary.

[0139] Next, in FIG. 18(b), handwritten data 302 "60s" is entered into the forced conversion input field 301. The handwriting recognition control unit 26 performs character recognition for "60s" using the second dictionary, and the display unit 22 displays multiple character string candidates 539 with high recognition scores in the operation guide 500. As shown in FIG. 18(c), multiple character string candidates 539 with high recognition scores using the second dictionary are displayed. When the user selects, for example, "60s" 312 from the character string candidates 539, the display unit 22 displays the character string 303 "60s" in the forced conversion input field 301, as shown in FIG. 18(d).

[0140] 17 and 18, the display device 2 cancels forced conversion in the forced conversion input field in response to a mode switching gesture by the user, so that it can now also input character types other than numbers that could not be input before. Numbers handwritten in the forced conversion input field 301 can be recognized with high accuracy.

[0141] [Second embodiment] In this embodiment, a display device 2 that notifies a user of past recognition results and changes the threshold value in accordance with the content input by the user will be described.

[0142] In the first embodiment, the display device 2 calculates a recognition score for handwritten data using a preset first dictionary (for example, numbers). If the calculated recognition score is less than a predetermined threshold, the display device 2 recognizes the handwritten data using a second dictionary (for example, a dictionary that recognizes hiragana, katakana, kanji, symbols, alphabets, etc.) different from the first dictionary. In such a case, the display device 2 can correct the conditions for character recognition (for example, thresholds) by prompting the user to input feedback based on past recognition results.

[0143] Fig. 19 is a functional block diagram of a display device 2 of this embodiment. The display device 2 of Fig. 19 newly includes an analysis unit 37 and a display reception unit 38. The analysis unit 37 accumulates a history of recognition results obtained by character recognition of handwritten data in a forced conversion input field, and corrects the conditions (e.g., thresholds) for determining whether to use the first dictionary or the second dictionary in response to an input from the user. The display reception unit 38 displays a determination result screen 350 of Fig. 21 and receives user operations on the determination result screen 350.

[0144] 20 is a flowchart for changing the conditions based on the results of character recognition of handwritten data in the forced conversion input field. The process in FIG. 20 is performed, for example, every time a character string that has been character-recognized is entered in the forced conversion input field.

[0145] First, the analysis unit 37 checks the number of times that character recognition has been performed on handwritten data in a forced conversion input field (S501). Next, the analysis unit 37 determines whether the number of times of recognition in a specific input field is 10 or more (S502). 10 times is an example, and it may be 1 or more. If the determination in step S502 is No, the process returns to step S501.

[0146] If the determination in step S502 is Yes, the display receiving unit 38 displays the determination result screen 350 (S503). An example of the determination result screen 350 is shown in FIG.

[0147] The display receiving unit 38 determines whether or not it has received a "No" operation from the user (S504). That is, the display receiving unit 38 displays the determination result screen 350 and receives input from the user as to whether or not the recognition result is what the user desires. If the determination in step S504 is No, the process returns to step S501.

[0148] If the "No" operation is received (Yes in step S504), the analysis unit 37 changes the condition (for example, threshold value) for determining whether to use the first dictionary or the second dictionary from the next time onwards (S505).

[0149] Specifically, the analysis unit 37 lowers the threshold value used when switching to the second dictionary (FIGS. 9 and 11). By lowering the threshold value, switching to the second dictionary becomes less likely (recognition will be more likely to be performed using the first dictionary), and if the recognition result using the first dictionary is what the operator wants, it becomes easier for the operator to perform the operation he or she desires. In addition, the analysis unit 37 increases the threshold value used to compare the difference in recognition scores between the first dictionary and the second dictionary (FIG. 13). By increasing the threshold value, switching to the second dictionary becomes less likely (recognition will be more likely to be performed using the first dictionary), and similarly, it becomes easier for the operator to perform the operation he or she desires.

[0150] 21 displays a list of the recognition results of handwritten input into the forced conversion input field 301 for which the forced conversion function has been set for the past 10 times. The judgment result screen 350 has a judgment table 351. The judgment table 351 has the number, the date and time of handwriting, the target input field, and the recognition result. Of these, the recognition result is the result of character recognition of the handwritten data in the forced conversion input field.

[0151] Furthermore, the determination result screen 350 displays a note 352 saying "The last 10 recognition results for handwriting entered into an input field for which the forced conversion function is set are displayed. Are the recognition results what the operator intended?", a Yes button 353, and a No button 354. If necessary, the display receiving unit 38 may also display the handwritten data that was the target of recognition on the determination result screen 350.

[0152] If the user presses the Yes button 353 on the judgment result screen 350, the recognition result is what the user intended, so the analysis unit 37 does not correct the condition (threshold) for determining whether the dictionary to be used in the forced conversion input field 301, in which the forced conversion function is set, is the first dictionary or the second dictionary.

[0153] On the other hand, if the user presses the No button 354, the recognition result is not what the user intended, so the analysis unit 37 corrects the condition (threshold) for determining whether the dictionary to be used in the forced conversion input field 301 with the forced conversion function set should be the first dictionary or the second dictionary. The correction is performed so that it becomes easier to determine whether to use the second dictionary.

[0154] The method for correcting the conditions is not limited to this. Alternatively, when the user clicks the "No" button on the screen, the setting conditions, for example, a list of dictionaries may be displayed on the screen and the user may change the dictionary by directly entering the dictionary.

[0155] <Major Effects> According to this embodiment, in addition to the effects of the first embodiment, the threshold value for determining which dictionary the display device 2 should use for character recognition can be adjusted by user input.

[0156] [Third embodiment] In this embodiment, a display system in a system form in which an information processing system on a network performs processing such as handwriting recognition and returns the recognition result to the display device 2 will be described.

[0157] In the description of this embodiment, components or contents of the figures that are given the same symbols as in the first embodiment perform the same functions, so the description of components that have already been described may be omitted or only the differences may be described.

[0158] 22 is an example of a system configuration diagram of a display system 100. The display system 100 has a display device 2 and an information processing system 10 that can communicate with each other via a network N.

[0159] The display device 2 is placed in a facility such as a company, and is connected to a LAN or Wi-Fi installed within the facility. The information processing system 10 is placed, for example, in a data center. The display device 2 is connected to the Internet i via a firewall 8, and the information processing system 10 is also connected to the Internet i via a high-speed LAN or the like within the data center.

[0160] The display device 2 may be connected to the Internet i using wireless communication such as a telephone line network. In this case, the wireless communication may be 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), LTE (Long Term Evolution), WiMAX (Worldwide Interoperability for Microwave Access), or the like.

[0161] The information processing system 10 has one or more information processing devices, and the one or more information processing devices act as servers to provide services to the display device 2. A server is a computer or software that performs the function of providing information and processing results in response to requests from clients. As will be described later, the information processing system 10 receives pen coordinates from the display device 2 and transmits information required to display the operation guide 500 shown in FIG. 8 to the display device 2.

[0162] The configuration of the display device 2 may be the same as that of the first embodiment, but in this embodiment, it is sufficient to have a touch panel, a display, and a communication function. The display device 2 may include multiple computing devices configured to communicate with each other.

[0163] In this embodiment, a general information processing device such as a PC or tablet can execute a web browser or a dedicated application. The web browser or the dedicated application communicates with the information processing system 10. When the web browser is running, the user inputs or selects the URL of the information processing system 10 to connect the display device to the information processing system 10. The display device 2 executes a web application provided by the information processing system 10 in the web browser. A web application refers to software or a mechanism thereof that operates through cooperation between a program written in a programming language (e.g., JavaScript (registered trademark)) that runs on the web browser and a program on the web server side.

[0164] When the dedicated application runs, it connects to a pre-registered URL of the information processing system 10. Since the dedicated application has a program and a user interface, the program sends and receives necessary information to and from the information processing system 10 and displays it on the user interface.

[0165] The communication method may be a general-purpose communication protocol such as HTTP, HTTPs, or WebSocket, or a dedicated communication protocol.

[0166] <Hardware configuration example> The hardware configuration of the display device 2 may be the same as that shown in Fig. 5. In this embodiment, an example of the hardware configuration of the information processing system 10 will be described.

[0167] Fig. 23 is a hardware configuration diagram of the information processing system 10. As shown in Fig. 23, the information processing system 10 is constructed by a computer, and includes a CPU 601, a ROM 602, a RAM 603, an HD 604, an HDD (Hard Disk Drive) controller 605, a display 606, an external device connection I / F (Interface) 608, a network I / F 609, a bus line 610, a keyboard 611, a pointing device 612, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I / F 616.

[0168] Of these, the CPU 601 controls the overall operation of the information processing system 10. The ROM 602 stores programs, such as an IPL, used to drive the CPU 601. The RAM 603 is used as a work area for the CPU 601. The HD 604 stores various data, such as programs. The HDD controller 605 controls the reading and writing of various data from and to the HD 604 under the control of the CPU 601. The display 606 displays various information, such as a cursor, menus, windows, characters, or images. The external device connection I / F 608 is an interface for connecting various external devices. In this case, external devices include, for example, USB (Universal Serial Bus) memories and printers. The network I / F 609 is an interface for data communication using a communication network. The bus line 610 is an address bus, a data bus, or the like, for electrically connecting the components, such as the CPU 601, shown in FIG. 23.

[0169] The keyboard 611 has a plurality of keys for inputting characters, numbers, various instructions, etc. The pointing device 612 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 614 controls reading and writing of various data from a DVD-RW 613, which is an example of a removable recording medium. The DVD-RW 613 may be a DVD-R or the like. The media I / F 616 controls reading and writing (storing) of data from a recording medium 615, such as a flash memory.

[0170] <About the device functions> Next, functions of the display system 100 will be described with reference to Fig. 24. Fig. 24 is an example of a functional block diagram showing functions of the display system 100 in blocks. Note that the description of Fig. 24 will mainly focus on the differences from Fig. 6.

[0171] In this embodiment, the display device 2 has a display unit 22, a display control unit 41, a handwriting input unit 21, and a communication unit 42. Each function of the display device 2 is a function or means realized when any of the components shown in Fig. 5 operates in response to an instruction from the CPU 201 in accordance with a program loaded from the SSD 204 onto the RAM 203.

[0172] The function of the handwriting input unit 21 in this embodiment may be the same as that in the first embodiment. The handwriting input unit 21 converts the user's pen input d1 into pen operation data da (pen up, pen down, or pen coordinate data) and transmits it to the display control unit 41.

[0173] The display control unit 41 controls the display of the display device 2. First, the display control unit 41 performs interpolation calculations on the coordinates between the discrete values ​​of the pen coordinate data, which are discrete values, and converts the pen coordinate data from pen-down to pen-up into one stroke db, and transmits it to the display unit 22.

[0174] Furthermore, the display control unit 41 transmits the pen operation data dc to the communication unit 42 and acquires various display data dd from the communication unit 42. The display data includes information for displaying the operation guide 500 shown in FIG. 8 and the like. The display control unit 41 transmits the display data de to the display unit 22.

[0175] The communication unit 42 (an example of a first communication unit) transmits pen operation data dc to the information processing system 10, receives various display data dd from the information processing system 10, and transmits the data to the display control unit 41.

[0176] The function of the display unit 22 may be the same as in the first embodiment. The display unit 22 displays the stroke db and display data de. The display unit 22 converts the stroke db and display data de written to the video memory by the display control unit 41 into data according to the characteristics of the display 220 and transmits them to the display 220.

[0177] <<Functions of the information processing system>> The information processing system 10 includes a communication unit 43, a handwriting input display control unit 23, a candidate display timer control unit 24, a handwriting input storage unit 25, a handwriting recognition control unit 26, a handwriting recognition dictionary unit 27, a character string conversion control unit 28, a character string conversion dictionary unit 29, a predictive conversion control unit 30, a predictive conversion dictionary unit 31, an operation command recognition control unit 32, an operation command definition unit 33, an input field definition unit 36, a handwriting recognition dictionary control unit 34, and a recognition point determination unit 35. Each function of the information processing system 10 is realized by any of the components shown in Fig. 23 operating in response to an instruction from the CPU 601 in accordance with a program loaded from the HD 604 onto the RAM 603.

[0178] The communication unit 43 (an example of a second communication unit) receives pen operation data dc from the display device 2, transmits pen operation data df to the handwriting input display control unit 23, receives display data dd from the handwriting input display control unit 23, and transmits it to the display device 2.

[0179] Other functions are the same as those of the first embodiment, or even if they are different, they will not hinder the explanation of this embodiment.

[0180] <<Action or Processing>> FIG. 25 is a sequence diagram illustrating a process in which the display device 2 and the information processing system 10 communicate with each other to display the recognition result of handwritten data.

[0181] S101: The display device 2 repeats steps S101 to S103 until the pen is lifted. The communication unit 42 of the display device 2 transmits pen operation data (stroke data) to the information processing system 10.

[0182] S102: The communication unit 43 of the information processing system 10 receives the pen operation data (stroke data) and performs character recognition each time one piece of stroke data is added. The information processing system 10 performs the processes described in Figures 9, 11, 13, 15, and 17, and performs character recognition using at least one of the first dictionary and the second dictionary.

[0183] S103: The display control unit 41 of the display device 2 causes the display 220 to display the stroke data in parallel with the transmission of the pen operation data.

[0184] S104: When the user lifts up the pen, the communication unit 43 of the information processing system 10 detects the pen-up by not receiving any pen operation data for a certain period of time. The communication unit 42 may transmit a message to the information processing system 10 that the pen has been lifted.

[0185] S105: In the case of a conversion example in which the operation guide 500 is not displayed, as shown in Figures 9 and 13, when the communication unit 43 of the information processing system 10 detects that the pen has been lifted, the handwriting input display control unit 23 transmits the character string that it has stored to the display device 2.

[0186] S106: The communication unit 42 of the display device 2 receives the character string, and the display control unit 41 deletes the handwritten data and displays the character string in the forced conversion input field.

[0187] S107: In the case of a conversion example in which an operation guide 500 is displayed as shown in Figures 11 and 17, when the communication unit 43 of the information processing system 10 detects that the pen has been lifted, the handwriting input display control unit 23 transmits the stored character string candidate 539 to the display device 2.

[0188] S108: The communication unit 42 of the display device 2 receives the character string candidate 539, and the display control unit 41 displays the operation guide 500.

[0189] S109: When the character string candidate 539 is selected, the handwriting input unit 21 accepts it, and the display control unit 41 deletes the handwritten data and displays the selected character string candidate 539 as the recognition result.

[0190] Note that, when the display device 2 receives the reconversion gesture as shown in FIG. 15, after step S106, the reconversion gesture is transmitted to the information processing system 10, and the process proceeds to step S107.

[0191] <Major Effects> According to this embodiment, in addition to the effects of the first embodiment, if the display device 2 is connected to a network, the display device 2 can display a character string converted from a handwritten input.

[0192] [Other application examples] The best mode for carrying out the present invention has been described above using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0193] For example, instead of the second dictionary, the display device 2 may perform character recognition using dictionaries for numbers, hiragana, katakana, kanji, and symbols, and display the character string recognized using the dictionary with the highest recognition score.

[0194] In addition, instead of the second dictionary, the display device 2 may perform character recognition using dictionaries for numbers, hiragana, katakana, kanji, and symbols, and if the highest recognition score is less than a threshold, display candidate character string 539 and display the selected character string.

[0195] Furthermore, the character types to be forcibly converted are not limited to numbers, but may be character types such as hiragana, katakana, symbols, etc. In other words, the first dictionary may be dedicated to hiragana, katakana, or symbols.

[0196] In addition, although the present embodiment has been described with reference to a case where a dictionary is set in an input field, the present embodiment can also be suitably applied to a case where handwriting is performed in any position on the screen other than the input field. In this case, a first dictionary and a second dictionary are set for the screen.

[0197] For example, although an electronic whiteboard has been described as an example in this embodiment, any information processing device having a touch panel can be suitably applied. Furthermore, devices having similar functions to an electronic whiteboard are also called electronic whiteboards, electronic information boards, interactive boards, etc. An information processing device equipped with a touch panel may also be, for example, a display device using a projector.

[0198] In this embodiment, the display device 2 detects the coordinates of the pen tip by detecting the coordinates of the pen tip using a touch panel, but the coordinates of the pen tip may also be detected using ultrasonic waves. The pen emits light and also emits ultrasonic waves, and the display device 2 calculates the distance based on the arrival time of the ultrasonic waves. The display device 2 can identify the position of the pen based on the direction and distance. The projector draws (projects) the trajectory of the pen as a stroke.

[0199] In addition, the configuration examples in Figure 6 and the like are divided according to main functions to make it easier to understand the processing by the display device 2. The method of dividing the processing units and their names do not limit the present invention. The processing by the display device 2 can be divided into even more processing units depending on the processing content. Also, it can be divided so that one processing unit includes even more processes.

[0200] Furthermore, each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions. Embodiments of the present invention provide significant improvements in computer power and functionality. These improvements allow users to utilize computers that provide more efficient and robust interaction with tables, the methods by which information is stored and presented in information processing devices. Furthermore, embodiments of the present invention provide a better user experience through the use of more efficient, powerful, and robust user interfaces. Such user interfaces provide better interaction between humans and machines.

[0201] <Additional Notes> [Appendix 1] A display device that recognizes handwritten input characters using a dictionary and converts them into a predetermined character type, a handwriting input unit that receives a handwritten input including a plurality of handwritten characters from a user and determines whether the received handwritten input has been entered into a predetermined input field that is to be forcibly converted; a handwriting recognition control unit that, when the accepted handwritten input is entered into the predetermined input field to be forcibly converted, performs character recognition on each handwritten character included in the handwritten input using a first dictionary and calculates accuracy information on the recognition result for each handwritten character; If the certainty information exceeds a predetermined threshold, the handwritten characters are subjected to character recognition using the first dictionary and converted into a first character string of a first character type; a handwriting recognition dictionary control unit that, when the accuracy information is less than or equal to a predetermined threshold, performs character recognition on the handwritten character using a second dictionary including a second character type different from the first character type predetermined in the first dictionary, and converts the handwritten character into a second character string of the second character type; a display control unit that displays a first character string of the first character type that has been character-recognized using the first dictionary and a second character string of the second character type that has been character-recognized using the second dictionary in the predetermined input field to be forcibly converted; A display device having: [Appendix 2] If the certainty information is less than or equal to a predetermined threshold, the handwriting recognition control unit displays a plurality of character string candidates obtained by character recognition using the second dictionary in a selectable manner; The display device according to claim 1, wherein the display control unit displays the character string candidate selected by the user as the second character string in the specified input field to be forcibly converted. [Appendix 3] If the certainty information is not less than or equal to a predetermined threshold, 3. The display device according to claim 1, wherein the display control unit displays the first character string recognized by the first dictionary in the predetermined input field to be forcibly converted. [Appendix 4] the handwriting recognition control unit performs character recognition of the handwritten input using the first dictionary and the second dictionary, and calculates first accuracy information and second accuracy information; When the difference obtained by subtracting the first accuracy information from the second accuracy information is equal to or greater than a second threshold, The display device according to claim 1, wherein the handwriting recognition control unit displays the second character string recognized by the second dictionary in the predetermined input field to be forcibly converted. [Appendix 5] If the difference obtained by subtracting the first accuracy information from the second accuracy information is not equal to or greater than a second threshold value, The display device according to claim 4, wherein the handwriting recognition control unit displays the first character string recognized by the first dictionary in the predetermined input field to be forcibly converted. [Appendix 6] When the display control unit displays the first character string recognized by the first dictionary in the predetermined input field to be forcibly converted and receives a reconversion gesture for the first character string, the handwriting recognition control unit displays a plurality of character string candidates obtained by character recognition using the second dictionary in a selectable manner; The display device according to claim 1, wherein the display control unit displays the character string candidate selected by the user as the second character string in the specified input field to be forcibly converted. [Appendix 7] When the handwriting input unit receives an operation to cancel forced conversion for the predetermined input field to be forcibly converted before receiving handwriting input from a user for the predetermined input field to be forcibly converted, the handwriting recognition control unit performs character recognition on the handwritten input from the user into the predetermined input field to be forcibly converted using the second dictionary without using the first dictionary; displaying a plurality of character string candidates obtained by character recognition using the second dictionary in a selectable manner; The display device according to claim 1, wherein the display control unit displays the character string candidate selected by the user as the second character string in the specified input field to be forcibly converted. [Appendix 8] 8. The display device according to claim 1, wherein the first dictionary is a dictionary that converts the handwritten input into numbers only. [Appendix 9] 10. The display device according to any one of appendices 1 to 9, wherein the second dictionary is a dictionary that converts the handwritten input into two or more of numbers, hiragana, katakana, kanji, symbols, and alphabets. [Appendix 10] an analysis unit that accumulates a recognition result in which the handwritten input handwritten in the predetermined input field to be forcibly converted is converted into a character string, and analyzes the recognition result; a display receiving unit that displays the recognition result by the analyzing unit and receives a change of the threshold value that determines whether to use the second dictionary instead of the first dictionary based on the recognition result; 2. The display device according to claim 1, further comprising: [Appendix 11] an analysis unit that accumulates a recognition result in which the handwritten input handwritten in the predetermined input field to be forcibly converted is converted into a character string, and analyzes the recognition result; a display receiving unit that displays the recognition result by the analysis unit and receives, based on the recognition result, a change of the second threshold value that is compared with the difference; 6. The display device according to claim 5, further comprising: [Explanation of symbols]

[0202] 2 Display device [Prior art documents] [Patent documents]

[0203] [Patent Document 1] Patent No. 3933449

Claims

1. A display device that recognizes handwritten input characters using a dictionary and converts them into a predetermined character type, a handwriting input unit that receives a handwritten input including a plurality of handwritten characters from a user and determines whether the received handwritten input has been entered into a predetermined input field that is to be forcibly converted; a handwriting recognition control unit that, when the accepted handwritten input is entered into the predetermined input field to be forcibly converted, performs character recognition on each handwritten character included in the handwritten input using a first dictionary and calculates accuracy information on the recognition result for each handwritten character; If the certainty information exceeds a predetermined threshold, the handwritten characters are subjected to character recognition using the first dictionary and converted into a first character string of a first character type; a handwriting recognition dictionary control unit that, when the accuracy information is less than or equal to a predetermined threshold, performs character recognition on the handwritten character using a second dictionary including a second character type different from the first character type predetermined in the first dictionary, and converts the handwritten character into a second character string of the second character type; a display control unit that displays a first character string of the first character type that has been character-recognized using the first dictionary and a second character string of the second character type that has been character-recognized using the second dictionary in the predetermined input field to be forcibly converted; A display device having:

2. If the certainty information is less than or equal to a predetermined threshold, the handwriting recognition control unit displays a plurality of character string candidates obtained by character recognition using the second dictionary in a selectable manner; The display device according to claim 1 , wherein the display control unit displays the character string candidate selected by the user as the second character string in the predetermined input field to be forcibly converted.

3. If the certainty information is not less than or equal to a predetermined threshold, The display device according to claim 1 , wherein the display control unit displays the first character string recognized by the first dictionary in the predetermined input field to be forcibly converted.

4. the handwriting recognition control unit performs character recognition of the handwritten input using the first dictionary and the second dictionary, and calculates first accuracy information and second accuracy information; When a difference obtained by subtracting the first certainty information from the second certainty information is equal to or greater than a second threshold, The display device according to claim 1 , wherein the handwriting recognition control unit displays the second character string recognized by the second dictionary in the predetermined input field to be forcibly converted.

5. If the difference obtained by subtracting the first accuracy information from the second accuracy information is not equal to or greater than a second threshold value, The display device according to claim 4 , wherein the handwriting recognition control unit displays the first character string recognized by the first dictionary in the predetermined input field to be forcibly converted.

6. When the display control unit displays the first character string recognized by the first dictionary in the predetermined input field to be forcibly converted and receives a reconversion gesture for the first character string, the handwriting recognition control unit displays a plurality of character string candidates obtained by character recognition using the second dictionary in a selectable manner; The display device according to claim 1 , wherein the display control unit displays the character string candidate selected by the user as the second character string in the predetermined input field to be forcibly converted.

7. When the handwriting input unit receives an operation to cancel forced conversion for the predetermined input field to be forcibly converted before receiving handwriting input from a user for the predetermined input field to be forcibly converted, the handwriting recognition control unit performs character recognition on the handwritten input from the user into the predetermined input field to be forcibly converted using the second dictionary without using the first dictionary; displaying a plurality of character string candidates obtained by character recognition using the second dictionary in a selectable manner; The display device according to claim 1 , wherein the display control unit displays the character string candidate selected by the user as the second character string in the predetermined input field to be forcibly converted.

8. The display device according to claim 1 , wherein the first dictionary is a dictionary that converts the handwritten input into only numbers.

9. The display device according to claim 1 , wherein the second dictionary converts the handwritten input into two or more of numbers, hiragana, katakana, kanji, symbols, and alphabets.

10. an analysis unit that accumulates a recognition result in which the handwritten input handwritten in the predetermined input field to be forcibly converted is converted into a character string, and analyzes the recognition result; a display receiving unit that displays the recognition result by the analyzing unit and receives a change of the threshold value that determines whether to use the second dictionary instead of the first dictionary based on the recognition result; The display device according to claim 1 , further comprising:

11. an analysis unit that accumulates a recognition result in which the handwritten input handwritten in the predetermined input field to be forcibly converted is converted into a character string, and analyzes the recognition result; a display receiving unit that displays the recognition result by the analysis unit and receives, based on the recognition result, a change of the second threshold value that is compared with the difference; The display device according to claim 5 , further comprising:

12. A display method performed by a display device that recognizes handwritten input into a predetermined input field using a dictionary and converts the input into a predetermined character type, a process of accepting handwritten input including a plurality of handwritten characters from a user and determining whether the accepted handwritten input has been entered into a predetermined input field that is to be forcibly converted; a process of, when the accepted handwritten input is entered into the predetermined input field to be forcibly converted, performing character recognition on each handwritten character included in the handwritten input using a first dictionary, and calculating accuracy information on the recognition result for each handwritten character; If the certainty information exceeds a predetermined threshold, the handwritten characters are subjected to character recognition using the first dictionary and converted into a first character string of a first character type; When the accuracy information is less than or equal to a predetermined threshold, a process of performing character recognition on the handwritten character using a second dictionary including a second character type different from the first character type predetermined in the first dictionary and converting the handwritten character into a second character string of the second character type; a process of displaying a first character string of the first character type that has been character-recognized using the first dictionary and a second character string of the second character type that has been character-recognized using the second dictionary in the predetermined input field to be forcibly converted; Display method.

13. A display device that recognizes handwritten input characters using a dictionary and converts them into a predetermined character type, a handwriting input unit that receives a handwritten input including a plurality of handwritten characters from a user and determines whether the received handwritten input has been entered into a predetermined input field that is to be forcibly converted; a handwriting recognition control unit that, when the accepted handwritten input is entered into the predetermined input field to be forcibly converted, performs character recognition on each handwritten character included in the handwritten input using a first dictionary and calculates accuracy information on the recognition result for each handwritten character; If the certainty information exceeds a predetermined threshold, the handwritten characters are subjected to character recognition using the first dictionary and converted into a first character string of a first character type; a handwriting recognition dictionary control unit that, when the accuracy information is less than or equal to a predetermined threshold, performs character recognition on the handwritten character using a second dictionary including a second character type different from the first character type predetermined in the first dictionary, and converts the handwritten character into a second character string of the second character type; a display control unit that displays a first character string of the first character type that has been character-recognized using the first dictionary and a second character string of the second character type that has been character-recognized using the second dictionary in the predetermined input field to be forcibly converted; A program to function as a

14. A display system in which a display device and an information processing system that recognizes handwritten input characters using a dictionary and converts them into a predetermined character type communicate with each other via a network, The display device includes: a handwriting input unit that receives a handwritten input including a plurality of handwritten characters from a user and determines whether the received handwritten input has been entered into a predetermined input field that is to be forcibly converted; a first communication unit that transmits the handwriting input accepted by the handwriting input unit to the information processing system, The information processing system includes: a handwriting recognition control unit that, when a handwritten input received from the display device is entered into the predetermined input field to be forcibly converted, performs character recognition on each handwritten character included in the handwritten input using a first dictionary and calculates accuracy information on the recognition result for each handwritten character; If the certainty information exceeds a predetermined threshold, the handwritten characters are subjected to character recognition using the first dictionary and converted into a first character string of a first character type; a handwriting recognition dictionary control unit that, when the accuracy information is less than or equal to a predetermined threshold, performs character recognition on the handwritten character using a second dictionary including a second character type different from the first character type predetermined in the first dictionary, and converts the handwritten character into a second character string of the second character type; a second communication unit that transmits the first character string of the first character type and the second character string of the second character type to the display device, A display system characterized in that the display device displays the received first character string of the first character type and the second character string of the second character type in the specified input field that is forcibly converted.

Images