Handwriting input device, handwriting input system, handwriting input method, and program

Abstract

To obtain a result of character recognition or predictive search desired by a user.SOLUTION: A handwriting input device capable of handwriting input comprises: conversion means of converting a handwritten character string input by handwriting into a recognition result character string through character recognition; search means of searching for a prediction candidate character string corresponding to the recognition result character string; display control means of displaying the prediction candidate character string on a screen in a selectable manner; and expansion means of expanding a search range of the prediction candidate character string by the search means when a handwriting speed of the handwritten character string is lower than a predetermined threshold or when a distance between adjacent characters in the handwritten character string is larger than a predetermined threshold. The display control means displays the prediction candidate character string searched by the search means using the expanded search range on the screen in a selectable manner.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a handwriting input device, a handwriting input system, a handwriting input method, and a program. [Background technology]

[0002] Patent Document 1 discloses a technology in which, when a handwriting input device receives handwritten input from a user, the handwriting input device performs character recognition and predictive search on the handwritten input content, and displays candidate character strings that have been recognized and predicted on the screen. Summary of the Invention [Problem to be solved by the invention]

[0003] In a handwriting input device, the user's handwriting input actions may vary depending on the content of the handwriting input. For example, when a user is inputting a word by hand while remembering the word they want to write, the handwriting input actions may become slow. Also, when a user is inputting a sentence by hand and cannot remember a word such as a person's name or a proper noun, the user may input the sentence by hand while leaving blank spaces.

[0004] However, the conventional technology does not anticipate changing the predictive search method in response to changes in the user's handwriting input behavior. For example, even if the user's handwriting input behavior slows down or the distance between the characters handwritten by the user increases, the predictive search search range remains the same, which may result in the word the user wants to handwrite not being found in the search results within the predictive search search range. As a result, the user is unable to obtain the predictive search results they desire.

[0005] In view of the above-mentioned problems, an object of the present invention is to obtain predictive search results that a user desires. [Means for solving the problem]

[0006] In view of the above problems, a handwriting input device according to one embodiment is a handwriting input device capable of handwriting input, and includes: a conversion means for converting a handwritten character string into a recognition result character string by performing character recognition on the handwritten character string; a search means for searching for predicted candidate character strings corresponding to the recognition result character string; a display control means for displaying the predicted candidate character strings on a screen in a selectable manner; and an expansion means for expanding a search range for predicted candidate character strings by the search means when the handwriting speed of the handwritten character string is lower than a predetermined threshold or when the distance between adjacent characters in the handwritten character string is greater than a predetermined threshold; and the display control means displays the predicted candidate character strings searched by the search means using the expanded search range in a selectable manner on the screen. [Effects of the Invention]

[0007] According to the handwriting input device of the embodiment, the user can obtain the predictive search results that he or she desires. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the overall configuration of a handwriting input device according to a first embodiment; [Figure 2] FIG. 1 is a diagram showing an example of a hardware configuration of a handwriting input device according to a first embodiment; [Figure 3] FIG. 1 is a diagram showing an example of the functional configuration of a handwriting input device according to a first embodiment; [Figure 4] FIG. 1 is a diagram showing an example of a processing sequence by the handwriting input device according to the first embodiment; [Figure 5] 1 is a flowchart showing an example of a procedure of an extension process by an extension unit included in the handwriting input device according to the first embodiment; [Figure 6] FIG. 10 is a diagram showing a specific example (first example) of a determination process by an extension unit included in the handwriting input device according to the first embodiment; [Figure 7] FIG. 10 is a diagram showing a specific example (second example) of a determination process by the extension unit included in the handwriting input device according to the first embodiment; [Figure 8] FIG. 10 is a diagram showing a specific example (third example) of a determination process by the extension unit included in the handwriting input device according to the first embodiment; [Figure 9] FIG. 10 is a diagram showing a specific example (fourth example) of a determination process by the extension unit included in the handwriting input device according to the first embodiment; [Figure 10] FIG. 1 is a diagram showing a display example (first example) of predicted candidate character strings by a handwriting input display control unit included in the handwriting input device according to the first embodiment; [Figure 11] FIG. 10 is a diagram showing a display example (second example) of predicted candidate character strings by a handwriting input display control unit included in the handwriting input device according to the first embodiment; [Figure 12] FIG. 10 is a diagram showing a display example (third example) of predicted candidate character strings by the handwriting input display control unit included in the handwriting input device according to the first embodiment; [Figure 13] FIG. 10 is a diagram showing a display example (fourth example) of predicted candidate character strings by the handwriting input display control unit included in the handwriting input device according to the first embodiment; [Figure 14] FIG. 10 is a diagram showing a display example (fifth example) of predicted candidate character strings by the handwriting input display control unit included in the handwriting input device according to the first embodiment; [Figure 15] 10 is a flowchart showing an example of a procedure for a blank space determination process performed by an extension unit included in the handwriting input device according to the first embodiment; [Figure 16] FIG. 10 is a diagram showing an example of the functional configuration of a handwriting input device according to a second embodiment; [Figure 17] FIG. 10 is a diagram showing an example of the functional configuration of a handwriting input device according to a third embodiment; [Figure 18] 10 is a flowchart showing an example of a procedure for setting condition change processing by a handwriting input device according to a third embodiment. [Figure 19] FIG. 13 is a diagram showing an example of a display screen displayed by a display accepting unit included in the handwriting input device according to the fifth embodiment; DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, as an example of an embodiment of the present invention, a handwriting input device 100 and a handwriting input method performed by the handwriting input device 100 will be described with reference to the drawings.

[0010] [First embodiment] (Overall configuration of handwriting input device 100) Fig. 1 is a diagram showing an example of the overall configuration of a handwriting input device 100 according to the first embodiment. In the example shown in Fig. 1, the handwriting input device 100 is a so-called electronic whiteboard. Note that, although the handwriting input device 100 in the example shown in Fig. 1 is freestanding, it is not limited to this and may be wall-mounted, placed flat, or the like.

[0011] 1, the handwriting input device 100 has a display 120. A user U can input (draw) characters and the like by handwriting on the display 120 using a pen 10.

[0012] Here, the handwriting input device 100 can convert the handwritten character string into a recognition result character string (data that can be processed as a character string by a computer and has a character code for each character) by performing character recognition on the handwritten character string handwritten and input by the user U using the pen 10. Then, the handwriting input device 100 can search for a predicted candidate character string corresponding to the recognition result character string and display the recognition result character string and the predicted candidate character string on the display 120.

[0013] In this embodiment, a pen 10 is used as an example of an "input means." In addition, in this embodiment, a so-called active pen that has a built-in power source and can send commands to the handwriting input device 100 is used as the pen 10. The pen 10 has one physical switch at the tip, one at the end, and two on the side of the pen. The switch at the tip is for writing. The switch at the end is for erasing. The switch on the side of the pen is for allocating user functions.

[0014] (An example of the hardware configuration of the handwriting input device 100) 2 is a diagram showing an example of a hardware configuration of the handwriting input device 100 according to the first embodiment. As shown in FIG. 2, the handwriting input device 100 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, and an SSD (Solid State Drive) 104.

[0015] The CPU 101 controls the overall operation of the handwriting input device 100. The ROM 102 stores the CPU 101 and programs used to drive the CPU 101, such as an IPL (Initial Program Loader). The RAM 103 is used as a work area for the CPU 101. The SSD 104 stores various data, such as an OS and programs for the handwriting input device 100. Note that this program may be an application program that runs on an information processing device equipped with a general-purpose OS (Windows (registered trademark), Mac OS (registered trademark), Android (registered trademark), iOS (registered trademark), etc.). In other words, the handwriting input device 100 may be an information processing device.

[0016] The handwriting input device 100 also includes a display controller 113 , a touch sensor controller 115 , a touch sensor 116 , a display 120 , a power switch 122 , and a bus line 110 .

[0017] The display controller 113 controls and manages the screen display in order to output an output image to the display 120, etc. The touch sensor 116 detects that the pen 10, the user's hand, etc. (the pen and the user's hand are input means) has come into contact with the display 120.

[0018] The touch sensor controller 115 controls the processing of the touch sensor 116. The touch sensor 116 inputs and detects coordinates.

[0019] For example, if the touch sensor 116 is an optical type, two light receiving and emitting devices installed at both ends of the upper side of the display 120 emit a plurality of infrared rays parallel to the display 120. The infrared rays are reflected by a reflecting member provided around the periphery of the display 120. The touch sensor 116 receives, by a light receiving element, light returning along the same optical path as the emitted light.

[0020] The touch sensor 116 outputs position information of the infrared rays that are blocked by the object among the multiple infrared rays emitted by the two light receiving and emitting devices to the touch sensor controller 115, and the touch sensor controller 115 identifies the coordinate position that is the contact position of the object.

[0021] The touch sensor 116 is not limited to an optical type, and various detection means may be used, such as a capacitive touch panel that identifies the contact position by detecting changes in capacitance, a resistive touch panel that identifies the contact position by voltage changes across two opposing resistive films, or an electromagnetic induction touch panel that identifies the contact position by detecting electromagnetic induction caused by an object touching the display unit. The touch sensor 116 may be of a type that does not require an electronic pen to detect whether or not the pen tip is touching. In this case, a fingertip or a pen-shaped rod can be used to perform a touch operation. The pen 10 does not have to be a long, thin pen-shaped panel.

[0022] The power switch 122 is a switch for switching ON / OFF the power supply of the handwriting input device 100. The bus line 110 is an address bus, a data bus, etc. for electrically connecting the components such as the CPU 101 shown in FIG.

[0023] (An example of the functional configuration of the handwriting input device 100) FIG. 3 is a diagram showing an example of the functional configuration of the handwriting input device 100 according to the first embodiment.

[0024] As shown in FIG. 3, the handwriting input device 100 includes a handwriting input unit 131, a display unit 132, a handwriting input display control unit 133, a candidate display timer control unit 134, a handwriting input storage unit 135, a handwriting recognition control unit 151, a handwriting recognition dictionary unit 152, a character string conversion control unit 153, a character string conversion dictionary unit 154, a predictive conversion control unit 155, a predictive conversion dictionary unit 156, an expansion unit 142, and a predictive conversion sentence generation unit 143.

[0025] The handwriting recognition control unit 151, the handwriting recognition dictionary unit 152, the character string conversion control unit 153, the character string conversion dictionary unit 154, the predictive conversion control unit 155, and the predictive conversion dictionary unit 156 constitute the recognition-conversion-prediction unit 150. The recognition-conversion-prediction unit 150 is an example of the “first search means”.

[0026] Each function of the handwriting input device 100 is a function or means realized by any of the components shown in Figure 2 operating in accordance with instructions from the CPU 101 in accordance with a program expanded from the SSD 104 onto the RAM 103.

[0027] The handwriting input unit 131 is realized by the touch sensor 116 or the like, accepts handwriting input by the user, and receives a pen ID from the pen 10. The handwriting input unit 131 converts the user's pen input into pen operation data (pen up, pen down, or pen coordinate data) to which the pen ID is assigned, and transmits the data to the handwriting input display control unit 133. The pen coordinate data is periodically transmitted as discrete values, and coordinates between the discrete values ​​are interpolated.

[0028] Display unit 132 is realized by display 120 or the like, and displays handwritten objects, operation menus, etc. For example, display unit 132 receives drawing data written to a video memory by handwriting input display control unit 133, converts the data into data according to the characteristics of display 120, and displays the data.

[0029] The handwriting input display control unit 133 performs overall control over handwriting input and display. The handwriting input display control unit 133 processes pen operation data from the handwriting input unit 131 and transmits it to the display unit 132, thereby causing the display unit 132 to display strokes based on the pen operation data.

[0030] The candidate display timer control unit 134 is a timer for controlling the display of selectable candidates. The candidate display timer control unit 134 starts or stops the timer to generate the timing for starting the display of selectable candidates and the timing for erasing the display. The selectable candidates are predicted candidate character strings and the like that are displayed on the display unit 132 so that the user can select one. The candidate display timer control unit 134 receives a timer start request (or a timer stop request) from the handwriting input display control unit 133, and transmits a timeout event to the handwriting input display control unit 133.

[0031] Handwriting input saving unit 135 receives and stores user data (handwriting objects and character string objects) transmitted from handwriting input display control unit 133. Furthermore, handwriting input saving unit 135 transmits the stored user data to handwriting input display control unit 133 in response to a request from handwriting input display control unit 133.

[0032] Here, the handwritten object is determined based on pen operation data indicating the operation of the pen 10 on the handwriting input unit 131, and includes strokes drawn with the pen 10. The handwritten object is a character, number, symbol, or figure represented by a stroke, and is data that is the target of handwriting recognition by the handwriting recognition control unit 151. Furthermore, the character string object is data that is displayed on the display 120 as a result of character string conversion based on the handwritten object.

[0033] The handwriting recognition control unit 151 is a recognition engine that performs online handwriting recognition. The handwriting recognition control unit 151 recognizes characters, numbers, symbols, figures, etc. in parallel with the user's pen operation. Various algorithms have been devised for the recognition method used by the handwriting recognition control unit 151, but a detailed description thereof will be omitted since this embodiment can utilize known techniques.

[0034] The handwriting recognition control unit 151 receives pen operation data from the handwriting input display control unit 133. The handwriting recognition control unit 151 also generates handwriting recognition character string candidates by performing handwriting recognition on the pen operation data. The handwriting recognition control unit 151 also converts the handwriting recognition character string candidates into recognition result character strings that are linguistically likely by referring to the handwriting recognition dictionary unit 152 (dictionary data for language conversion in handwriting recognition).

[0035] The character string conversion control unit 153 receives the recognition result character string from the handwriting recognition control unit 151. Then, the character string conversion control unit 153 converts the recognition result character string into a conversion character string candidate by referring to the character string conversion dictionary unit 154 (dictionary data for character string conversion). The conversion character string candidate is a character string that is likely to be generated and includes the recognition result character string.

[0036] The predictive conversion control unit 155 receives the recognition result character string from the handwriting recognition control unit 151. Then, the predictive conversion control unit 155 converts the recognition result character string into a predicted candidate character string with reference to the predictive conversion dictionary unit 156 (dictionary data for predictive conversion). The predictive conversion character string is a character string that is likely to be generated and includes the recognition result character string.

[0037] The predictive conversion sentence generation unit 143 is an example of a "second search means" and receives the recognition result character string from the handwriting recognition control unit 151. Then, the predictive conversion control unit 155 converts the recognition result character string into a predicted candidate character string using a predetermined search method (for example, a generation AI). Note that the search range for predicted candidate character strings by the predictive conversion sentence generation unit 143 is wider than the search range for predicted candidate character strings by the predictive conversion control unit 155.

[0038] When a predetermined condition for determining whether or not there is uncertainty in the handwritten input is satisfied, the expansion unit 142 expands the search range of predicted candidate character strings by the "search means." Expanding the search range means widening the search target or increasing the number of search targets. As an example, as will be described later, for the handwritten character "water," in addition to the predicted candidate character strings "water, oneself, water, swimsuit, lake," the search target may be expanded to include "oneself," "fish landing," and "light blue."

[0039] For example, the expansion unit 142 can use, as a predetermined condition for determining whether or not there is hesitation in the handwritten input, whether or not the time interval between adjacent characters or the distance interval between adjacent characters in the handwritten input is equal to or greater than a predetermined threshold value.

[0040] In this case, if the time interval between adjacent characters or the distance interval between adjacent characters in handwritten input is equal to or less than a predetermined threshold, the expansion unit 142 determines not to expand the search range, and uses the recognition / conversion / prediction unit 150 as a "search means" to cause the predictive conversion control unit 155 to search for predicted candidate character strings.

[0041] In this case, if the time interval between adjacent characters or the distance interval between adjacent characters in handwritten input is greater than a predetermined threshold, the expansion unit 142 determines to expand the search range, and uses the predictive conversion sentence generation unit 143, which has a wider search range than the predictive conversion control unit 155, as a ``search means'' to cause the predictive conversion sentence generation unit 143 to search for predicted candidate character strings.

[0042] (An example of a processing sequence by the handwriting input device 100) FIG. 4 is a diagram showing an example of a processing sequence by the handwriting input device 100 according to the first embodiment.

[0043] First, when the user performs handwriting input, the handwriting input unit 131 transmits pen operation data (handwritten character string data) to the handwriting input display control unit 133 (sequence S1.1).

[0044] Next, the handwritten input display control unit 133 transmits the pen operation data (handwritten character string data) to the expansion unit 142 (sequence S1.2).

[0045] Next, the expansion unit 142 determines whether to expand the search range for predicted candidate character strings based on the handwritten input identified by the pen operation data (handwritten character string data), and transmits the determination result to the handwritten input display control unit 133 (sequence S1.3).

[0046] For example, when the time interval between adjacent characters or the distance interval between adjacent characters in handwritten input is equal to or less than a predetermined threshold, the expansion unit 142 determines not to expand the search range of predicted candidate character strings, and transmits the determination result to the handwritten input display control unit 133.

[0047] Also, for example, when the time interval between adjacent characters or the distance interval between adjacent characters in handwritten input is greater than a predetermined threshold, the expansion unit 142 determines to expand the search range of predicted candidate character strings, and transmits the determination result to the handwritten input display control unit 133.

[0048] Next, the handwriting input display control unit 133 transmits a timer start request to the candidate display timer control unit 134 (sequence S1.4).

[0049] Next, when the handwriting input display control unit 133 receives from the expansion unit 142 a determination result that the search range for predicted candidate character strings should be expanded, it transmits pen operation data to the handwriting recognition control unit 151 (sequence S1.5a), and the handwriting recognition control unit 151 converts the pen operation data into a recognition result character string and transmits the recognition result character string to the predictively converted sentence generation unit 143 (sequence S1.5b). In this case, the predictively converted sentence generation unit 143 searches for a predicted candidate character string corresponding to the recognition result character string and transmits the searched predicted candidate character string to the handwriting input display control unit 133 (sequence S1.5c). The handwriting input display control unit 133 causes the display unit 132 to display the predicted candidate character strings received from the predictively converted sentence generation unit 143 (sequence S1.7).

[0050] Alternatively, when the handwriting input display control unit 133 receives from the expansion unit 142 a determination result that the search range for predicted candidate character strings will not be expanded, the handwriting input display control unit 133 transmits pen operation data to the handwriting recognition control unit 151 (sequence S1.6a), which then converts the pen operation data into a recognition result character string and transmits the recognition result character string to the recognition-conversion-prediction unit 150 (sequence S1.6b). In this case, the recognition-conversion-prediction unit 150 searches for a predicted candidate character string corresponding to the recognition result character string and transmits the searched predicted candidate character string to the handwriting input display control unit 133 (sequence S1.6c). The handwriting input display control unit 133 causes the display unit 132 to display the predicted candidate character strings received from the recognition-conversion-prediction unit 150 (sequence S1.7).

[0051] (An example of the procedure of the extension process by the extension unit 142) FIG. 5 is a flowchart showing an example of the procedure of the expansion process by the expansion unit 142 included in the handwriting input device 100 according to the first embodiment.

[0052] First, the expansion unit 142 determines whether a predetermined condition for determining whether or not there is hesitation in the handwriting input is satisfied (step S501). Specifically, the expansion unit 142 determines whether or not the handwriting speed of the handwritten character string input by handwriting is lower than a predetermined threshold, and whether or not the distance between adjacent characters in the handwritten character string input by handwriting is greater than a predetermined threshold. If the handwriting speed of the handwritten character string input by handwriting is lower than the predetermined threshold, or if the distance between adjacent characters in the handwritten character string input by handwriting is greater than the predetermined threshold, the expansion unit 142 determines that "the predetermined condition for determining whether or not there is hesitation in the handwriting input is satisfied."

[0053] In step S501, if it is determined that a predetermined condition for determining whether or not there is hesitation in the handwritten input is met (step S501: YES), the expansion unit 142 determines to expand the search range and causes the predictive conversion sentence generation unit 143, which has a wider search range than the recognition-conversion-prediction unit 150, to search for predicted candidate character strings using the predictive conversion sentence generation unit 143 as a "search means" (step S502). After that, the expansion unit 142 ends the series of processes shown in FIG. 5.

[0054] In step S501, if it is determined that the predetermined condition for determining whether or not the handwritten input is uncertain is not met (step S501: NO), the expansion unit 142 determines not to expand the search range, and causes the predictive conversion control unit 155 to search for predicted candidate character strings using the recognition-conversion-prediction unit 150 as a "search means" (step S503). After that, the expansion unit 142 ends the series of processes shown in FIG. 5.

[0055] In this way, the handwriting input device 100 according to the first embodiment can expand the search range of predicted candidate character strings by the "search means" when it determines that a predetermined condition for determining whether or not there is uncertainty in handwriting input is met.

[0056] As a result, the handwriting input device 100 according to the first embodiment can increase the number of predicted candidate character strings to be presented when the user is unsure about handwriting input, thereby increasing the likelihood that the user will select the desired predicted candidate character string.

[0057] Therefore, according to the handwriting input device 100 of the first embodiment, the character recognition and predictive search methods can be changed in accordance with changes in the user's handwriting input actions, thereby enabling the user to obtain the character recognition and predictive search results desired.

[0058] In particular, in the handwriting input device 100 according to the first embodiment, when the expansion unit 142 (an example of an "expansion means") determines that a predetermined condition for determining whether or not there is uncertainty in the handwriting input is met, the "search means" used to search for predicted candidate character strings is switched from the recognition / conversion / prediction unit 150 (an example of a "first search means") to the predictive conversion sentence generation unit 143 (an example of a "second search means"), thereby expanding the search range of predicted candidate character strings by the "search means".

[0059] The method of expanding the search range of predicted candidate character strings by the expansion unit 142 may be other methods.

[0060] For example, the expansion unit 142 may expand the search range of predicted candidate character strings by adding predicted candidate character strings that have been searched for in the past to predicted candidate character strings searched for by the recognition-conversion-prediction unit 150.

[0061] For example, when the recognition-conversion-prediction unit 150 searches for predicted candidate character strings, the expansion unit 142 may expand the search range for predicted candidate character strings by increasing fuzzy searches.

[0062] The expansion unit 142 may also add a condition related to the operating state of the handwriting input device 100 to the predetermined conditions for determining whether or not to expand the search range for predicted candidate character strings.

[0063] For example, when the handwriting input device 100 is powered by a battery, the expansion unit 142 may expand the search range for predicted candidate character strings when the above-mentioned predetermined condition is satisfied and the remaining charge of the battery is equal to or greater than a predetermined threshold.

[0064] Furthermore, for example, when the handwriting input device 100 is driven by power from an AC adapter, the expansion unit 142 may expand the search range for predicted candidate character strings when the above-mentioned predetermined condition is satisfied.

[0065] (First Example of a Specific Example of Determination Processing by the Expansion Unit 142) FIG. 6 is a diagram showing a specific example (first example) of the determination process by the expansion unit 142 included in the handwriting input device 100 according to the first embodiment.

[0066] For example, the expansion unit 142 determines whether the handwriting input is hesitant based on a change in the moving average of the time required for one stroke in the handwriting input. This example is an example of the predetermined condition "when the handwriting speed is lower than a predetermined threshold value."

[0067] For example, FIG. 6 shows the time for each stroke and the moving average (time) of the three strokes when "akebi" is handwritten.

[0068] In particular, in the example shown in FIG. 6(a), the speed of handwriting input is normal and all moving averages are below a predetermined threshold (e.g., 26 ms), so the expansion unit 142 determines that "there is no hesitation in handwriting input."

[0069] On the other hand, in the example shown in FIG. 6(b), the speed of the handwriting input is slow, and there is a moving average (e.g., 26.7 ms) that exceeds a predetermined threshold (e.g., 26 ms), so the expansion unit 142 determines that "there is uncertainty in the handwriting input."

[0070] (Second Example of Determination Process by the Expansion Unit 142) FIG. 7 is a diagram showing a specific example (second example) of the determination process by the expansion unit 142 included in the handwriting input device 100 according to the first embodiment.

[0071] As another example, the expansion unit 142 determines whether there is hesitation in the handwriting input based on the time required to handwrite each character. This example is an example of the predetermined condition "when the handwriting speed is lower than a predetermined threshold value."

[0072] For example, FIG. 7 shows the time required for each stroke and the time required to handwrite each character when "akebi" is handwritten.

[0073] In particular, in the example shown in Figure 7(a), the speed of handwriting input is normal, and the time required to handwrite all characters ("a", "ke", "bi") is less than a predetermined threshold (e.g., 75 ms), so the expansion unit 142 determines that "there is no hesitation in the handwriting input."

[0074] On the other hand, in the example shown in Figure 7(b), the handwriting input speed is slow and there is a character ("a") (e.g., 76 ms) that took longer to handwrite than a predetermined threshold (e.g., 75 ms), so the expansion unit 142 determines that "there is hesitation in the handwriting input."

[0075] (Third Example of Specific Example of Determination Process by the Expansion Unit 142) FIG. 8 is a diagram showing a specific example (third example) of the determination process by the expansion unit 142 included in the handwriting input device 100 according to the first embodiment.

[0076] As another example, the expansion unit 142 determines whether there is hesitation in the handwriting input based on the time interval between strokes within a character. This example is an example of the predetermined condition "when the handwriting speed is lower than a predetermined threshold value."

[0077] For example, FIG. 8 shows the time of each stroke and the time interval between each stroke when "akebi" is handwritten.

[0078] In particular, in the example shown in FIG. 8(a), the handwriting input speed is normal and the time intervals between strokes are relatively short, so the expansion unit 142 determines that "the handwriting input is without hesitation."

[0079] On the other hand, in the example shown in FIG. 8(b), the speed of the handwriting input is slow and the time interval between each stroke is relatively large (e.g., the time interval between the third stroke of "a" and the first stroke of "ke" is longer than a predetermined threshold), so the expansion unit 142 determines that "there is hesitation in the handwriting input."

[0080] (Fourth Example of a Specific Example of Determination Processing by the Expansion Unit 142) FIG. 9 is a diagram showing a specific example (fourth example) of the determination process by the expansion unit 142 included in the handwriting input device 100 according to the first embodiment.

[0081] As another example, the expansion unit 142 determines whether or not there is uncertainty in the handwritten input based on whether or not there is a predetermined writing such as deletion or correction after character conversion or during pen input.

[0082] For example, FIG. 9 shows a character string handwritten by the user when the user finally selects "I want to eat akebi in the field."

[0083] In particular, in the example shown in Figure 9(a), the handwritten character string input by the user is "In the field where I want to eat Akebia," and since there is no specific writing such as deletion or correction, the expansion unit 142 determines that "the handwritten input is without hesitation."

[0084] On the other hand, in the example shown in Figure 9(b), the handwritten character string input by the user is "I want to eat Akebia in the field of △" (△ is multiple strikethroughs), and since there is a deletion entry (multiple strikethroughs), which is a specified entry, the expansion unit 142 determines that "there is uncertainty in the handwritten entry."

[0085] (Example of predicted character string display (Example 1)) FIG. 10 is a diagram showing a display example (first example) of predicted candidate character strings by the handwriting input display control unit 133 included in the handwriting input device 100 according to the first embodiment.

[0086] FIG. 10(a) shows an example of predicted candidate character strings that the handwriting input display control unit 133 displays on the screen when the expansion unit 142 determines that there is no hesitation in the handwriting input.

[0087] In the example shown in FIG. 10(a), the normal recognition-conversion-prediction unit 150 is used as the “search means,” and therefore, for the character string “mizu” (water) handwritten in the handwriting input window 431, five predicted candidate character strings, “mizu,” “jikan (person), “mizu” (water), “suijutsu” (swimsuit), and “ko” (lake), are displayed in the candidate display window 432.

[0088] 10(b) shows an example of predicted candidate character strings that the handwriting input display control unit 133 displays on the screen when the expansion unit 142 determines that the handwritten input is uncertain. In the example shown in Fig. 10(b), the predictive conversion sentence generation unit 143 is used as the "search means," and the search range is expanded compared to the example shown in Fig. 10(a). Therefore, for the character string "water" handwritten in the handwriting input window 431, eight predicted candidate character strings, "water," "himself," "water," "swimsuit," "lake," "himself," "landing," and "light blue," are displayed in the candidate display window 432.

[0089] The candidate display window 432 is displayed by expanding the handwriting input window 431, and is deleted when a predetermined time has passed or when the user selects one of the predicted candidate character strings.

[0090] (Example of predicted character string display (Example 2)) FIG. 11 is a diagram showing a display example (second example) of predicted candidate character strings by the handwriting input display control unit 133 included in the handwriting input device 100 according to the first embodiment.

[0091] FIG. 11 shows an example of predicted candidate character strings that the handwriting input display control unit 133 displays on the screen when the expansion unit 142 determines that there is uncertainty in the handwriting input.

[0092] 11, for the character string "otoga" handwritten in handwriting input window 431, the predicted candidate character string "oto ga kiketa" (sound was heard) searched for by predictive conversion sentence generation unit 143 is displayed in candidate display window 433. This candidate display window 433 is always displayed separately from handwriting input window 431.

[0093] (Example of display of predicted character strings (Example 3)) FIG. 12 is a diagram showing a display example (third example) of predicted candidate character strings by the handwriting input display control unit 133 included in the handwriting input device 100 according to the first embodiment.

[0094] 12 shows an example of predicted candidate character strings that the handwriting input display control unit 133 displays on the screen when the expansion unit 142 determines that there is uncertainty in the handwritten input. In this example, both the recognition-conversion-prediction unit 150 and the predictive conversion sentence generation unit 143 are used as the "search means."

[0095] 12, for the character string "oto" handwritten in handwriting input window 431, four predicted candidate character strings "oto," "man," "oto," and "otona" are searched for by recognition / conversion / prediction unit 150 and are displayed in candidate display window 432. This candidate display window 432 is displayed by expanding handwriting input window 431, and is deleted when a predetermined time has passed or when one of the predicted candidate character strings is selected by the user.

[0096] 12, for the character string "oto" handwritten in handwriting input window 431, the predictive conversion sentence generation unit 143 retrieves a predicted candidate character string "oto ni kiketa" (I heard the sound), and the candidate character string is displayed in candidate display window 433. This candidate display window 433 is always displayed separately from handwriting input window 431.

[0097] (Example of predicted character string display (Example 4)) FIG. 13 is a diagram showing a display example (fourth example) of predicted candidate character strings by the handwriting input display control unit 133 included in the handwriting input device 100 according to the first embodiment.

[0098] FIG. 13 shows an example of predicted candidate character strings that the handwriting input display control unit 133 displays on the screen when the expansion unit 142 determines that there is uncertainty in the handwriting input.

[0099] 13, for a character string "the day before yesterday..." that is handwritten in handwriting input window 431, a predicted candidate character string "February 28, 2024..." that is searched for by predictive conversion sentence generation unit 143 is displayed in candidate display window 433. This candidate display window 433 is always displayed separately from handwriting input window 431.

[0100] 13, a predetermined term and a predetermined abbreviation are included in the character string "The day before yesterday..." that is handwritten in the handwriting input window 431. In this case, the predictive conversion sentence generation unit 143 can convert the predetermined term and the predetermined abbreviation into a corresponding character string and generate a predicted candidate character string "February 28, 2024...".

[0101] For example, in the example shown in Figure 13, the predictive conversion sentence generation unit 143 converts "the day before yesterday", "K", "X", "2311", and "QQ call" into "February 28, 2024", "Patient (Yamada Osuke)", "none", "11:11 pm", and "119 call", respectively.

[0102] (Example of predicted character string display (Example 5)) FIG. 14 is a diagram showing a display example (fifth example) of predicted candidate character strings by the handwriting input display control unit 133 included in the handwriting input device 100 according to the first embodiment.

[0103] FIG. 14 shows an example of predicted candidate character strings that the handwriting input display control unit 133 displays on the screen when the expansion unit 142 determines that there is uncertainty in the handwriting input.

[0104] 14, for a character string "Atlantic..." handwritten in handwriting input window 431, a predicted candidate character string "Atlantic..." searched for by predictive conversion sentence generation unit 143 is displayed in candidate display window 433. This candidate display window 433 is always displayed separately from handwriting input window 431.

[0105] In the example shown in FIG. 14, the character string "... in the ocean" handwritten in the handwriting input window 431 contains multiple spaces. These multiple spaces are caused by the user being unable to remember the characters to be input. In this case, the expansion unit 142 determines that there are spaces by the space determination process shown in FIG. 14, and determines that "the user is unsure about the handwritten input," and switches the search for predicted candidate character strings to the predictive conversion sentence generation unit 143. The predictive conversion sentence generation unit 143 can use the handwritten character string "... in the ocean" to display the predicted candidate character string "... in the candidate display window 433 as a search result.

[0106] (An example of a procedure for a blank space determination process by the expansion unit 142) FIG. 15 is a flowchart showing an example of the procedure of the blank space determination process by the expansion unit 142 included in the handwriting input device 100 according to the first embodiment.

[0107] First, the expansion unit 142 sets the size of the font to be converted into handwritten text (step S601). Next, the expansion unit 142 calculates the distance between characters in the handwritten character string input by hand (step S602). Then, the expansion unit 142 determines whether the distance between characters calculated in step S602 is greater than 1.8 times the font size set in step S601 (an example of a "predetermined threshold value") (step S603).

[0108] If it is determined in step S603 that the distance between characters is greater than 1.8 times the font size (step S603: YES), the expansion unit 142 determines that there is a space between the characters (step S604). After that, the expansion unit 142 ends the series of processes shown in FIG. 15.

[0109] If it is determined in step S603 that the distance between characters is equal to or less than 1.8 times the font size (step S603: NO), the expansion unit 142 determines that there is no space between the characters (step S605). After that, the expansion unit 142 ends the series of processes shown in FIG. 15.

[0110] In this way, when a handwritten character string contains a space (i.e., when the distance between adjacent characters is greater than a predetermined threshold), the handwriting input device 100 according to the first embodiment determines that "the handwritten input is unsure" and switches the search for predicted candidate character strings to the predictive conversion sentence generation unit 143.

[0111] If a handwritten character string input by handwriting contains a predetermined non-target character (for example, "*") that serves the same role as a space, the expansion unit 142 may calculate the distance between characters by regarding the predetermined non-target character as a space.

[0112] Second Embodiment (An example of the functional configuration of the handwriting input device 100-2) 16 is a diagram showing an example of the functional configuration of a handwriting input device 100-2 according to the second embodiment. Hereinafter, regarding the handwriting input device 100-2 according to the second embodiment, changes from the handwriting input device 100 according to the first embodiment will be described.

[0113] The functional configuration of handwriting input device 100-2 shown in Fig. 16 differs from the functional configuration of handwriting input device 100 shown in Fig. 3 in that it has wireless communication unit 146 but does not have predictive conversion sentence generation unit 143. Instead, handwriting input device 100-2 shown in Fig. 16 constitutes a handwriting input system that communicates with server 20 via a network, and predictive conversion sentence generation unit 143 is provided in the server.

[0114] 16, for example, when it is determined that there is uncertainty in the handwriting input, the handwriting recognition control unit 151 converts the pen operation data into a recognition result character string and transmits the recognition result character string to the predictively converted sentence generation unit 143 provided in the server 20 via the wireless communication unit 146. This allows the handwriting input device 100-2 to cause the predictively converted sentence generation unit 143 provided in the server 20 to search for a predicted candidate character string corresponding to the recognition result character string. Then, the handwriting input device 100-2 can obtain the searched predicted candidate character string from the predictively converted sentence generation unit 143 provided in the server 20 via the wireless communication unit 146.

[0115] In this way, the handwriting input device 100 and the server constitute a "handwriting input system," and part of the processing that is performed by the handwriting input device 100 in this embodiment may be performed by the server.

[0116] Third Embodiment (An example of the functional configuration of the handwriting input device 100-3) 17 is a diagram showing an example of the functional configuration of a handwriting input device 100-3 according to the third embodiment. Hereinafter, regarding the handwriting input device 100-3 according to the third embodiment, changes from the handwriting input device 100 according to the first embodiment will be described.

[0117] The functional configuration of the handwriting input device 100-3 shown in FIG. 17 differs from the functional configuration of the handwriting input device 100 shown in FIG.

[0118] The analysis unit 144 analyzes the result of the determination made by the device itself as to whether a character is an uppercase letter or a lowercase letter. For example, the analysis unit 144 acquires the results of the previous determinations made by the device itself as to whether a character is an uppercase letter or a lowercase letter (for example, 10 times) from a memory or the like included in the handwriting input device 100-3.

[0119] The display receiving unit 145 displays the analysis results by the analysis unit 144 on the display 120 (display unit), and receives changes to the setting conditions to widen the search range based on the analysis results by the analysis unit 144.

[0120] (Example of procedure for changing setting conditions) FIG. 18 is a flowchart showing an example of the procedure of the setting condition change process by the handwriting input device 100-3 according to the third embodiment.

[0121] First, the analysis unit 144 checks the number of searches for predicted candidate character strings by handwriting input (step S701). Next, the handwriting input device 100-3 determines whether the number of searches checked in step S701 is 10 or more (step S702).

[0122] In step S702, if it is determined that the number of determinations is not 10 or more (step S702: NO), the handwriting input device 100-3 ends the series of processes shown in FIG.

[0123] If it is determined in step S702 that the number of determinations is 10 or more (step S703: YES), the display receiving unit 145 displays a display screen (see FIG. 19) showing past determination results on the display 120 (step S703).

[0124] Next, the display receiving unit 145 determines whether the past judgment result displayed in step S703 is the result intended by the operator (step S704). For example, the display screen displayed in step S703 allows the operator to select whether the result is the result intended by the operator. In this case, the display receiving unit 145 determines whether the past judgment result is the result intended by the operator in accordance with the selection by the operator.

[0125] In step S704, if it is determined that the previous determination result is the result intended by the operator (step S704: YES), the handwriting input device 100-3 ends the series of processes shown in FIG.

[0126] In step S704, if it is determined that the previous determination result is not the result intended by the operator (step S704: NO), the handwriting input device 100-3 changes the setting conditions (setting conditions for widening the search range in this embodiment) for the next time and thereafter (step S706). After that, the handwriting input device 100-3 ends the series of processes shown in FIG.

[0127] (An example of a display screen displayed by the display receiving unit 145) FIG. 19 is a diagram showing an example of a display screen displayed by the display accepting unit 145 included in the handwriting input device 100-3 according to the third embodiment.

[0128] 19 is an example of a display screen displayed on the display 120 by the display receiving unit 145. On the display screen 420, search results of predicted candidate character strings based on handwritten inputs for the past 10 times are displayed in a list together with the date and time of handwriting and the content of the handwritten input (handwritten character strings input by hand).

[0129] As an example, display screen 420 shows that the date and time of the first handwritten input is "2023 / 08 / 15 16:05." It also shows that handwriting input device 100-3 recognizes the character string handwritten by the operator as "water," and that predicted candidate character strings "water, myself, water, swimsuit, lake" are displayed on the screen as search results for predicted candidate character strings using the recognized character string "water."

[0130] Additionally, on the display screen 420, as an example of the notification content to the user, the following string is displayed: "The results of the last 10 searches for handwritten input strings and predicted candidate strings will be displayed. Regarding the judgment results, are the search results intended by the operator displayed at the top?"

[0131] Also displayed on the display screen 420 are a "Yes" button and a "No" button that can accept a selection from the operator.

[0132] For example, if the operator selects the "Yes" button, the judgment result displayed on the display screen 420 is the judgment result intended by the operator, so the handwriting input device 100-3 does not change the setting conditions for widening the search range.

[0133] Also, if the operator selects the "No" button, the judgment result displayed on the display screen 420 is not the judgment result intended by the operator, so the handwriting input device 100-3 changes the setting conditions to widen the search range.

[0134] For example, when the operator selects the "No" button, the handwriting input device 100-3 may display a check box for each of a plurality of predicted candidate character strings (e.g., "water, oneself, water, swimsuit, lake") displayed as search results on the display screen 420 (or another display screen), and allow the user to select the check box of any one of the appropriate predicted candidate character strings. In this case, when the user selects the check box of any one of the predicted candidate character strings (e.g., "lake"), the handwriting input device 100-3 may change the setting condition for widening the search range so that the selected predicted candidate character string will be displayed at the top from the next time onwards.

[0135] As another example, when the operator selects the "No" button, the handwriting input device 100-3 may display a check box for each of a plurality of predicted candidate character strings (e.g., "water, oneself, water, swimsuit, lake") displayed as search results on the display screen 420 (or another display screen), and may prompt the user to select the check box of one or more inappropriate predicted candidate character strings. In this case, when the user selects the check box of one or more predicted candidate character strings, the handwriting input device 100-3 may change the setting condition for widening the search range so that the selected one or more predicted candidate character strings are displayed at the bottom or not displayed from the next time onwards.

[0136] The best mode for carrying out the present invention has been described above using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0137] For example, in this embodiment, an electronic whiteboard is used as an example of the "handwriting input device," but the present invention is not limited to this, and other devices may be used as the "handwriting input device" as long as they are capable of at least handwriting input. For example, the "handwriting input device" may be an output device such as a PJ (Projector), digital signage, a HUD (Head Up Display) device, industrial machinery, an imaging device, a sound collection device, medical equipment, network appliances, a notebook PC (Personal Computer), a mobile phone, a smartphone, a tablet terminal, a game console, a PDA (Personal Digital Assistant), a digital camera, a wearable PC, a desktop PC, or the like.

[0138] Furthermore, in this embodiment, the coordinates of the pen tip of the pen 10 are detected by the touch sensor 116, but the present invention is not limited to this. For example, the coordinates of the pen tip of the pen 10 may be detected by ultrasonic waves. In this case, for example, the pen 10 emits light and emits ultrasonic waves, and the handwriting input device 100 calculates the distance based on the arrival time of the ultrasonic waves from the pen 10. Then, the handwriting input device 100 can identify the coordinates of the pen tip of the pen 10 based on the direction and distance.

[0139] Furthermore, in this embodiment, the pen 10 is used as an example of the "input means," but the "input means" is not limited to this, and may be, for example, a human finger, a rod-shaped member, etc., as long as it is capable of at least handwritten input.

[0140] The functional configuration of the handwriting input device 100 shown in Fig. 3 is divided according to main functions to facilitate understanding of the processing by the handwriting input device 100. The present invention is not limited by the way in which the processing units are divided or the names of the processing units. The processing of the handwriting input device 100 can be divided into more processing units depending on the processing content. Also, it can be divided so that one processing unit includes more processes.

[0141] Furthermore, each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions. [Explanation of symbols]

[0142] 10 pens 20 servers 100, 100-2, 100-3 Handwriting input device 120 Display 131 Handwriting input section 132 Display section 133 Handwriting input display control unit 134 Candidate display timer control section 135 Handwriting input storage unit 144 Analysis Department 145 Display Reception Department 146 Radio Communication Department 147 Voice Recognition Unit 150 Recognition, Conversion, and Prediction 151 Handwriting recognition control unit 152 Handwriting Recognition Dictionary 153 String conversion control section 154 String Conversion Dictionary 155 Predictive conversion control unit 156 Predictive conversion dictionary section 420 display screen 431 Handwriting Input Window 432,433 Candidate display window U User [Prior art documents] [Patent documents]

[0143] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-179529

Claims

1. A handwriting input device capable of handwriting input, a conversion means for converting the handwritten input character string into a recognition result character string by performing character recognition on the handwritten input character string; a search means for searching for a predicted candidate character string corresponding to the recognition result character string; a display control means for displaying the predicted candidate character strings on a screen in a selectable manner; an extension means for extending a search range of the predicted candidate character string by the search means when the handwriting speed of the handwritten character string is lower than a predetermined threshold or when the distance between adjacent characters in the handwritten character string is greater than a predetermined threshold, The display control means The predicted candidate character strings searched by the search means using the expanded search range are displayed on the screen in a selectable manner. A handwriting input device characterized by:

2. The expansion means If the distance between adjacent characters in the handwritten character string is greater than a predetermined threshold, it is determined that a space exists.

2. The handwriting input device according to claim 1, wherein:

3. The expansion means If the handwritten character string contains a predetermined non-target character, the predetermined non-target character is regarded as a space, and the distance between adjacent characters is calculated.

3. The handwriting input device according to claim 2.

4. The expansion means The search means is switched from the first search means to the second search means having a wider search range than the first search means, thereby expanding the search range of the predicted candidate character strings by the search means.

2. The handwriting input device according to claim 1, wherein:

5. The second search means is a generating AI provided on an external server.

5. The handwriting input device according to claim 4.

6. The expansion means The search range of the predicted candidate character strings is expanded by adding the predicted candidate character strings previously searched for by the search means to the predicted candidate character strings searched for by the search means.

2. The handwriting input device according to claim 1, wherein:

7. The expansion means When the search means searches for the predicted candidate character string, the search range for the predicted candidate character string is expanded by increasing fuzzy searches.

2. The handwriting input device according to claim 1, wherein:

8. The expansion means When the handwriting input device is driven by power from a battery, if the predetermined condition is satisfied and the remaining charge of the battery is equal to or greater than a predetermined threshold, the search range of the predicted candidate character string by the search means is expanded.

2. The handwriting input device according to claim 1, wherein:

9. The expansion means When the handwriting input device is driven by power from an AC adapter, the search range of the predicted candidate character strings by the search means is expanded when the predetermined condition is satisfied.

2. The handwriting input device according to claim 1, wherein:

10. an analysis unit that analyzes the determination result of the device itself; a display receiving unit that displays the analysis result by the analysis unit on a display unit and receives changes to setting conditions in the device itself based on the analysis result; The handwriting input device according to claim 1 , further comprising:

11. A handwriting input system in which a handwriting input device capable of handwriting input and a server communicate with each other via a network, The server a conversion means for converting the handwritten input character string into a recognition result character string by performing character recognition on the handwritten input character string; a search means for searching for a predicted candidate character string corresponding to the recognition result character string; a display control means for displaying the predicted candidate character strings on a screen in a selectable manner; an extension means for extending a search range of the predicted candidate character string by the search means when the handwriting speed of the handwritten character string is lower than a predetermined threshold or when the distance between adjacent characters in the handwritten character string is greater than a predetermined threshold, The display control means The predicted candidate character strings searched by the search means using the expanded search range are displayed on the screen in a selectable manner. A handwriting input system characterized by:

12. A handwriting input device capable of handwriting input, a conversion means for converting the handwritten input character string into a recognition result character string by performing character recognition on the handwritten input character string; a search means for searching for a predicted candidate character string corresponding to the recognition result character string; a display control means for displaying the predicted candidate character strings on a screen; a program for causing the program to function as an extension means for extending a search range of the predicted candidate character string by the search means when a handwriting speed of the handwritten character string is lower than a predetermined threshold or when a distance between adjacent characters in the handwritten character string is greater than a predetermined threshold, The display control means The predicted candidate character strings searched by the search means using the expanded search range are displayed on the screen in a selectable manner. program.

13. A handwriting input method using a handwriting input device capable of handwriting input, comprising: a conversion step of converting the handwritten input character string into a recognition result character string by performing character recognition on the handwritten input character string; a search step of searching for a predicted candidate character string corresponding to the recognition result character string; a display control step of displaying the predicted candidate character strings on a screen in a selectable manner; an expansion configuration for expanding a search range of the predicted candidate character string in the searching step when a handwriting speed of the handwritten character string is lower than a predetermined threshold or when a distance between adjacent characters in the handwritten character string is greater than a predetermined threshold, The display control step includes: The predicted candidate character strings searched for by the searching step using the expanded search range are displayed on the screen in a selectable manner. A handwriting input method comprising:

Images