Information processing apparatus, method, and program

Abstract

To assist a user to perform a desired operation upon completion of input to a specific entry field by handwriting.SOLUTION: An information processing apparatus has: a determination unit that determines completion of input by handwriting to a specific entry field that is a preset entry field, of a plurality of entry fields for receiving input by a user's handwriting displayed on a display; a display control unit that, when the determination unit determines the completion of input by handwriting to the specific entry field, displays, on the display, display components for receiving instructions for a predetermined operation in a selective manner; and a control unit that, when one of the display components is selected by the user, controls to execute processing according to the predetermined operation corresponding to the display component.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an information processing device, method, and program. [Background technology]

[0002] There is known a technology that allows various operations to be performed by gestures on a display device that allows handwriting input. For example, Patent Document 1 discloses a display device that allows scrolling, finishing writing medical records, and the like, by drawing operations using a pen used for handwriting input. Summary of the Invention [Problem to be solved by the invention]

[0003] When a user completes handwriting input in an input field displayed on a display unit, if the user does not know the gesture for performing the next desired operation, the user may be unable to perform the operation. An object of the present invention is to support the user in performing the desired operation in such cases. [Means for solving the problem]

[0004] The disclosed technology is an information processing device characterized by having a judgment unit that judges that handwritten input has been completed in a specific input field, which is a pre-set input field, among multiple input fields displayed on a display unit that accept handwritten input by a user; a display control unit that, when the judgment unit judges that handwritten input has been completed in the specific input field, causes a display component that accepts instructions for a specified operation to be selectable on the display unit; and a control unit that, when the display component is selected by the user, executes processing in accordance with the specified operation corresponding to the display component. [Effects of the Invention]

[0005] When input into a specific input field is completed by handwriting input, the user can be assisted in performing a desired operation. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 illustrates an example of a system configuration. [Figure 2] FIG. 1 is a diagram showing an example of the arrangement of display devices. [Figure 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a display device. [Figure 4] FIG. 2 is a functional block diagram for explaining functions of the display device in blocks. [Figure 5] 10A and 10B are diagrams showing screen transitions when input is made to a specific input field on the display device. [Figure 6] FIG. 10 is a flowchart illustrating the operation of the display device. [Figure 7] 10A and 10B are diagrams showing screen transitions when "display next page" is selected on the display device. [Figure 8] 10A and 10B are diagrams showing screen transitions when "display blank page" is selected on the display device. [Figure 9] 10A and 10B are diagrams showing screen transitions when "copy while keeping background" is selected on the display device. [Figure 10] FIG. 10 is a diagram showing the screen transition when "Save to cloud" is selected on the display device. [Figure 11] FIG. 10 is a diagram showing a screen transition when "Edit" is selected on the display device. [Figure 12] 10A and 10B are diagrams illustrating a form in which display components are displayed without screen transition on a display device. [Figure 13] 10 is an example of a flowchart when a display component that accepts an instruction for a back operation is selected on the display device. [Figure 14] FIG. 10 is a diagram showing the setting of a specific input field. [Figure 15] FIG. 3 is a diagram illustrating data definitions contained in a file according to an embodiment. [Figure 16] 10 is a flowchart relating to setting of a specific input field. [Figure 17]FIG. 10 is a functional block diagram of a modified example. [Figure 18] FIG. 10 is a diagram showing an example of a screen displayed in a modified example. [Figure 19] FIG. 10 is a flowchart illustrating a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0007] <Terminology> The display device refers to a device that displays a screen, such as an electronic whiteboard or a terminal device with a display.

[0008] Handwriting input refers to a user touching a display with a finger or pen to input information into the displayed screen. Continuous input by a user using an input means is called a stroke. Information continuously input by strokes is called stroke information. The stroke information may be appropriately corrected for hand shake or the like. The content input by handwriting input is a collection of this stroke information.

[0009] Character recognition refers to recognizing the content entered by handwriting as described above and converting it into text data. When converting the input content, conversion candidate text data may be presented to the user and converted according to the user's selection. The timing for character recognition of the input content may be after the user has completed handwriting input, or after the handwritten input content has been recognized as text, such as character by character.

[0010] An input field is an area on the screen that accepts handwritten input from a user. The input field may be displayed so that the area is visible on the screen, or it may not be displayed so that it is distinguishable from other areas. In the embodiment described below, an example will be described in which the input field is displayed so that it is distinguishable as a field for user input.

[0011] Display data is data that can be displayed on a display device, such as a PDF file, a Word file, or HTML (HyperText Markup Language).

[0012] In the following embodiment, an example in which a user inputs handwriting to the input device 2 will be described.

[0013] FIG. 1 is a diagram showing a system configuration according to an embodiment of the present invention.

[0014] In this embodiment, the display device 2 displays a file, but the present invention is not limited to this. The screen displayed on the display device 2 may be a website screen displayed in HTML (Hyper Text Markup Language) or the like, in addition to a file.

[0015] The communication system 1 in the embodiment includes a display device 2, a server device 3, and a terminal device 4. The display device 2, the server device 3, and the terminal device 4 communicate with each other via a communication network 100.

[0016] <Device configuration example> The overall configuration of the display device 2 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the arrangement of the display device 2. Fig. 2(a) shows, as an example of the display device 2, a display device 2 used as a horizontally long electronic whiteboard hung on a wall.

[0017] 2(a), a display 220 as an example of the display device 2 is provided on the upper part of the display device 2. A user can input characters and the like by handwriting on the display 220 using a pen 2500.

[0018] FIG. 2(b) shows a display device 2 used as a vertically long electronic whiteboard hung on a wall.

[0019] 2(c) shows the display device 2 placed flat on a desk 230. The display device 2 is about 1 cm thick, so there is no need to adjust the height of the desk even when it is placed flat on a regular desk. In addition, the user can easily move the display device 2.

[0020] The pen 2500 can input coordinates using electromagnetic induction, active electrostatic coupling, etc. The pen 2500 may also have functions such as pressure detection, tilt detection, and a hover function (displaying the cursor before the pen touches the screen).

[0021] A possible use example of the display device 2 is a medical setting such as a hospital. In a medical setting, when a user of the display device 2 accepts an emergency patient, such as a patient brought in by ambulance, the user records details of the accepted emergency patient on the display device 2. In this case, the user operating the display device 2 is not limited to a doctor, but is also likely to be a nurse or office worker. Nurses and office workers do not necessarily know the gestures to operate the display device 2, and may not even know the gestures. In such cases, it may be difficult to get nurses or office workers to memorize the gestures to operate the display device 2. Therefore, this embodiment makes it possible to realize smooth operation upon completion of input.

[0022] Another possible use case is a manufacturing site. In a manufacturing site, the display device 2 may display production records, patrol records, and the like in a format that allows on-site workers to input and confirm data. In such a case, it may be difficult to have all of the users, or workers, memorize the gestures for operating the display device 2. Therefore, this embodiment allows the display device 2 to be operated even if the users have not memorized the gestures.

[0023] In addition to gestures, the display device 2 can also be operated by constantly displaying an operation menu, but if the four sides of the display device 2 are large, it may be difficult for the user to find the area where the operation menu is displayed. In this embodiment, display components are displayed selectably upon completion of the user's input operation in the input field, thereby prompting the user to select the next operation.

[0024] <Device hardware configuration> Next, the hardware configuration of the display device 2 will be described with reference to Fig. 3. As shown in the figure, the display device 2 has the configuration of an information processing device or a computer. Fig. 3 is an example of a hardware configuration diagram of the display device 2. As shown in Fig. 3, the display device 2 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and an SSD (Solid State Drive) 204.

[0025] Of these, the CPU 201 controls the overall operation of the display device 2. The ROM 202 stores the CPU 201 and programs used to drive the CPU 201, such as an IPL (Initial Program Loader). The RAM 203 is used as a work area for the CPU 201.

[0026] The SSD 204 stores various data such as an OS and programs for the display device 2. This program may be an application program that runs on an information processing device equipped with a general-purpose OS (Windows (registered trademark), Mac OS (registered trademark), Android (registered trademark), iOS (registered trademark), etc.). In this case, the device is normally used as a general-purpose information processing device, but when a user runs the application program, the user can input handwritten text in the same way as on a dedicated device for the display device 2.

[0027] The display device 2 also includes a display controller 213, a touch sensor controller 215, a touch sensor 216, a display 220, a power switch 227, a tilt sensor 217, a serial interface 218, a speaker 219, a microphone 221, a wireless communication device 222, an infrared I / F 223, a power control circuit 224, an AC adapter 225, and a battery 226. The display 220 is an example of a display unit.

[0028] The display controller 213 controls and manages the screen display in order to output an output image to the display 220, etc. The touch sensor 216 detects that the pen 2500, the user's hand, etc. (the pen and the user's hand are input means) has touched the display 220. The touch sensor 216 also receives a pen ID.

[0029] The touch sensor controller 215 controls the processing of the touch sensor 216. The touch sensor 216 inputs and detects coordinates. The method for inputting and detecting coordinates will be described below. For example, in the case of an optical type, two light receiving and emitting devices installed at both ends of the upper side of the display 220 emit a plurality of infrared rays parallel to the display 220. The two light receiving and emitting devices receive light that is reflected by a reflecting member installed around the periphery of the display 220 and returns along the same optical path as the light emitted by the light receiving element.

[0030] The touch sensor 216 outputs position information of the infrared rays emitted by the two light receiving and emitting devices that are blocked by the object to the touch sensor controller 215, and the touch sensor controller 215 identifies the coordinate position of the contact position of the object. The touch sensor controller 215 also has a communication unit 215a and can communicate wirelessly with the pen 2500. For example, when communication is based on a standard such as Bluetooth (registered trademark), a commercially available pen can be used. If one or more pens 2500 are registered in the communication unit 215a in advance, the user can communicate with the pen 2500 without having to perform connection settings to communicate with the display device 2.

[0031] The power switch 227 is a switch for switching ON / OFF the power of the display device 2. The tilt sensor 217 is a sensor for detecting the tilt angle of the display device 2. It is mainly used to detect whether the display device 2 is being used in the installation state shown in FIG. 2(a), FIG. 2(b), or FIG. 2(c), and can automatically change the thickness of characters, etc. depending on the installation state.

[0032] The serial interface 218 is an external communication interface such as a USB. The serial interface 218 is used for inputting information from the outside. The speaker 219 is used for audio output, and the microphone 221 is used for audio input. The wireless communication device 222 communicates with a terminal carried by the user, and relays a connection to the Internet, for example. The wireless communication device 222 communicates using Wi-Fi, Bluetooth (registered trademark), or the like, but any communication standard is acceptable. The wireless communication device 222 forms an access point, and when the user sets the SSID (Service Set Identifier) ​​and password they have obtained in the terminal they carry, they can connect to the access point.

[0033] Preferably, the wireless communication device 222 is provided with two access points. a. Access point → Internet b. Access point → Internal network → Internet Access point a is for users outside the company, and users cannot access the company network but can use the Internet. Access point b is for users inside the company, and users can use the company network and the Internet.

[0034] The infrared I / F 223 detects the display device 2 arranged adjacently. The infrared I / F 223 can detect only the display device 2 arranged adjacently by taking advantage of the linearity of infrared rays. It is preferable to provide one infrared I / F 223 on each side, and it can detect in which direction another display device 2 is arranged relative to the display device 2. This expands the screen, making it possible to display handwritten information that was previously handwritten on the adjacent display device 2 (handwritten information on another page, with the area of ​​one display 220 being one page), etc.

[0035] The power supply control circuit 224 controls an AC adapter 225 and a battery 226, which are power sources for the display device 2. The AC adapter 225 converts AC common to commercial power sources into DC.

[0036] If the display 220 is so-called electronic paper, it consumes little or no power to maintain the image display, and therefore can be driven by a battery 226. This makes it possible to use the display device 2 for purposes such as digital signage even in places where it is difficult to connect to a power source, such as outdoors.

[0037] The display device 2 further includes a bus line 210. The bus line 210 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 201 shown in FIG.

[0038] The touch sensor 216 is not limited to an optical type, and may be a capacitive touch panel that identifies the contact position by detecting changes in capacitance. The touch sensor 216 may also be a resistive film type touch panel that identifies the contact position by detecting voltage changes across two opposing resistive films. The touch sensor 216 may also be an electromagnetic induction type touch panel that identifies the contact position by detecting electromagnetic induction generated when a contacting object touches the display unit, or various other detection means may be used. The touch sensor 216 may be of a type that does not require an electronic pen to detect whether or not the pen tip is touching. In this case, a fingertip or a pen-shaped stick can be used to perform a touch operation. The pen 2500 does not have to be a long, thin pen-shaped panel.

[0039] FIG. 4 is a functional block diagram illustrating the functions of the display device 2 and the terminal device 4 in separate blocks.

[0040] The display device 2 has, as its functions, a display control unit 21, a determination unit 22, a storage unit 23, a detection unit 24, a communication unit 25, a judgment unit 26, and a control unit 27. Each function of the display device 2 is a function or means realized by operating in accordance with an instruction from the CPU 201.

[0041] The display control unit 21 controls the display of a screen displayed on the display 220 of the display device 2. The display control unit 21 controls the display on the display 220 under the control of the control unit 27. When a user inputs by handwriting into an input field displayed on the display 220, the display control unit 21 sequentially displays the input content that has been recognized as characters. The display control unit 21 displays display data including multiple input fields on the display 220. Furthermore, the display control unit 21 displays a display component 510 in response to determining that input into a specific input field has been completed. The specific input field refers to a preset input field among the multiple input fields. Furthermore, the display component 510 is a general term for display components 511 to 515, which will be described later. The display component 510 may be displayed so that an option can be selected by touching the display component 510, or may be displayed so that an option can be selected by a drop-down list or radio buttons.

[0042] The determination unit 22 determines that the user's input into the input fields displayed on the display 220 has been completed. The determination unit 22 determines whether the user's handwritten input into a specific input field has been completed. As a specific method of determination, the determination unit 22 determines that the user's input has been completed when the user's handwritten input has been interrupted and no handwritten input has been made for a certain period of time. Alternatively, a completion button may be provided for each input field on the input screen, and after the user's handwritten input, the determination may be made that the user's input has been completed in response to the operation of the completion button provided for each input field.

[0043] The storage unit 23 stores information about the screen to be displayed on the display 220 and information about the files to be displayed on the display 220. The information stored in the storage unit 23 will be described later.

[0044] The detection unit 24 detects which of the multiple input fields displayed on the display 220 is being input by handwriting by the user. Specifically, the detection unit 24 detects which of the input fields is being input by the user based on information about the screen displayed on the display 220 and coordinates on the screen detected by the touch sensor 216. The detection unit 24 detects which of the input fields is being input by handwriting by the user before the user completes input of handwriting. In other words, the detection unit 24 detects that the user has started inputting by handwriting and which of the multiple input fields has been input by handwriting.

[0045] The determination unit 26 determines whether or not the input field detected by the detection unit 24 is a specific input field that has been set in advance. Before the user completes handwritten input, the determination unit 26 determines whether or not the detected input field is a specific input field that has been set in advance.

[0046] The communication unit 25 controls communication with the server device 3 and the terminal device 4 via the communication network 100. The communication unit 25 transmits and receives information. Examples of the communication unit 25 include a transmitting unit and a receiving unit.

[0047] The control unit 27 controls processing corresponding to a predetermined operation according to the selection of the display component.

[0048] The terminal device 4 has a setting reception unit 41, a display control unit 42, and a communication unit 43. Each function of the terminal device 4 is a function or means realized by operating in accordance with an instruction from a CPU in accordance with a program loaded from an auxiliary storage device onto a RAM.

[0049] The setting receiving unit 41 receives settings for specific input fields that have been manually set by the user, and further sets specific input fields automatically.

[0050] The display control unit 42 controls the display of a screen displayed on the display 220 of the terminal device 4. The screen whose display is controlled is a screen for setting a specific input field, etc.

[0051] The communication unit 43 communicates with other devices such as the display device 2 and the server device 3 via the communication network 100.

[0052] Next, an example of a screen displayed on the display 220 of the display device 2 will be described with reference to FIG.

[0053] FIG. 5 is a diagram showing a screen transition when an input is made to a specific input field on the display device.

[0054] FIG. 5A is a diagram showing an example of an input screen 5001 displayed on the display 220 of the display device 2. As shown in FIG.

[0055] FIG. 5B is a diagram showing an example of a selection screen 5002 displayed on the display 220 of the display device 2. As shown in FIG.

[0056] A plurality of input fields are displayed on input screen 5001 by display control unit 21. The plurality of input fields include, for example, a description format input field 501, a name input field 502, an age input field 503, a birth date and address input field 504, and a date and time input field 505. Of these plurality of input fields, the date and time input field is set as a specific input field, and identification information that identifies the date and time input field as a specific input field is stored in storage unit 23.

[0057] When the user completes inputting the date and time into the input fields on the input screen 5001, the display control unit 21 displays a selection screen 5002 (FIG. 5(B)). The selection screen 5002 displayed at this time may be displayed by switching from the input screen 5001, or may be displayed so as to be superimposed on the input screen 5001. It may also be displayed as a pop-up. FIG. 5(B) shows an example in which the selection screen 5002 is displayed by switching from the input screen 5001.

[0058] Selection screen 5002 includes display components 511 to 515. In selection screen 5002, display component 511 displays "Display next page," display component 512 displays "Add blank page," display component 513 displays "Add new page," display component 514 displays "Save to cloud," and display component 515 displays "Back." The number and types of display components are not limited to these. Only one display component may be displayed on selection screen 5002. Another display method for display component 510 is a display method that does not switch screens to display the component. The display method that does not switch screens to display the component will be described later.

[0059] Next, the processing of the display device 2 will be described with reference to FIG.

[0060] 6 is a flowchart showing the operation of the display device 2. This flowchart shows the operation when the input screen 5001 is displayed on the display 220 of the display device 2.

[0061] In step S1, the detection unit 24 detects that handwritten input has been made in any of the input fields on the screen displayed on the display 220 by the display control unit 21.

[0062] In step S2, the determination unit 26 determines whether the input field into which input has been made, detected by the detection unit 24, is a specific input field. If it is determined in step S2 that the input field into which input has been made is a specific input field, the process proceeds to step S3.

[0063] In step S3, the determination unit 22 determines whether or not input into the input field determined by the determination unit 26 to be a specific input field has been completed. That is, the determination unit 22 determines that input into the specific input field entered by the user has been completed. If the determination unit 22 determines in step S3 that input has been completed, the process proceeds to step S5. If it is not determined in step S that input has been completed, the process proceeds to step S1.

[0064] The order of the processes in steps S1 to S3 by the detection unit 24, the determination unit 26, and the judgment unit 22 may be reversed as long as it is possible to determine in the processes of steps S1 to S3 whether or not the user's handwritten input in a specific input field has been completed.

[0065] If the judgment unit 26 determines in step S2 that the input field detected by the detection unit 24 is not a specific input field, and if the judgment unit 22 determines in step S3 that input into the specific input field has not been completed, the process proceeds to step S1, and the display control unit 21 continues to display the input screen 5001 without displaying any display components so that the user can continue to enter handwritten input.

[0066] In step S5, the display control unit 21 displays, on the display 220, a display component that accepts a selection operation and that is stored in advance in the storage unit 23. The number of display components displayed by the display control unit 21 may be one or more. Details of the displayed display component will be described later.

[0067] In step S6, the control unit 27 determines whether a selectable display element has been selected. If it is determined that a display element has been selected, the control unit 27 controls processing corresponding to a predetermined operation according to the selected display element. Next, an example of control of processing corresponding to the predetermined operation will be described.

[0068] Step S7 is a process performed when a display component that accepts an instruction to display the next page is selected in step S6. When a display component that accepts an instruction to display the next page is selected, control unit 27 recognizes the page next to the page displayed on input screen 5001 and performs control to display the next page. Display control unit 21, under the control of control unit 27, causes display 220 to display the next page.

[0069] Step S8 is a process performed when a display component that accepts an instruction to add a blank page is selected in step S6. When a display component that accepts an instruction to add a blank page is selected, the control unit 27 adds a blank page as the next page of the page displayed on the input screen 5001 and controls the display 220 to display the added blank page in accordance with the control of the control unit 27.

[0070] Step S9 is a process performed when a display component that accepts an instruction to add a new page is selected in step S6. When a display component that accepts an instruction to add a new page is selected, the control unit 27 adds a page having the same input fields as the input fields displayed on the input screen 5001 as a new page and controls the display 220 to display the added new page. At this time, if the user has already entered handwritten or other input on the input screen 5001, the new page may be a page from which the user's handwritten or other input has been deleted, or a format including multiple input fields that is pre-stored in the storage unit 23 may be used as the new page. In other words, the new page is a page with no input fields. The display control unit 21 controls the display 220 to display the new page in accordance with the control of the control unit 27.

[0071] Steps S10 to S12 are processes performed when a display component that accepts an instruction to save in the cloud is selected in step S6.

[0072] In step S10, when a display component that accepts an instruction for an operation to save to the cloud is selected, the control unit 27 controls sign-in to the cloud. Specifically, if the user has already signed in to the cloud, the control unit 27 controls sign-in to the cloud without requiring a sign-in operation by the user. If the user has not signed in to the cloud, the control unit 27 requests the user to sign in to the cloud, and controls sign-in to the cloud when the sign-in operation is performed.

[0073] In step S11, the control unit 27 requests the user to select a destination for saving the data. Once the user has selected a destination for saving the data, the process proceeds to step S12.

[0074] In step S12, the control unit 27 stores the contents input on the input screen 5001 and the file input on the input screen 5001 in the storage destination selected in step S11.

[0075] Step S13 is a process performed when a display component that accepts a back operation instruction is selected in step S6. In step S6, when the display component is displayed by switching the screen from the input screen 5001 or by a pop-up, and the display component that accepts a back operation instruction is selected, the control unit 27 identifies the input screen 5001 that was displayed before the screen was switched or the pop-up was displayed, and performs control to return to the input screen 5001. The input screen 5001 includes the content that was handwritten by the user before the screen was switched or the pop-up was displayed. The display control unit 21 displays the back screen, i.e., the input screen 5001, in accordance with the control of the control unit 27.

[0076] Next, an example of screen transitions and an example of a screen displayed in step S7 will be described with reference to FIG.

[0077] FIG. 7 is a diagram showing screen transitions when a display component that accepts an instruction to display the next page is selected on the display device.

[0078] Fig. 7(A) shows a selection screen 5002. When a display component 511 "Display next page," which is an example of a display component that accepts an instruction to display the next page, is selected on the selection screen 5002, the screen transitions to a screen 5003 shown in Fig. 7(B).

[0079] A screen 5003 in FIG. 7B is a screen that displays the page next to the page that was displayed on the input screen 5001.

[0080] Next, an example of screen transitions and an example of a screen displayed in step S8 will be described with reference to FIG.

[0081] FIG. 8 is a diagram showing screen transitions when a display component that accepts an instruction for an operation to add a blank page is selected on the display device.

[0082] Fig. 8(A) shows a selection screen 5002. When a display component 512 "Add blank page," which is an example of a display component that accepts an instruction for an operation to add a blank page, is selected on the selection screen 5002, the screen transitions to a screen 5004 shown in Fig. 8(B).

[0083] Unlike the input screen 5001, the screen 5004 in FIG. 8B is a blank page that does not have an input field with a defined input area, and allows input to be made anywhere.

[0084] Next, an example of screen transitions and an example of a screen displayed in step S9 will be described with reference to FIG.

[0085] FIG. 9 is a diagram showing screen transitions when a display component that accepts an instruction for an operation to add a new page is selected on the display device.

[0086] 9A shows a selection screen 5002. When a display component 513 "Add a new page," which is an example of a display component that accepts an instruction for an operation to add a new page, is selected on the selection screen 5002, the screen transitions to a screen 5005 shown in FIG.

[0087] Unlike the input screen 5001 on which the user has entered handwritten input, the screen 5005 in FIG. 9B is a new page in which no input has been entered into the input fields.

[0088] Next, an example of screen transitions and an example of screens displayed in steps S10 and S11 will be described with reference to FIG.

[0089] FIG. 10 is a diagram showing screen transitions when a display component 514 for receiving an instruction to save in the cloud is selected on the display device.

[0090] Fig. 10(A) shows a selection screen 5002. When display component 514 "Save to cloud," which is an example of a display component that accepts an instruction to save to the cloud, is selected on selection screen 5002, the screen transitions to screen 5006 in Fig. 10(B) or screen 5007 in Fig. 10(C).

[0091] A screen 5006 in FIG. 10B is displayed when the display component 514 "Save to cloud" is selected and the user is not signed in to the cloud, and is a screen that prompts the user to sign in.

[0092] Screen 5007 in Figure 10(C) is a screen that is displayed when the user signs in on a screen requesting a sign-in request, or when display component 514 "Save to cloud" is selected on selection screen 5002 and the user is already signed in to the cloud, and is a screen for selecting the folder in which to save the file.

[0093] Next, an example of screen transition and an example of a screen displayed in step S13 will be described with reference to Fig. 11. Fig. 11 shows an example in which the display of an input screen 5001 and a selection screen 5002 is switched by a screen transition.

[0094] FIG. 11 is a diagram showing a screen transition when a display component that accepts a back operation instruction is selected on the display device.

[0095] Fig. 11(A) shows the selection screen 5002. When a display component 515 "Back," which is an example of a display component that accepts a back operation instruction, is selected on the selection screen 5002, the screen transitions to the screen shown in Fig. 11(B).

[0096] The screen in Fig. 11(B) is an input screen 5001 that includes the same input fields and input content as the input screen 5001 that was displayed before the selection screen 5002 was displayed. The input screen 5001 displays the input content that was handwritten before the selection screen 5002 was displayed. In addition, when the user starts inputting into an input field on the input screen 5001 in Fig. 11(B), a complete button 1201 may be displayed in that input field. When the user completes inputting into that input field and operates the complete button 1201, the screen switches back to the selection screen 5002.

[0097] In this way, if the user selects to return to enter data in the input field on selection screen 5002, which is displayed after it has been determined that the user has completed their input on input screen 5001, it is expected that the user wishes to continue entering data. In this case, by displaying the complete button 1201 on input screen 5001 as shown in Fig. 11(B), it is possible to display selection screen 5002, which allows the user to select the next operation at any timing desired by the user, unlike input screen 5001 that was displayed before selection screen 5002 was displayed.

[0098] On the input screen 5001, when the user is entering data into a certain input field, the completion button 1201 is displayed in a predetermined position, such as the bottom right of that input field. However, if the user enters data into another input field without operating the completion button 1201, the completion button 1201 will be displayed in a predetermined position, such as the bottom right of that other input field.

[0099] Fig. 12 is a flowchart showing the processing when "Back" is selected from display component 515 in step S6. The flowchart in Fig. 12 shows the processing after input screen 5001 is displayed on display 220 by display control unit 21 in step S13 in Fig. 6.

[0100] In step S14, the detection unit 24 detects an input field in which the user has entered handwritten input.

[0101] In step S15, the display control unit 21 displays a completion button 1201 in the input field detected by the detection unit 24.

[0102] In step S16, the detection unit 24 detects that the Complete button 1201 displayed on the display 220 has been operated, and if the Complete button has been operated, the process proceeds to step S17. In step S17, the display control unit 21 displays the selection screen 5002, and the process ends. In S16, if the Complete button has not been operated, the detection unit 24 returns to S14 and detects that an entry has been made in the input field.

[0103] In step S16, if the completion button 1201 is not operated, the process returns to the process before step S14. At this time, the display control unit 21 causes the display 220 to continue displaying the input screen 5001.

[0104] FIG. 13 is a diagram illustrating a mode in which the display component 510 is displayed without switching the screen.

[0105] FIG. 13B shows an input screen 5001 including a display area 1301 that displays display components 511 to 515.

[0106] 13(A), when the determination unit 22 determines that input into a specific input field, which is an input field detected by the detection unit 24, has been completed, the control unit 27 performs control to display a display area 1301 on the input screen 5001. The display control unit 21, in accordance with the control of the control unit 27, causes the display 220 to display the display area 1301.

[0107] The display area 1301 is displayed in a portion of the screen that was previously displayed as the input screen 5001. This allows the display component 510 to be displayed on the display 220 without switching the screen.

[0108] When the input screen 5001 including the display area 1301 is displayed on the display 220, if a display component 510 that accepts an instruction for a back operation, for example, the display component 515 "back," is selected, the control unit 27 may perform control so as not to display the display area 1301. In this case, the display control unit 21, in accordance with the control of the control unit 27, causes the input screen 5001 that does not include the display area 1301 to be displayed on the display 220.

[0109] 12, which shows the process when the display component 515 "Back" is selected, may be performed, and the input screen 5001 including the complete button 1201 may be displayed instead of the input screen 5001 not including the display area 1301. When the complete button 1201 is operated on the input screen 5001 including the complete button 1201, the control unit 27 may perform control so that the display area 1301 is displayed on the screen again.

[0110] Next, the settings of specific input fields will be explained.

[0111] Next, the information in the input field will be described with reference to FIG.

[0112] 14 shows an example of information in input fields contained in a file to be displayed by the display device 2 in this embodiment. When the screen to be displayed is a website, the information in the following input fields is held by the website. Note that, although information in the input fields is generally included in the display data, the display data and the information in the input fields corresponding to that display data may be managed separately.

[0113] The input field number 701 is identification information that identifies the input field included in the file.

[0114] The input field name 702 is the name set for the input field.

[0115] Attribute 703 is an attribute of the input field that is set for the input field. The attribute of the input field is information that specifies the display mode and input format of the content entered in that input field. For example, if an input field that only accepts input in katakana is set, the attribute [katakana] is set. If an input field that displays the entered content center-justified is set, the attribute [center] is set. Attributes that identify specific input fields are set for specific input fields.

[0116] In this embodiment, the attribute [last_form] is set as an attribute that identifies a specific input field. In the example of Fig. 14, the attribute: last_form is set for the input field with input field number: N and input field name: date and time.

[0117] In the example of FIG. 14, no attribute is set for input field number: 2, input field name: address.

[0118] Multiple attributes may be set for one input field. In the example of Fig. 14, the attributes [numerics] and [center] are set for input field number: 3, input field name: phone number. The attribute [numerics] indicates that only numbers can be entered, and the attribute [center] indicates that the content entered in the input field will be displayed center-justified.

[0119] Note that the number of specific input fields set on one page may not be one, but multiple fields. In this case, when it is determined that input into one specific input field among the multiple specific input fields has been completed, the display component 510 is displayed.

[0120] The information in the input fields is not limited to the above. It is sufficient if the information can identify a specific input field among multiple input fields. For example, the information may be information in which a flag is assigned to only a specific input field among multiple input fields.

[0121] The display device 2 identifies a specific input field based on the information about the input field included in the display data, such as a file or HTML. The display data may be transmitted from the terminal device 4 to the display device 2 via a communication network, or the display data transmitted from the terminal device 4 to the server device 3 may be transmitted to the display device 2. The communication unit 25 of the display device 2 receives the transmitted display data, and the display control unit 21 displays the input screen 5001 on the display 220 based on the received display data. Furthermore, the determination unit 26 of the display device 2 determines whether the input field detected by the detection unit 24 is a specific input field that has been set in advance, based on the information about the input field included in the display data. In other words, it determines whether information for identifying the specific input field is set in the input field into which the user has entered data.

[0122] FIG. 15 is a diagram showing the settings of a specific input field.

[0123] As shown in FIG. 15, on a page including multiple input fields, the configuration and location of the input fields vary. On each page, the user can set any specific input field from among the multiple input fields. The operation for setting a specific input field is performed by the user on the terminal device 4, and the specific input field is set for display data such as a file or HTML. The display data such as a file or HTML contains information about the input field in which the specific input field is set.

[0124] The example of screen 1501 shows that the input field in the upper right corner is set as a specific input field as an example of a field arbitrarily selected by the user from among multiple input fields. The user can select and set the input field to be set as the specific input field from among the multiple input fields.

[0125] Screens 1502 to 1505 are screens that respectively display multiple pages included in the display data. Screens 1502 to 1505 each have multiple input fields arranged in a different layout. The input fields at the bottom right of each page (screen) can be set to specific input fields all at once. The method for setting all at once will be described later.

[0126] Next, the process of setting a specific input field for information in the input field of the display data will be described with reference to FIG.

[0127] 16 is a flowchart relating to the setting of a specific input field. The setting of a specific input field is set by the user on the terminal device 4.

[0128] In step S21, the display control unit 42 displays a screen on the display 220 of the terminal device 4 for accepting a selection as to whether or not to set a specific input field, and the user can select whether or not to set a specific input field. If the user selects not to set a specific input field, the process proceeds to step S22. If the user selects to set a specific input field, the process proceeds to step S22.

[0129] In step S22, no specific input field is set in the input field information of the display data, and the setting process ends. Because no specific input field is set, when the display data is displayed on the display device 2, the display component 510 will not be selected no matter what input field is entered.

[0130] In step S23, the display control unit 42 displays on the display 220 a screen for accepting a selection of whether or not to set specific input fields collectively, and the user can select whether or not to set specific input fields collectively. If the user selects not to set specific input fields collectively, the process proceeds to step S24. If the user selects to set specific input fields collectively, the process proceeds to step S25.

[0131] In step S24, the display control unit 42 displays on the display 220 a screen that allows the user to set any input field on the page as a specific input field, and the user can set any input field on the page as a specific input field. In other words, the user manually sets the specific input field. The setting method can be to select an input field from multiple input fields to set as the specific input field, or to assign identification information that identifies the specific input field. The setting receiving unit 41 receives the setting of the specific input field set by the user.

[0132] In step S25, the setting receiving unit 41 uses a computer to identify the coordinates of the bottom right corner of the page among the coordinates of the vertices of the rectangles of the multiple input fields included on each page. Once the coordinates are identified, the process proceeds to step S26.

[0133] In step S26, the setting receiving unit 41 sets the input field having the vertices of the coordinates of each page identified in step S25 as a specific input field by the computer, and the setting process is completed.

[0134] In the above explanation, the reason why the input field located at the bottom right is set as the specific input field is because of the custom of inputting from the top left to the bottom right of the screen. When handwritten input is performed in the input field at the top left, the input field at the bottom right is the last to be input. When the display component 510 is displayed upon completion of handwritten input in the last input field, the user can smoothly select the next operation.

[0135] As described above, by setting specific input fields in advance, it is possible to solve the problem of users becoming confused about how to perform other operations after handwriting input. Furthermore, when the display device 2 is used by multiple users, even users who are unfamiliar with the operation can input handwriting and then save or switch pages simply by completing the handwriting input by setting up the display device 2 in advance, so all users do not need to remember all the operation methods.

[0136] In the above example, the operation of setting a specific input field in the terminal device 4 using the setting reception unit 41 possessed by the terminal device 4 was described, but by giving the display device 2 the functionality of the setting reception unit 41, the process of setting a specific input field in the above-mentioned display data may also be realized in the display device 2.

[0137] <Modification> Next, a modification of the above-described embodiment will be described.

[0138] 17 is a functional block diagram of a modified example, in which the display device 2 further includes a recognition unit 28 in addition to the functional blocks described in FIG.

[0139] The recognition unit 28 changes a specific input field on the input screen 5001 in accordance with the input handwritten by the user.

[0140] FIG. 18 is a diagram showing an example of a screen displayed in the modified example.

[0141] In this modification, similarly to the above-described embodiment, the display control unit 21 of the display device 2 displays an input screen 5001 on the display 220. The input screen 5001 shown in FIG. 18(A) is a screen in a state where no handwritten input has been made by the user, that is, a screen in an uninputted state. The input screen 5010 shown in FIG. 18(B) is a diagram showing the user inputting handwritten input into a specific input field and then inputting handwritten input into a new specific input field. Details of FIGS. 18(A) and 18(B) will be described later.

[0142] FIG. 19 is a flowchart showing a modified example.

[0143] In step S31, similar to step S1, the detection unit 24 detects that input has been made into any of the multiple input fields on the screen displayed on the display 220 by the display control unit 21 that have not been filled in yet.

[0144] In step S32, the determination unit 26 determines whether the input field into which an input has been made, detected by the detection unit 24, is a specific input field, as in step S2.

[0145] In step S33, the determination unit 22 determines whether or not input into the input field determined by the determination unit to be the specific input field has been completed, similar to step S3. If it is determined in step S33 that input into the input field has not been completed, the process proceeds to step S34, where the user can continue inputting on the input screen 5001.

[0146] After proceeding to step S34, the process returns to step S31 again.

[0147] In step S35, if it is determined in step S33 that input into the specific input field is complete, the determination unit 22 further determines whether input fields other than the specific input field are left empty. In other words, the determination unit 22 determines whether the user has input into the specific input field when all of the multiple input fields are empty.

[0148] In step S36, if the determination unit 22 determines that the user has entered information into the specific input field when all of the multiple input fields are left blank, unlike the embodiment described above, the control unit 27 does not control the display components to be selectable, and the recognition unit 28 recognizes an input field among the multiple input fields that is different from the specific input field as a new specific input field. Then, the process proceeds to step S37.

[0149] In step S37, the detection unit 24 detects that input has been made in any of the multiple input fields on the screen displayed on the display 220 by the display control unit 21, as in steps S1 and S31.

[0150] In step S38, the determination unit 26 determines whether the input field into which input has been made, detected by the detection unit 24, is a new specific input field. If the determination unit 26 determines that it is not a new specific input field, the process proceeds to step S34. If the determination unit 26 determines that it is a new input field, the process proceeds to step S39.

[0151] In step S39, the determination unit 22 determines whether or not input into the input field determined by the determination unit 26 to be a new specific input field has been completed.

[0152] In step S40, if the decision unit 22 decides that input into the new specific input field has been completed, the same processes as steps S5 to S13 in FIG. 6 are executed.

[0153] Fig. 18(B) shows an input screen 5010 in the case where a new specific input field has been set. As shown in Fig. 18(A), when the input screen 5001 is empty and the user first performs handwritten input into a specific input field, the recognition unit 28 recognizes the new specific input field, and even after the input into the specific input field is completed, the selection screen 5002 including the display component 510 is not displayed, and input can continue on the input screen 5010 in which the new specific input field has been recognized.

[0154] In the example of FIG. 18(B), the input field immediately above the specific input field 1701 set at the bottom right is recognized by the recognition unit 28 as a new specific input field 1702.

[0155] When the user inputs information into the new specific input field 1702 and the judgment unit 22 determines that the input into the new specific input field 1702 has been completed, the display control unit 21 displays the selection screen 5002 including the display component 510 (Figure 18 (C)).

[0156] In this way, when handwritten input is made in a specific input field when input into multiple input fields has not yet been made, i.e., when handwritten input is made in a specific input field first on a displayed page, the user may want to make further input into input fields other than the specific input field. In such a case, it would be inconvenient for the user to have the display component 510 displayed when input into a specific input field is completed. In such a case, by having the recognition unit 28 recognize a new specific input field as described above, the display component 510 is not displayed even when handwritten input into a specific input field is completed, allowing the user to continue inputting on the input screen 5010, and the display component 510 can be displayed when input into the new specific input field is completed.

[0157] At this time, the recognition unit 28 may recognize the input field located to the upper right of the specific input field as a new specific input field. Because users have the habit of inputting data from the upper left to the lower right, even if the user mistakenly inputs data into the specific input field located at the bottom right, it is highly likely that the user will next input data from the input field located at the upper left, and the input field located to the upper right of the specific input field will be the last to be input. The recognition unit 28 may recognize a new input field different from the specific input field using AI or the like.

[0158] In step S35 described above, it is determined whether or not input has been made in all input fields other than the specific input field, but different criteria may be used for this determination. For example, it may be determined whether or not input has been made in predetermined input fields other than the specific input field. In other words, by determining whether or not input has been made in input fields other than the specific input field, it is possible to prevent the selection screen 5002 from being displayed when the user intends to continue inputting.

[0159] In the above-described embodiment and modified examples, the display control unit 21 may display the specific input field and the new specific input field in a distinguishable manner. For example, the display control unit 21 may display the specific input field and the new specific input field in a color that distinguishes them from other input fields, surround them with a color that distinguishes them from other input fields, surround them with a frame that is thicker than other input fields, or display messages such as "Please enter data in this input field last" or "Once you enter data, a selection screen will be displayed" for the specific input field and the new specific input field. These display formats are merely examples. It is sufficient for the user to recognize that the specific input field and the new specific input field are different from other input fields. Furthermore, in the above-described embodiment and modified examples, the display device 2 includes the display 220, but this is not limited thereto. The display device 2 may display each screen described in the above-described embodiment and modified examples on a display unit externally connected via a wired or wireless connection.

[0160] Although the present invention has been described above based on the embodiments, the present invention is not limited to the requirements shown in the above embodiments. These requirements can be changed without departing from the spirit of the present invention, and can be appropriately determined depending on the application form. [Explanation of symbols]

[0161] 1. Communication Systems 2 Display device 21 Display control unit 22 Judgment Department 23 Memory section 24 Detection unit 25 Communications Department 26 Judgment section 27 Control Unit 28 Recognition part 3. Server equipment 4 Terminal Devices 41 Settings reception section 42 Display control unit 43 Communications Department 100 Communication Network [Prior art documents] [Patent documents]

[0162] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-225209

Claims

1. a determination unit that determines whether handwritten input has been completed in a specific input field that is a preset input field among a plurality of input fields that are displayed on the display unit and that accept handwritten input by a user; a display control unit that, when the determination unit determines that the handwriting input into the specific input field has been completed, causes a display component that accepts a predetermined operation instruction to be selectable on a display unit; and a control unit that, when the display component is selected by a user, executes a process corresponding to a predetermined operation corresponding to the selected display component; An information processing device comprising:

2. the display control unit displays a plurality of display components in a selectable manner; The control unit executes a process in response to a predetermined operation corresponding to a selected display element from among the plurality of display elements. The information processing device according to claim 1 .

3. The information processing device includes: a detection unit that detects an input field in which a user has entered handwritten input among the plurality of input fields; a determination unit that determines whether the input field detected by the detection unit is the specific input field; and when the determination unit determines that the input field into which handwritten input has been made is the specific input field, the determination unit determines whether input into the specific input field has been completed. The information processing device according to claim 1 .

4. the determination unit determines that the handwritten input into the specific input field has been completed when no handwritten input into the specific input field has been made for a predetermined period after the user has made a handwritten input into the specific input field; The information processing device according to claim 3 .

5. The determination unit determines that input into the specific input field has been completed in response to a user performing an input completion operation for the specific input field. The information processing device according to claim 3 .

6. the display component receives an instruction for an operation to display a second screen that is a page subsequent to the page displayed on the first screen including the plurality of input fields; the control unit displays the second screen, which is the next page, in response to a selection of an operation to display the second screen. The information processing device according to claim 1 .

7. The first screen including the plurality of input fields is a screen for displaying a file including the plurality of input fields. The information processing device according to claim 1 .

8. the display component receives an instruction for an operation to store the file, for which input has been made into the plurality of input fields by a user, in a storage unit; the control unit stores the file in the storage unit in response to an instruction for an operation to store the file in the storage unit; The information processing device according to claim 7 .

9. The display component receives an instruction for an operation to add a page, the control unit adds a page in response to an instruction for an operation to add a page. The information processing device according to claim 1 .

10. The display component receives an instruction for an operation to add a blank page, the control unit adds a blank page in response to an instruction for an operation to add a blank page; The information processing device according to claim 9 .

11. the display component accepts an instruction for an operation to add a new page for inputting the plurality of input fields; the control unit adds a new page for inputting the plurality of input fields in response to an instruction for an operation to add a new page for inputting the plurality of input fields, The information processing device according to claim 9 .

12. When it is determined that input into the specific input field is completed, the display control unit switches from the first screen including the plurality of input fields to a selection screen that selectably displays a display component that accepts selection of a predetermined operation, or displays the selection screen as a pop-up. The information processing device according to claim 1 .

13. the display component accepts an instruction for an operation to return to the first screen, The control unit returns to the first screen in response to an instruction for an operation to return to the first screen. The information processing device according to claim 12.

14. when it is determined that input into the specific input field is completed, the display control unit displays a display component for accepting selection of a predetermined operation on a first screen including the plurality of input fields; The information processing device according to claim 1 .

15. The display control unit displays the specific input field and input fields other than the specific input field among the plurality of input fields in different modes. The information processing device according to claim 1 .

16. a recognition unit that, when the determination unit determines that the user has made a handwritten input into the specific input field among the plurality of input fields while the input fields other than the specific input field are left blank, recognizes the input field different from the specific input field as a new specific input field; and the determination unit determines that input into the new specific input field has been completed when the recognition unit has set a new specific input field; The display control unit, when it is determined by the determination unit that input into the new specific input field has been completed, selectably displays the display component that accepts selection of a predetermined operation, and does not display the display component when it is determined by the determination unit that input into the specific input field has been completed. The information processing device according to claim 1 .

17. a step of determining that handwritten input has been completed in a specific input field, which is a preset input field, among a plurality of input fields displayed on a display unit and which accept handwritten input by a user; a step of displaying, on a display unit, a selectable display component for receiving a predetermined operation instruction when it is determined that the handwritten input into the specific input field has been completed; a step of controlling a process in response to a predetermined operation instruction when the display component is selected by a user; A method characterized by being executed by an information processing device.

18. a step of determining that handwritten input has been completed in a specific input field, which is a preset input field, among a plurality of input fields displayed on a display unit and which accept handwritten input by a user; a step of displaying, on a display unit, a selectable display component for receiving a predetermined operation instruction when it is determined that the handwritten input into the specific input field has been completed; a step of controlling a process in response to a predetermined operation instruction when the display component is selected by a user; A program characterized by causing an information processing device to execute the above.

Images