Display device, method for display, and program

Abstract

To provide a display device by which a user can set the connection between one object and another object.SOLUTION: The display device is for displaying a handwriting-input object, and includes: reception means for receiving a first object displayed by the display device and a second object related to the first object; and setting means for relating the first object and the second object to each other.SELECTED DRAWING: Figure 20

Description

[Technical Field]

[0001] The present invention relates to a display device, a display method, and a program. [Background technology]

[0002] Display devices that use handwriting recognition technology to convert handwritten data into characters and display them on a display are known. Display devices with relatively large touch panels are placed in conference rooms and public facilities and are used by multiple users as electronic whiteboards, etc.

[0003] In such a display device, the page size becomes too large, and there are cases where related writings (also called objects) exist in areas far from the handwriting area, and it is also possible to lose track of which pages already have writings related to the handwriting content.

[0004] To address this problem, a technology has been devised for automatically extracting and displaying related descriptions (see, for example, Patent Document 1). Patent Document 1 discloses a display device that, when a specific object is selected from files, handwritten information, etc., identifies portions of the file or handwritten information that are related to the automatically selected object, and, when displaying the selected object, displays the identified group of objects so that they can be referenced. Summary of the Invention [Problem to be solved by the invention]

[0005] However, the conventional technology does not allow the user to set the association between objects.

[0006] In view of the above-mentioned problems, an object of the present invention is to provide a display device that allows a user to set associations between objects. [Means for solving the problem]

[0007] In view of the above problems, the present invention provides a display device that displays handwritten input objects, comprising: a receiving means that receives a first object displayed by the display device and a second object associated with the first object; and a setting means that associates the first object with the second object. [Effects of the Invention]

[0008] A display device can be provided that allows a user to set object-to-object relationships. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 10 is a diagram showing an example of a display range displayed by a display device in which one page is infinitely large. [Figure 2] FIG. 10 is a diagram showing an example of an entire page. [Figure 3] FIG. 10 is a diagram showing an example of a display of one page in which objects are associated with each other. [Figure 4] FIG. 10 is a diagram showing an example of displaying associated objects. [Figure 5] FIG. 10 is a diagram illustrating an example of an object associated with a page. [Figure 6] FIG. 10 is a diagram illustrating an example of selecting an object associated with a page. [Figure 7] FIG. 10 is a diagram showing an example of a display of an associated page. [Figure 8] FIG. 1 is a diagram illustrating an example of the overall configuration of a display device. [Figure 9] FIG. 2 is a diagram illustrating a hardware configuration of an example of a display device. [Figure 10] FIG. 2 is an example of a functional block diagram illustrating functions of the display device, divided into blocks. [Figure 11] FIG. 10 is a diagram showing an example of objects that an entire page has. [Figure 12] FIG. 10 is a diagram showing an example (part 1) of a screen displayed by the display device. [Figure 13]FIG. 10 is a diagram showing a second example of a screen displayed by the display device. [Figure 14] FIG. 10 is a diagram showing a third example of a screen displayed by the display device. [Figure 15] FIG. 10 is a diagram showing an example (part 4) of a screen displayed by the display device. [Figure 16] FIG. 10 is a diagram showing a fifth example of a screen displayed by the display device. [Figure 17] FIG. 10 is a diagram illustrating the specification of a link source object. [Figure 18] FIG. 10 is a diagram illustrating the specification of a link destination object. [Figure 19] FIG. 10 is a diagram illustrating the addition of a record to a link storage unit. [Figure 20] 10 is a sequence diagram illustrating an example of a procedure in which the display device sets an association between two objects. [Figure 21] FIG. 10 is a diagram illustrating an example of selection of a link source object. [Figure 22] FIG. 10 is a diagram showing an example of a display range including a link destination object. [Figure 23] FIG. 10 is a diagram illustrating a search for a link source object in an object storage unit. [Figure 24] FIG. 10 is a diagram illustrating a search for a link source object in a link storage unit. [Figure 25] FIG. 10 is a diagram illustrating a search for a link destination object in an object storage unit. [Figure 26] 10 is an example of a sequence diagram illustrating a procedure in which a display device displays a link destination object based on link information when a user presses a link source object. [Figure 27] FIG. 10 is a diagram showing an example of stroke data handwritten by a user to select an area. [Figure 28] 10 is a diagram illustrating an area set as a link destination in a link storage unit. FIG. [Figure 29] 10 is a sequence diagram illustrating an example of a procedure in which the display device sets an association between an object and an area. [Figure 30]10 is a flowchart illustrating an example of a procedure for a display control unit to determine a display magnification. [Figure 31] 10A and 10B are diagrams illustrating selection of a link source object in a state where the display range is expanded. [Figure 32] FIG. 10 is a diagram showing an example of a link destination object displayed at the current display magnification. [Figure 33] FIG. 10 is a diagram illustrating a change in display magnification. [Figure 34] FIG. 10 is a diagram showing an example of a link destination object displayed in the center or upper left of the display. [Figure 35] 10 is an example of a flowchart illustrating a process in which a link specifying unit controls a display position of a link destination object or a link destination area. [Figure 36] FIG. 10 is a diagram illustrating an example of an object list displayed by a display control unit. [Figure 37] 10 is a sequence diagram illustrating an example of a procedure in which the display device sets an association between two objects. [Figure 38] FIG. 10 is a diagram illustrating an example of selection of a link source object. [Figure 39] FIG. 39 is a diagram showing an example of a subview including a link destination object associated with the link source object selected in FIG. 38. [Figure 40] FIG. 10 is a diagram showing an example of a link source object that a user presses. [Figure 41] FIG. 10 is a diagram showing an example of a subview that displays a link destination object associated with a link source object. [Figure 42] FIG. 10 is a sequence diagram illustrating an example of a procedure for displaying a subview. [Figure 43] FIG. 10 is a diagram illustrating the reservation of a linked object. [Figure 44] FIG. 10 is a diagram illustrating the association of two objects by reserving a link destination object. [Figure 45] FIG. 10 is an example of a sequence diagram illustrating the association of two objects using reserved. [Figure 46]10A and 10B are diagrams illustrating pressing of a link source object to which a plurality of link destination objects are set. [Figure 47] FIG. 10 is a diagram showing an example thumbnail of a linked object presented to a user. [Figure 48] FIG. 48 is a diagram showing an example of a display range displayed on the display device when the link destination object in FIG. 47 is selected. [Figure 49] 10 is an example of a sequence diagram illustrating a procedure in which a display device displays a link destination object when a user selects any link destination object displayed as a thumbnail. [Figure 50] FIG. 10 is a diagram illustrating an example of a time-series object list held by a display control unit. [Figure 51] FIG. 10 is a diagram illustrating a preview function. [Figure 52] FIG. 10 is a diagram showing an example of a link destination object displayed on a display device. [Figure 53] FIG. 10 is a diagram showing an example of a display range displayed when a preview button is pressed. [Figure 54] 10 is a flowchart illustrating an example of a procedure in which a display device displays a link destination object in accordance with a preview function or a next function. [Figure 55] FIG. 10 is a diagram illustrating another configuration example of the display device. [Figure 56] FIG. 10 is a diagram illustrating another configuration example of the display device. [Figure 57] FIG. 10 is a diagram illustrating another configuration example of the display device. [Figure 58] FIG. 10 is a diagram illustrating another configuration example of the display device. DETAILED DESCRIPTION OF THE INVENTION

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A display device and a display method performed by the display device will be described below as an example of an embodiment of the present invention with reference to the accompanying drawings. [Example]

[0011] <Comparative Example> First, a comparative example suitable for explaining the display device of this embodiment will be described with reference to Figures 1 and 2. Figure 1 shows an example of a screen displaying a partial display range of an infinitely large page. Figure 2 shows the entire page. The dotted line 302 in Figure 2 indicates the display range in Figure 1.

[0012] In Figures 1 and 2, the three objects 301 in the upper left, "Task A," "Task B," and "Task C," are each shown with lines to indicate other objects that are related to each other. Even if a user associates objects with lines of different colors, multiple lines will intersect, making it difficult to understand the relationships between the objects.

[0013] <Outline of the display device according to this embodiment> An overview of the display device of this embodiment will be described with reference to Figures 3 and 4. Figure 3 shows an example of a display of one page in which objects are associated with each other. Figure 3 shows the area indicated by dotted line 302 on one page in Figure 2. In Figure 3, an object 311 called "Assignment C" in the upper left and an object 312 called "Explanation of Assignment C" in the lower right (which does not fit in Figure 3) are associated, as will be described later.

[0014] When the user presses the object 311 called "Task C" as shown in Fig. 3, the display device identifies the object 312 called "Explanation of Task C" that is associated with the object 311 called "Task C." Then, as shown in Fig. 4, the display device displays the object 312 called "Explanation of Task C" in, for example, the center of the display.

[0015] In this embodiment, pages are also treated as objects. Figures 5 to 7 are diagrams for explaining display examples of associated pages when the associated objects are pages. In Figure 5, an object 321 called "Assignment C" at the top left is associated with a page 322 at the bottom, as will be described later.

[0016] When the user clicks on the object 321 called "Assignment C" as shown in Fig. 6, the display device identifies the object 322 of the page associated with the "Assignment C" object 321. Then, as shown in Fig. 7, the display device displays the corresponding page on the screen.

[0017] In this way, the display device of this embodiment can eliminate the need for the user to remember the relationships between objects or to search for and display objects related to a certain object. Furthermore, the display device can also enable the user to view peripheral information about objects related to a certain object.

[0018] <Terminology> The input means may be any means that allows handwriting by specifying coordinates on a touch panel, such as a pen, a human finger or hand, or a rod-shaped member.

[0019] A stroke is a series of operations in which a user presses an input means against a display, moves it continuously, and then releases it from the display. Stroke data is information displayed on a display based on the trajectory of coordinates input by the input means. Stroke data may be interpolated as appropriate. Handwritten data is data that has one or more strokes. Handwritten input refers to the input of handwritten data by a user.

[0020] An object is an object that is displayed on a display based on stroke data. Object means a target, but in this embodiment it means a display target. The character string converted from handwritten data through character recognition may include not only text data, but also data displayed based on user operations, such as fixed characters such as "Done," stamps displayed as marks, shapes such as circles and stars, and straight lines.

[0021] The confirmed data is data that has been converted into a character code (font) by character recognition of handwritten data and selected by the user, or handwritten data that has been confirmed not to be converted into a character code (font).

[0022] An operation command is a command prepared for operating a handwriting input device to instruct the execution of a specific process. For example, character string editing, modification, input / output, etc. can be executed by an operation command.

[0023] A string is one or more characters used by a computer. The actual entity of a string is a character code. Characters include numbers, alphabets, and symbols. Strings are also called text data.

[0024] Association means that two or more things are related to each other. Association means linking two or more things in a database or other way so that they are related to each other.

[0025] <Device configuration example> The overall configuration of the display device 2 according to this embodiment will be described with reference to Fig. 8. Fig. 8 is a diagram showing the overall configuration of the display device 2. Fig. 8(a) shows, as an example of the display device 2, a display device 2 used as a horizontally long electronic whiteboard hung on a wall.

[0026] 8(a), a display 220 as an example of a display device is installed on the upper part of the display device 2. A user U can handwrite (also referred to as input or drawing) characters and the like on the display 220 using a pen 2500.

[0027] FIG. 8(b) shows a display device 2 used as a vertically long electronic whiteboard hung on a wall.

[0028] 8(c) shows the display device 2 placed flat on a desk 230. The display device 2 is about 1 cm thick, so there is no need to adjust the height of the desk even when it is placed flat on a regular desk. In addition, the user can easily move the display device 2.

[0029] The pen 2500 can input coordinates using electromagnetic induction, active electrostatic coupling, etc. The pen 2500 may also have functions such as pressure detection, tilt detection, and a hover function (displaying the cursor before the pen touches the screen).

[0030] <Device hardware configuration> Next, the hardware configuration of the display device 2 will be described with reference to Fig. 9. As shown in the figure, the display device 2 has the configuration of an information processing device or a computer. Fig. 9 is an example of a hardware configuration diagram of the display device 2. As shown in Fig. 9, the display device 2 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and an SSD (Solid State Drive) 204.

[0031] Of these, the CPU 201 controls the overall operation of the display device 2. The ROM 202 stores the CPU 201 and programs used to drive the CPU 201, such as an IPL (Initial Program Loader). The RAM 203 is used as a work area for the CPU 201.

[0032] The SSD 204 stores various data such as the OS and programs for the display device. These programs may be application programs that run on an information processing device equipped with a general-purpose OS (Windows (registered trademark), Mac OS (registered trademark), Android (registered trademark), iOS (registered trademark), etc.).

[0033] The display device 2 also includes a display controller 213, a touch sensor controller 215, a touch sensor 216, a display 220, a power switch 227, a tilt sensor 217, a serial interface 218, a speaker 219, a microphone 221, a wireless communication device 222, an infrared I / F 223, a power control circuit 224, an AC adapter 225, and a battery 226.

[0034] The display controller 213 controls and manages the screen display in order to output an output image to the display 220, etc. The touch sensor 216 detects that the pen 2500, the user's hand, etc. (the pen and the user's hand are input means) has come into contact with the display 220. The touch sensor 216 also receives a pen ID.

[0035] The touch sensor controller 215 controls the processing of the touch sensor 216. The touch sensor 216 inputs and detects coordinates. For example, in the case of an optical method, two light-emitting / receiving devices installed at both ends of the upper side of the display 220 emit multiple infrared rays parallel to the display 220, and receive the light that is reflected by a reflecting member installed around the periphery of the display 220 and returns along the same optical path as the emitted light by the light-receiving element. The touch sensor 216 outputs position information of the infrared rays emitted by the two light-emitting / receiving devices that are blocked by an object to the touch sensor controller 215, and the touch sensor controller 215 identifies the coordinate position, which is the contact position of the object. The touch sensor controller 215 also has a communication unit 215a and can wirelessly communicate with the pen 2500. For example, if communication is based on a standard such as Bluetooth (registered trademark), a commercially available pen can be used. If one or more pens 2500 are registered in advance in the communication unit 215a, the user can communicate with the display device 2 without having to perform connection settings to allow the pen 2500 to communicate with the display device 2.

[0036] The power switch 227 is a switch for switching ON / OFF the power of the display device 2. The tilt sensor 217 is a sensor for detecting the tilt angle of the display device 2. It is mainly used to detect whether the display device 2 is being used in the installation state shown in FIG. 8(a), FIG. 8(b), or FIG. 8(c), and can automatically change the thickness of characters, etc. depending on the installation state.

[0037] The serial interface 218 is an external communication interface such as a USB. The serial interface 218 is used for inputting information from the outside. The speaker 219 is used for audio output, and the microphone 221 is used for audio input. The wireless communication device 222 communicates with a terminal carried by the user, and relays a connection to the Internet, for example. The wireless communication device 222 communicates using Wi-Fi, Bluetooth (registered trademark), or the like, but any communication standard is acceptable. The wireless communication device 222 forms an access point, and when the user sets the SSID (Service Set Identifier) ​​and password they have obtained in the terminal they carry, they can connect to the access point.

[0038] Preferably, the wireless communication device 222 is provided with two access points. a. Access point → Internet b. Access point → Internal network → Internet Access point a is for users outside the company, and users cannot access the company network but can use the Internet. Access point b is for users inside the company, and users can use the company network and the Internet.

[0039] The infrared I / F 223 detects the display device 2 arranged adjacently. The infrared I / F 223 can detect only the display device 2 arranged adjacently by taking advantage of the rectilinearity of infrared rays. It is preferable to provide one infrared I / F 223 on each side, and it can detect in which direction another display device 2 is arranged relative to the display device 2. This expands the screen, making it possible to display handwritten information that was previously handwritten on the adjacent display device 2 (handwritten information on another page, with the area of ​​one display 220 being one page), etc.

[0040] The power supply control circuit 224 controls an AC adapter 225 and a battery 226, which are power sources for the display device 2. The AC adapter 225 converts AC common to commercial power sources into DC.

[0041] If the display 220 is so-called electronic paper, it consumes little or no power to maintain the image display, and therefore can be driven by a battery 226. This makes it possible to use the display device 2 for purposes such as digital signage even in places where it is difficult to connect to a power source, such as outdoors.

[0042] The display device 2 further includes a bus line 210. The bus line 210 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 201 shown in FIG.

[0043] The touch sensor 216 is not limited to an optical type, and various detection means may be used, such as a capacitive touch panel that identifies the contact position by detecting changes in capacitance, a resistive film touch panel that identifies the contact position by voltage changes between two opposing resistive films, or an electromagnetic induction touch panel that identifies the contact position by detecting electromagnetic induction caused by an object touching the display unit. The touch sensor 216 may be of a type that does not require an electronic pen to detect whether or not the pen tip is touching. In this case, a fingertip or a pen-shaped stick can be used to perform a touch operation. The pen 2500 does not have to be a long, thin pen-shaped panel.

[0044] <About the function> Next, the functions of the display device 2 will be described with reference to FIG. 10. FIG. 10 is an example of a functional block diagram that explains the functions of the display device 2 in blocks. The display device 2 has a contact position detection unit 21, a drawing data generation unit 22, a character recognition unit 23, a display control unit 24, a data recording unit 25, a network communication unit 26, an operation reception unit 27, a link generation unit 28, a link identification unit 29, an area management unit 30, a subview creation unit 31, and an object management unit 32. Each function of the display device 2 is a function or means that is realized when any of the components shown in FIG. 9 operates in response to an instruction from the CPU 201 in accordance with a program loaded from the SSD 204 onto the RAM 203.

[0045] The contact position detection unit 21 detects the coordinates of the position where the pen 2500 contacts the touch sensor 216. The drawing data generation unit 22 acquires the coordinates where the tip of the pen 2500 contacts from the contact position detection unit 21. The drawing data generation unit 22 generates stroke data by connecting this sequence of coordinate points by interpolating them.

[0046] The character recognition unit 23 performs character recognition processing on one or more stroke data (handwritten data) handwritten by the user and converts them into character codes. The character recognition unit 23 recognizes characters (not only Japanese but also multiple languages ​​such as English), numbers, symbols (%, $, &, etc.), shapes (lines, circles, triangles, etc.), etc. in parallel with the user's pen operation. Various algorithms have been devised for recognition methods, but details will be omitted in this embodiment as known techniques can be used.

[0047] The display control unit 24 displays the handwritten data, the character string converted from the handwritten data, and an operation menu for the user to operate on the display.

[0048] The data recording unit 25 stores handwritten data written on the display device 2, converted character strings, PC screens, files, etc. in the object storage unit 41 of the storage unit 40. Handwritten data, character strings (including graphics), images such as PC screens, files, etc. are handled as objects. For handwritten data, a group of stroke data becomes one object according to a time division due to an interruption in handwriting or a distance division due to a difference in handwriting location.

[0049] The network communication unit 26 is connected to a network such as a LAN, and transmits and receives data to and from other devices via the network.

[0050] The object management unit 32 determines which object the user has selected based mainly on the coordinates detected by the contact position detection unit 21. The object management unit 32 holds the selected object in a selected state.

[0051] The link generating unit 28 sets a link between two objects specified by the user (generates link information). A link means associating two or more objects.

[0052] The link identification unit 29 identifies link information set for an object specified by the user. The area management unit 30 manages the coordinates of the area set as the link destination when the user selects an area using stroke data instead of an object as the link destination.

[0053] When displaying a link destination, the subview creation unit 31 does not completely switch the display, but instead creates a thumbnail containing information about the area surrounding the link destination as a subview. A thumbnail is an image that has been reduced in size so that the entire image is displayed.

[0054] The display device 2 also has a storage unit 40 constructed in the SSD 204 or RAM 203 shown in FIG. 9, and the storage unit 40 has an object storage unit 41 and a link storage unit 42 constructed therein.

[0055] [Table 1] Table 1 shows a schematic diagram of the object information stored in the object storage unit 41. The object information is information for managing the objects displayed by the display device 2.

[0056] · Object ID is identification information that identifies an object.

[0057] Type is the type of object, and can be handwritten, text, shape, image, etc. Handwritten text is stroke data (a sequence of coordinate points). Text is a string of characters (character code) converted from handwritten data. Text is sometimes called text data. Shapes are geometric shapes such as triangles and squares converted from handwritten data. Images are image data such as JPEG, PNG, and TIFF imported from a PC or the Internet.

[0058] One screen of the display device 2 is called a page. The item of the page is the page number.

[0059] The coordinates indicate the position of the object relative to a predetermined origin of the display device 2. The object position is, for example, the upper left vertex of the bounding rectangle of the object. The coordinates are expressed, for example, in pixel units of the display.

[0060] The size is the width and height of the object's bounding rectangle.

[0061] [Table 2] Table 2 shows a schematic diagram of the link information held in the link storage unit 42. The link information is information that associates two objects.

[0062] The link ID is the identification information that identifies the link.

[0063] The link source object ID is an object ID that identifies the link source object. The link source object is the object that is selected first by the user out of two related objects (an example of the first object).

[0064] The link destination object ID is an object ID that identifies the link destination object. The link destination object is the second object selected by the user out of two related objects (an example of the second object).

[0065] The link destination area coordinates are the coordinates of the area that specifies the link destination area when the link destination is an area rather than an object.

[0066] The link destination area size is the size of the link destination area if the link destination is an area rather than an object.

[0067] If the link destination is a page, the linked page number is the page number. However, pages are also treated as objects.

[0068] <Detailed example of association> The association of objects on the screen will be described with reference to FIGS. 11 to 20. First, FIG. 11 shows the objects of one entire page for the following explanation. Display ranges 331, 332, and 333 are the display ranges displayed by the display device 2. In other words, the display device 2 can manage pages that are larger than the display ranges. There is no limit to the size of one page, or even if there is a limit, it is large enough that it does not cause any problems in normal use. The sizes of the display ranges 331, 332, and 333 change according to the user's zoom-in and zoom-out operations.

[0069] Fig. 12 shows an example screen (part 1) displayed by the display device 2. The display device 2 can always display the tool tray 334 regardless of the display range. The user can switch between displaying and hiding the tool tray 334. Also, while the tool tray 334 is displayed at the bottom of the display in Fig. 12, the user can change its display position.

[0070] 13 shows an example screen (part 2) displayed by the display device 2. When the user wants to associate two objects with each other, the user selects the "link destination setting tool" 335 from the tool tray 334. When the operation receiving unit 27 receives the operation, the "link destination setting tool" 335 enters a selected state. In the selected state, for example, the "link destination setting tool" 335 is displayed in inverted or its brightness is increased.

[0071] FIG. 14 shows a third example of a screen displayed by the display device 2. After selecting the "link destination setting tool" 335, the user presses the link source object 336 or encircles it with stroke data. At this point, the link source object 336 is not technically a link source object because it is not associated with any other object. However, once it is associated with any other object, it becomes a link source object. "Pressing" refers to the user touching it with the pen 2500 or a fingertip, or clicking it with a pointing device such as a mouse. In FIG. 14, the user is encircling the link source object 336 with the pen 2500.

[0072] The contact position detection unit 21 detects the coordinates pressed by the user. The object management unit 32 identifies an object from the object storage unit 41 based on these coordinates. The object management unit 32 identifies, for example, the object whose center of gravity is closest within a certain distance from the pressed coordinates. If the pressed coordinates have an expanse as shown in FIG. 14, the object management unit 32 may identify the center of gravity of the expanse.

[0073] Fig. 15 shows an example screen (part 4) displayed by the display device 2. The display control unit 24 displays a menu 337 that the user can operate for the link source object 336 selected by the object management unit 32. In Fig. 15, the menu 337 has a color change option 338 and a link setting option 339. A user who wants to set a link between objects presses the link setting option 339.

[0074] The operation receiving unit 27 receives the notification that link setting 339 has been pressed, and the object management unit 32 transmits the object information of the selected link source object 336 to the link generation unit 28. The link generation unit 28 transitions to a link destination selection waiting state and waits for information on the link destination object to be sent.

[0075] 16 shows an example screen (part 5) displayed by the display device 2. The user changes the display range to select a link destination object. The user clicks another link destination object 340. Any object becomes a link destination object by being associated with the link source object.

[0076] The contact position detection unit 21 detects the coordinates pressed by the user. The object management unit 32 identifies a link destination object 340 from the object storage unit 41 based on these coordinates. The object management unit 32 identifies, for example, the link destination object 340 whose center of gravity is closest within a certain distance from the coordinates pressed by the user. The object management unit 32 transmits object information of the identified link destination object to the link generation unit 28.

[0077] After obtaining the object information for the link source object and the link destination object, the link generation unit 28 associates these two object IDs and adds them to the link storage unit 42 in Table 2. If the user selects an area with no objects, the link generation unit 28 adds the coordinates and size of this area to the link storage unit 42. If the user selects a page (saved pages are displayed as thumbnails), the link generation unit 28 adds the link source object and the linked page to the link storage unit 42.

[0078] <Searching the object memory, adding to the link memory> Referring to Figures 17 to 19, the association of two objects will be described with a focus on the object storage unit 41 and the link storage unit 42. Figure 17 is a diagram for explaining the identification of a link source object in the object storage unit 41. As shown in Figure 14, when the user selects a link source object, the object management unit 32 searches the object storage unit 41. This search refers to identifying an object with the closest center of gravity within a certain distance from the coordinates pressed by the user. The link source object may be limited to objects within a certain distance from the coordinates pressed by the user, or there may be no such restriction. The object management unit 32 identifies the object with object ID = 1.

[0079] Figure 18 is a diagram for explaining the identification of a link destination object. As shown in Figure 16, when the user selects a link destination object, the object management unit 32 searches the object storage unit 41 for an object whose center of gravity is closest to the coordinates pressed by the user within a certain distance. The object management unit 32 identifies the object with object ID = 2.

[0080] FIG. 19 is a diagram illustrating the addition of a record to the link storage unit 42. A record refers to one piece of data in a database. In this embodiment, a record is referred to as link information. The link generation unit 28 receives object ID=1 as the link source object ID and object ID=2 as the link destination object ID. The link generation unit 28 sets 1 as the link source object ID and 2 as the link destination object ID in the link storage unit 42.

[0081] By setting link information in the link storage unit 42, when the user selects a link source object, the display device 2 can display a link destination object.

[0082] <Procedure for associating objects> FIG. 20 is an example of a sequence diagram illustrating a procedure in which the display device 2 sets the association between two objects.

[0083] S1: After the user clicks on the "Link Destination Setting Tool" 335 in FIG. 13 or the link setting 339 in FIG. 15, the user clicks on the link source object.

[0084] S2: The contact position detection unit 21 detects the coordinates pressed by the user.

[0085] S3: The contact position detection unit 21 designates the coordinates pressed by the user and requests the object management unit 32 to select an object.

[0086] S4: The object management unit 32 searches the object storage unit 41 for the coordinates pressed by the user and identifies the link source object. The object management unit 32 notifies the link generation unit 28 of the object ID of this object as the link source object.

[0087] S5: Next, the user presses the linked object.

[0088] S6: The contact position detection unit 21 detects the coordinates pressed by the user.

[0089] S7: The contact position detection unit 21 designates the coordinates pressed by the user and requests the object management unit 32 to select an object.

[0090] S8: The object management unit 32 searches the object storage unit 41 for the coordinates pressed by the user and identifies the link destination object. The object management unit 32 notifies the link generation unit 28 of the object ID of this object as the link destination object.

[0091] S9: The link generating unit 28 sets the link source object ID and the link destination object ID in the link storage unit 42.

[0092] In this way, the display device 2 of this embodiment allows the user to associate two objects and set them together.

[0093] <Displaying objects using link information> Next, the display of objects using link information will be described with reference to FIGS.

[0094] 21 is a diagram illustrating the selection of a link source object. The user presses a link source object 341 for which a generated link has been set. The object management unit 32 identifies the link source object 341 based on the coordinates pressed by the user. The object management unit 32 sends the identified object ID to the link identification unit 29. The link identification unit 29 searches for an object ID from the link source object IDs in the link storage unit 42. If an object ID that is the same as the link source object ID is found, the link identification unit 29 sends the link destination object ID to the object management unit 32 and acquires object information for the object identified by the link destination object ID.

[0095] The link specification unit 29 acquires object information of the link destination object from the object management unit 32, and requests the display control unit 24 to display this object and its surroundings. The display control unit 24 switches the display range to the specified link destination object and its surroundings, and displays the link destination object.

[0096] Fig. 22 shows an example of a display range including a link destination object. In Fig. 22, a link destination object 342 associated with a link source object 341 pressed by the user and its surroundings are displayed.

[0097] In this way, the user can display the linked object by pressing the link source object.

[0098] <Searching the object memory, searching the link memory> 23 to 25, the display of link destination objects will be described with a focus on the object storage unit 41 and the link storage unit 42. FIG. 23 is a diagram for explaining a search for a link source object in the object storage unit 41. As shown in FIG. 21, when the user presses the link source object 341, the object management unit 32 searches the object storage unit 41 for the coordinates pressed by the user. The search method may be the same as that for associating two links. The object management unit 32 identifies the object with object ID=1.

[0099] 24 is a diagram illustrating a search for a link source object in the link storage unit 42. The link identification unit 29 searches for link information in which object ID=1 is set as the link source object ID. The link identification unit 29 finds link information with link ID=5 and identifies the link destination object with link destination object ID=2.

[0100] FIG. 25 is a diagram illustrating a search for a link destination object in the object storage unit 41. The object management unit 32 searches the object storage unit 41 for an object with object ID=2 (link destination object ID=2). This allows the object management unit 32 to identify the page, coordinates, and size of the link destination object. Therefore, the display control unit 24 can display the link destination object in a display range that includes the link destination object and its surroundings. The display range is, for example, a range centered on the center of gravity of the link destination object. The size of the display range is determined according to the current display magnification set by the user. Changing the display magnification will be described later.

[0101] <Operation procedure for displaying objects using link information> FIG. 26 is an example of a sequence diagram illustrating a procedure in which the display device 2 displays a link destination object based on link information when the user presses a link source object.

[0102] S11: The user presses the link source object. Note that some menu may be selected before selecting the link source object.

[0103] S12: The contact position detection unit 21 detects the coordinates pressed by the user, and requests the object management unit 32 to select an object by specifying the coordinates pressed by the user. The object management unit 32 identifies the link source object ID from the object storage unit 41 based on the coordinates pressed by the user.

[0104] S13: The object management unit 32 specifies the link source object ID and requests the link identification unit 29 to search for a link destination object. In response to the request, the link identification unit 29 searches the link source object ID field in the link storage unit 42 with the link source object ID sent from the object management unit 32, and identifies the link destination object ID.

[0105] S14: The link specifying unit 29 specifies the link destination object ID and acquires the object information of the link destination object from the object management unit 32.

[0106] S15: The link identification unit 29 generates a display range (view) based on the object information of the link destination object. The display range is, for example, a display range having a predetermined number of pixels above, below, left, and right, centered on the center of gravity of the link destination object.

[0107] S16: By specifying this display range, the link specifying unit 29 requests the display control unit 24 to switch the display range to display the link destination object.

[0108] In this way, the display device 2 of this embodiment allows the user to set an association between objects. When two objects are associated with each other, the user can display the link destination object by pressing the link source object.

[0109] In this embodiment, the link destination object is displayed when the link source object is selected, but the link source object may be displayed when the link destination object is selected. [Example]

[0110] <When an area, not an object, is selected as the link destination> In this embodiment, a case where an area is selected as a link destination object will be described. In this embodiment, the hardware configuration diagram of Fig. 9 and the functional block diagram of Fig. 10 described in the above embodiment can be used.

[0111] 27 and 28, a case where the user selects a link destination by enclosing an area with stroke data will be described. In this case, the link generating unit 28 sets an area as the link destination, rather than a text box or the like.

[0112] 27 shows an example of stroke data handwritten by the user to select an area. When the user selects an area, the area management unit 30 detects the coordinates of the upper left vertex of the enclosed circumscribing rectangle or inscribing rectangle as the coordinates of the link destination based on the stroke data 351 detected by the contact position detection unit 21. The area management unit 30 transmits the coordinates of the upper left vertex and the height and width of the circumscribing rectangle or inscribing rectangle to the link generation unit 28.

[0113] 28 is a diagram illustrating an area set as a link destination in the link storage unit 42. The link generation unit 28 sets the link source object ID, link destination area coordinates, and link destination area size in the link storage unit 42. Since there is no link destination object, the link destination object ID field is blank.

[0114] In this way, the user can set any area as the link destination object. Note that although a text box is enclosed by stroke data 351 in Figure 27, the area specified by stroke data 351 does not have to contain any text box or stroke data.

[0115] Furthermore, in this embodiment, the link destination object is specified by an area, but the link source object may also be specified by an area.

[0116] <Procedure for associating objects> 29 is an example of a sequence diagram illustrating a procedure for setting the association between an object and an area by the display device 2. The explanation of FIG. 29 will mainly focus on the differences from FIG.

[0117] S21 to S24: These steps may be the same as steps S1 to S4 in FIG.

[0118] S25: The user surrounds the linked area with stroke data.

[0119] S26: The contact position detection unit 21 detects the coordinates of the stroke data handwritten by the user.

[0120] S27: The contact position detection unit 21 requests the area management unit 30 to determine an area by specifying the coordinates of the stroke data, since the stroke data is not one point but has a spread.

[0121] S28: The region management unit 30 determines whether the circumscribed rectangle of the coordinates of the stroke data from the start to the end of writing is equal to or greater than a threshold. If it is equal to or greater than the threshold, the region management unit 30 determines the coordinates of the upper left vertex, and the height and width of the circumscribed rectangle or inscribed rectangle of the stroke data.

[0122] S29: The region management unit 30 transmits the coordinates of the upper left vertex of the circumscribing rectangle or the inscribing rectangle, as well as the height and width, to the link generation unit 28 as region information.

[0123] S30: The link generating unit 28 associates the link source object ID, the coordinates of the area, and the height and width sizes and sets them in the link storage unit 42.

[0124] In this way, the display device 2 of this embodiment allows the user to associate an object with any area and set it accordingly. [Example]

[0125] <Scaling of linked object or linked area> In this embodiment, a magnified display of a linked object or a linked area will be described. In this embodiment, the hardware configuration diagram of Fig. 9 and the functional block diagram of Fig. 10 described in the above embodiment can be used.

[0126] The link identification unit 29 determines the display range of the link source object or the link destination area to be the same as when the link source object was selected by the user. However, if the display range of the link source object or the link destination area is large, the display device 2 may not be able to display the entire link destination object or the link destination area on the display.

[0127] Therefore, the link specifying unit 29 calculates an appropriate display magnification, switches the display magnification, and displays the link destination object or the link destination area.

[0128] 30 is an example of a flowchart showing a procedure for determining the display magnification ratio by the display control unit 24. The link identification unit 29 determines whether the entire link destination object can be displayed at the current display magnification ratio (S101). The details of this determination will be described later.

[0129] If the determination in step S101 is No, the link identification unit 29 calculates an appropriate display magnification that allows the entire linked object to be displayed (S102).

[0130] 31 is a diagram illustrating the selection of a link source object when the display range is expanded. When displaying a link destination object, link identification unit 29 obtains the size of the link destination object from object management unit 32. It then determines whether the entire link destination object can be displayed on the display if the display range is changed while keeping the current display magnification.

[0131] FIG. 32 shows a link destination object displayed at the current display magnification. Since the link destination object does not fit on the display, FIG. 32 is not actually displayed. The method for making this determination will be explained below. The link identification unit 29 holds the standard width and standard height that can be displayed on the display at the standard magnification. Furthermore, when the display magnification is n times, the width and height of the object or area are each multiplied by n. Therefore, if the current display magnification is n times, the link identification unit 29 compares the standard width and standard height with the values ​​obtained by multiplying the width and height of the link destination object or link destination area by n.

[0132] In the case of FIG. 32, the link identification unit 29 can determine that the link destination object does not fit on the display in the width direction.

[0133] 33 is a diagram illustrating a change in display magnification. The object management unit 32 calculates the ratio of the standard width to the n-fold increased width of the link destination object (width ÷ standard width). Then, the object management unit 32 multiplies n, the current display magnification, by the reciprocal of this ratio.

[0134] In this way, the link identification unit 29 can determine a display magnification at which the entire link destination object can be displayed in the display range. Although a link destination object such as a text box is used as an example in Figures 31 to 33, the same applies when the link destination object is an area.

[0135] Furthermore, if the link destination object is too small for the display range at the current display magnification, the link identifying unit 29 reduces the display magnification (reducing means, for example, reducing it from 10x to about 2x). If the display range is switched while keeping the current display magnification, the link identifying unit 29 determines whether the size of the link destination object (the smaller of width or height) is equal to or smaller than the threshold. In this case, the link identifying unit 29 determines the display magnification so that the size of the link destination object is approximately the threshold. For example, the link identifying unit 29 multiplies the current display magnification by the reciprocal of "size of link destination object divided by threshold". In this way, the link identifying unit 29 can determine a display magnification at which the link destination object is displayed in the display range at a size that is not too small. [Example]

[0136] <Display position of linked object or linked area> In this embodiment, the display position of the linked object or linked area will be described. In this embodiment, the hardware configuration diagram of Fig. 9 and the functional block diagram of Fig. 10 described in the above embodiment can be used.

[0137] 22, the link identification unit 29 displays the link destination object in the center of the display. However, the link identification unit 29 may control the position on the display at which the link destination object or link destination area is displayed.

[0138] 34(a) shows an example of a link destination object 361 displayed in the center of the display. When the link destination object 361 is displayed in the center of the display, the user can also grasp the objects around the link destination object 361.

[0139] 34(b) shows an example of a link destination object 361 displayed in the upper left corner of the display. When the link destination object 361 is displayed in the upper left corner of the display, it is easy for the user to continue writing by hand on the link destination object 361.

[0140] Therefore, when the link destination object 3 is a character string, the link identification unit 29 controls the position on the display at which the link destination object or link destination area is displayed depending on the writing direction of the language of the character string.

[0141] FIG. 35 is an example of a flowchart illustrating how the link specifying unit 29 controls the display position of a link destination object or a link destination area.

[0142] The link identification unit 29 determines whether the link destination object 3 is a character string based on the type item of the object information (S111).

[0143] If it is a character string, the link identification unit 29 determines whether the language of the character string is right-to-left or left-to-right (S112). Since the language can be determined from the character code of the character string, the link identification unit 29 first determines the language. The link identification unit 29 then references a pre-prepared list of a small number of right-to-left languages ​​to determine whether the language is right-to-left or left-to-right.

[0144] The link identification unit 29 determines the upper right as the display position in the case of right-to-left writing (S113), and determines the upper left as the display position in the case of left-to-right writing (S114). The link identification unit 29 instructs the display control unit 24 to switch the display range.

[0145] This makes it easier for users to use the linked object as a starting point to handwrite another object. [Example]

[0146] <Link destination setting by object list> In this embodiment, the setting of a link destination using an object list will be described. In this embodiment, the hardware configuration diagram of Fig. 9 and the functional block diagram of Fig. 10 described in the above embodiment can be used.

[0147] In addition to selecting a link destination object with the pen 2500, the user can also select a link destination from a list of objects.

[0148] 36 shows an example of an object list displayed by the display control unit 24. For example, when the user presses a link source object, the link generating unit 28 causes the display control unit 24 to display the object list.

[0149] The object list is a list of each record in the object storage unit 41 in Table 1. The object list may include thumbnails of the objects. The user selects any object from the object list as a link destination. The selected link destination object is received by the link generation unit 28, which then sets it in the link storage unit 42 in Table 2 in association with the link source object.

[0150] FIG. 37 is an example of a sequence diagram illustrating a procedure in which the display device 2 sets the association between two objects.

[0151] S31 to S34: These steps may be the same as steps S1 to S4 in FIG.

[0152] S35: The link generating unit 28 requests the display control unit 24 to display a list of objects.

[0153] S36: The user selects an object from the object list, and the contact position detection unit 21 receives the coordinates.

[0154] S37: The contact position detection unit 21 designates the coordinates pressed by the user and requests the object management unit 32 to select an object.

[0155] S38: The object management unit 32 determines which object in the object list is closest to the coordinates pressed by the user, and identifies the link destination object. The object management unit 32 notifies the link generation unit 28 of the object ID of this object as the link destination object.

[0156] S39: The link generating unit 28 sets the link source object ID and the link destination object ID in the link storage unit 42.

[0157] In this way, the display device 2 of this embodiment allows the user to select a link destination object from the object list, thereby reducing the effort required for the user to change the display range to search for the link destination object. [Example]

[0158] <Subview display> In this embodiment, the display of a link destination by displaying a subview will be described. In this embodiment, the hardware configuration diagram of Fig. 9 and the functional block diagram of Fig. 10 described in the above embodiment can be used.

[0159] By displaying the linked object or the like in a subview rather than switching the display range to the linked object or the like, the display device 2 allows the user to check the linked object or the like before the display range is switched. Note that a subview refers to displaying a display range separate from the current display range. For example, it may be called a thumbnail display or a pop-up display.

[0160] Fig. 38 is a diagram showing an example of a selected link source object 371. Fig. 39 is an example of a subview 370 including a link destination object 372 associated with the link source object 371 selected in Fig. 38. The subview 370 is displayed near the link source object 371. Therefore, the user can check the link destination object 372 and the like without switching the display range.

[0161] 40 and 41 are diagrams illustrating subviews when the link destination object is a page. In Fig. 40, the user has pressed link source object 373. Pages 374 and 375, which are displayed as thumbnails, are associated with link source object 373.

[0162] 41, two pages 374 and 375, which are link destination objects associated with a link source object 373, are displayed in subviews 376 and 377. In this way, the display device 2 can display all link destination objects associated with one link source object 373 in the subviews 376 and 377.

[0163] 42 is an example of a sequence diagram showing a procedure for displaying a subview. In the explanation of FIG. 42, the differences from FIG. 26 will be mainly explained.

[0164] S61 to S64: These steps may be the same as steps S21 to S24 in FIG.

[0165] S65: The link specifying unit 29 transmits the object information of each link destination object to the subview creating unit 31.

[0166] S66: The subview creating unit 31 creates a thumbnail including the linked object and its surroundings based on the received object information.

[0167] S67: The subview creation unit 31 transmits the subview to the display control unit 24. The display control unit 24 displays the subview including the link destination object and its surroundings near the link source object.

[0168] In this manner, in this embodiment, the display device 2 displays the link destination object in the subview, allowing the user to check the link destination before the display range is switched. [Example]

[0169] <Link destination setting by link destination reservation> In this embodiment, the setting of a link destination by reservation of the link destination will be described. In this embodiment, the hardware configuration diagram of Fig. 9 and the functional block diagram of Fig. 10 described in the above embodiment will be used.

[0170] While writing by hand, a user of the display device 2 may want to use this object as a link destination later. In this case, the user sets an object that can be a link destination in the object information, and when the user associates two objects, the link generation unit 28 accepts this link destination from the object information.

[0171] 43 is a diagram for explaining the reservation of a link destination object. First, the user selects an icon from the tool tray to switch the operation mode of the display device 2 to the link destination list setting mode. In the link destination list setting mode, the user presses any object 381.

[0172] The object manager 32 registers in the object information that the object 381 has been reserved. Therefore, the object storage 41 has items different from those in Table 1.

[0173] [Table 3] Table 3 shows another example of object information stored in the object storage unit 41. The object storage unit 41 in Table 3 has a "Reserved" field. "Reserved" means that this object has been reserved as a link destination. The "Reserved" field can be used to store whether or not each object has been set as a link destination.

[0174] 44 is a diagram illustrating the association of two objects by reserving a link destination object. When the user presses a link source object 382, ​​the display control unit 24 displays a reserved object list 383 on the screen. The reserved object list 383 is similar to the object list, but only objects for which the "Reserved" item is set to True are displayed, making it easier for the user to select a reservation destination object.

[0175] When the user selects one of the objects, the link generating unit 28 associates the link source object 382 with the link destination object selected by the user and sets the association in the link storage unit 42.

[0176] FIG. 45 is an example of a sequence diagram illustrating the association of two objects using reserved.

[0177] First, the flow of reserving an object will be described.

[0178] S41: In the link destination list setting mode, the user presses an object that the user wants to set as reserved.

[0179] S42: The contact position detection unit 21 detects the coordinates pressed by the user.

[0180] S43: The contact position detection unit 21 designates the coordinates pressed by the user and requests the object management unit 32 to select an object.

[0181] S44: The object management unit 32 searches the object storage unit 41 for the coordinates pressed by the user and identifies the object.

[0182] S45: The object management unit 32 sets the reserved item of this object to True.

[0183] Next, the association flow will be described.

[0184] S46 to S49: These steps may be the same as steps S1 to S4 in FIG.

[0185] S50: The link generating unit 28 requests the display control unit 24 to display a list of reserved objects. The display control unit 24 acquires objects whose reserved item is set to True from the object information, and displays them as a list of reserved objects.

[0186] S51: The user selects an object from the list of reserved objects, and the contact position detection unit 21 receives the coordinates.

[0187] S52: The contact position detection unit 21 designates the coordinates pressed by the user and requests the object management unit 32 to select an object.

[0188] S53: The object management unit 32 determines which object in the list of reserved objects is closest to the coordinates pressed by the user, and identifies the link destination object. The object management unit 32 notifies the link generation unit 28 of the object ID of this object as the link destination object.

[0189] S54: The link generating unit 28 sets the link source object ID and the link destination object ID in the link storage unit 42.

[0190] In this way, the display device 2 displays only reserved objects, making it easier for the user to select a link destination object when there are a large number of objects on a page. [Example]

[0191] <Selective display of link destination> In this embodiment, movement to a link destination by selecting and displaying the link destination will be described. In this embodiment, the hardware configuration diagram of Fig. 9 and the functional block diagram shown in Fig. 107 described in the above embodiment will be described as being applicable.

[0192] In some cases, multiple destination objects are associated with one source object. In this case, the method for the user to select an object to link to is explained below.

[0193] 46 is a diagram illustrating the pressing of a link source object 391 to which multiple link destination objects are set. The link source object 391 pressed by the user has multiple link destination objects set in the link storage unit 42. When a link source object 391 to which multiple link destination objects are set is pressed, the link identification unit 29 returns multiple search results. The display control unit 24 presents the link destination objects in the search results to the user as a list.

[0194] Figure 47 is a diagram showing thumbnails of link destination objects 392 and 393 presented to the user. In Figure 47, thumbnails of the two link destination objects 392 and 393 are displayed near the link source object. When the user presses link destination object 392 or 393, link identification unit 29 determines the display range so that the link destination object is included.

[0195] Fig. 48 shows the display range displayed by the display device 2 when the link destination object in Fig. 47 is selected. In Fig. 48, the thumbnail of the link destination object 392 selected in Fig. 47 is the display range.

[0196] FIG. 49 is an example of a sequence diagram illustrating a procedure in which the display device 2 displays a link destination object when a user selects any of the link destination objects displayed as thumbnails.

[0197] S71 to S75: These steps may be the same as steps S11 to S15 in FIG.

[0198] S76: The link specifying unit 29 requests the display control unit 24 to display thumbnails of the created display range.

[0199] S77: The user presses the thumbnail to be designated as the display range.

[0200] S78: The touch position detection unit 21 notifies the display control unit 24 of the coordinates pressed by the user.

[0201] S79: The display control unit 24 switches the display range to the thumbnail specified by the coordinates pressed by the user.

[0202] In this way, when multiple link destination objects are associated with one link source object, the user can select the link destination object and then switch the display range. [Example]

[0203] <Preview function, Next function> In this embodiment, the movement of the display range using the preview function or the next function will be described. In this embodiment, the hardware configuration diagram of Fig. 9 and the functional block diagram of Fig. 10 described in the above embodiment can be used.

[0204] Users may want to temporarily display the linked object and then immediately return to the original display range. The preview function is known to meet such needs. The next view function is also known when users want to display the linked object again.

[0205] FIG. 50 is an example of a time-series object list held by the display control unit 24. The time-series object list is a list in which a starting display range and one or more link destination objects are associated in display order. The starting display range is the display range that was displayed immediately before the user displayed the link destination object. The link destination object ID following the starting display range indicates, in chronological order, the link destination objects that the user selected and displayed by the display device 2.

[0206] 51 is a diagram illustrating the preview function. The user presses the link source object 401. The display control unit 24 saves the start point display range and the link destination object ID.

[0207] 52 shows an example of a link destination object 402 displayed by the display device 2 in response to a press of a link source object 401. A preview button 403 and a next button 404 are displayed in the tool tray. The user presses the preview button 403. The display control unit 24 identifies the previous link destination object ID or the starting display range based on the currently displayed link destination object ID, and determines it as the display range.

[0208] 53 shows the display range displayed when the preview button 403 is pressed. Therefore, the user can switch the linked object to be displayed.

[0209] Fig. 54 is an example of a flowchart showing the procedure for displaying a link destination object in response to the preview function or the next function by the display device 2. The process in Fig. 50 starts with the display device 2 displaying the link destination object.

[0210] The display control unit 24 displays the link destination object selected by the user (S201). After displaying the link destination object, the display control unit 24 creates a time-series object list (S202).

[0211] The display control unit 24 determines whether the preview button or the next button has been pressed (S203).

[0212] When the preview button or the next button is pressed, the display control unit 24 refers to the time-series object list and displays the link destination object (S204).

[0213] In this way, the user can temporarily display the linked object and quickly return to the original display area. [Example]

[0214] In the following embodiment, another configuration example of the display device 2 will be described.

[0215] <<Another configuration example 1 of the display device>> Although the display device 2 of this embodiment has been described as having a large touch panel, the display device 2 is not limited to having a touch panel.

[0216] Fig. 55 is a diagram showing another example of the configuration of the display device 2. In Fig. 55, a projector 411 is installed on the upper side of a normal whiteboard 413. This projector 411 corresponds to the display device 2. The normal whiteboard 413 is not a flat panel display integrated with a touch panel, but a whiteboard on which the user directly writes by hand with a marker. The whiteboard may also be a blackboard, as long as it is flat and large enough to project an image.

[0217] The projector 411 has an ultra-short focus optical system and can project an image with little distortion from a distance of about 10 cm onto the whiteboard 413. This image may be transmitted from the PC 400-1 connected wirelessly or via a wire, or may be stored in the projector 411.

[0218] The user writes by hand on the whiteboard 413 using a dedicated electronic pen 2501. The electronic pen 2501 has a light-emitting part, for example at its tip, that is switched on and emits light when the user presses it against the whiteboard 413 to write. The light has a wavelength of near-infrared or infrared, so it is invisible to the user's eyes. The projector 411 has a camera that captures an image of the light-emitting part and analyzes the image to identify the direction of the electronic pen 2501. The electronic pen 2501 also emits sound waves along with the light emission, and the projector 411 calculates the distance based on the time it takes for the sound waves to reach the screen. The projector 411 can identify the position of the electronic pen 2501 based on the direction and distance. The handwritten data is drawn (projected) at the position of the electronic pen 2501.

[0219] The projector 411 projects the menu 430, so when the user presses a button with the electronic pen 2501, the projector 411 identifies the pressed button based on the position of the electronic pen 2501 and the ON signal of the switch. For example, when the save button 431 is pressed, the projector 411 saves the handwritten data (sequence of coordinate points) handwritten by the user. The projector 411 saves the handwritten information in a predetermined server 412, USB memory 2600, or the like. The handwritten information is saved for each page. Since the coordinates are saved as they are, rather than as image data, the user can re-edit the information. However, in this embodiment, the menu 430 does not need to be displayed because operation commands can be called up by handwriting. [Example]

[0220] <<Another configuration example 2 of the display device>> 56 is a diagram showing another example of the configuration of the display device 2. In the example of Fig. 56, the display device 2 includes a terminal device 600, an image projection device 700A, and a pen motion detection device 810.

[0221] The terminal device 600 is connected by wire to the image projection device 700A and the pen motion detection device 810. The image projection device 700A projects image data input by the terminal device 600 onto the screen 800.

[0222] The pen operation detection device 810 communicates with the electronic pen 820 and detects the operation of the electronic pen 820 in the vicinity of the screen 800. Specifically, the electronic pen 820 detects coordinate information indicating a point pointed at by the electronic pen 820 on the screen 800 (the detection method may be the same as in FIG. 55 ), and transmits the coordinate information to the terminal device 600.

[0223] The terminal device 600 generates image data of handwritten data input by the electronic pen 820 based on the coordinate information received from the pen operation detection device 810, and causes the image projection device 700A to draw an image of the handwritten data on the screen 800.

[0224] Furthermore, the terminal device 600 generates superimposed image data that indicates a superimposed image obtained by combining the background image projected by the image projection device 700A and the image of the handwritten data input by the electronic pen 820. [Example]

[0225] <<Another Configuration Example 3 of the Display Device>> 57 is a diagram showing an example of the configuration of the display device 2. In the example of Fig. 57, the display device 2 includes a terminal device 600, a display 800A, and a pen motion detection device 810.

[0226] The pen operation detection device 810 is placed near the display 800A, detects coordinate information indicating a point indicated by the electronic pen 820A on the display 800A (the detection method may be the same as in FIG. 55), and transmits the coordinate information to the terminal device 600. In the example of FIG. 57, the electronic pen 820A may be charged by the terminal device 600 via a USB connector.

[0227] Based on the coordinate information received from the pen operation detection device 810, the terminal device 600 generates image data of the image of the handwritten data input by the electronic pen 820A, and displays the image data on the display 800A. [Example]

[0228] <<Another Configuration Example 4 of the Display Device>> Fig. 58 is a diagram showing a configuration example of the display device 2. In the example of Fig. 58, the display device 2 includes a terminal device 600 and an image projection device 700A.

[0229] The terminal device 600 performs wireless communication (Bluetooth (registered trademark) or the like) with the electronic pen 820B to receive coordinate information of a point indicated by the electronic pen 820B on the screen 800. The coordinate information may be obtained by the electronic pen 820B reading minute position information formed on the screen 800, or the coordinate information may be received from the screen 800.

[0230] Then, based on the received coordinate information, the terminal device 600 generates image data of the image of the handwritten data input by the electronic pen 820B, and causes the image projection device 700A to project the image of the handwritten data.

[0231] Furthermore, the terminal device 600 generates superimposed image data that indicates a superimposed image obtained by combining the background image projected by the image projection device 700A and the image of the handwritten data input by the electronic pen 820.

[0232] As described above, the above-described embodiments can be applied to various system configurations.

[0233] <Other application examples> The best mode for carrying out the present invention has been described above using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0234] The character string is stored as character codes, and the handwritten data is stored as coordinate point data on the display device 2. The data can also be stored on various storage media or on a storage device on a network, and then downloaded from the display device 2 for reuse later. The display device 2 to be reused can be any display device, or even a general information processing device. Therefore, the user can reproduce the handwritten content on a different display device 2 and continue a meeting, etc.

[0235] For example, although an electronic whiteboard has been described as an example in the present embodiment, the electronic whiteboard may also be called an electronic whiteboard, an electronic information board, or the like. Furthermore, the present embodiment can be suitably applied to any information processing device having a touch panel. Examples of information processing devices equipped with a touch panel include output devices such as PJs (Projectors), digital signage, HUDs (Head Up Display) devices, industrial machinery, imaging devices, sound collection devices, medical equipment, network appliances, notebook PCs (Personal Computers), mobile phones, smartphones, tablet devices, game consoles, PDAs (Personal Digital Assistants), digital cameras, wearable PCs, and desktop PCs.

[0236] In this embodiment, the display device 2 detects the coordinates of the pen tip using a touch panel, but the coordinates of the pen tip may also be detected using ultrasonic waves. The pen emits light and also emits ultrasonic waves, and the display device 2 calculates the distance based on the time it takes for the ultrasonic waves to reach the screen. The position of the pen can be identified based on the direction and distance. The projector draws (projects) the trajectory of the pen as stroke data.

[0237] In addition, the configuration examples in Figure 10 and the like are divided according to main functions to make it easier to understand the processing by the display device 2. The method of dividing the processing units and their names do not limit the present invention. The processing by the display device 2 can be divided into even more processing units depending on the processing content. Also, it can be divided so that one processing unit includes even more processes.

[0238] Furthermore, part of the processing performed by the display device 2 may be performed by a server connected to the display device 2 via a network.

[0239] Furthermore, each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions.

[0240] Furthermore, in this embodiment, even if a threshold value is exemplified as a comparison target, the threshold value is not limited to the exemplified value. Therefore, in this embodiment, with respect to all threshold values, the terms "less than threshold" and "below threshold" have the same meaning, and the terms "exceeding threshold" and "above threshold" have the same meaning. For example, when the threshold value is 11, "below threshold" has the same meaning as "below threshold" when the threshold value is 10. Furthermore, when the threshold value is 10, "exceeding threshold" has the same meaning as "above threshold" when the threshold value is 11.

[0241] The object management unit 32 is an example of a receiving means. The link generation unit 28 is an example of a setting means. The display control unit 24 is an example of a display control means. The operation receiving unit 27 is an example of an operation receiving means. The area management unit 30 is an example of an area receiving means. [Explanation of symbols]

[0242] 2 Display device [Prior art documents] [Patent documents]

[0243] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-051419

Claims

1. A display device for displaying a handwritten object, a first object displayed by the display device; a receiving means for receiving a second object associated with the first object; a setting means for associating the first object with the second object; A display device comprising:

2. a display control means for displaying an icon that invokes a function of the display device; an operation receiving means for receiving a selection of the icon; 2. The display device according to claim 1, wherein after a predetermined icon is pressed, the receiving means receives the first object based on coordinates input by the input means, and the receiving means receives the second object based on coordinates input by the input means.

3. 3. The display device according to claim 2, further comprising area accepting means for accepting an area including the coordinates input by the input means as the second object when the coordinates have a size equal to or larger than a threshold value.

4. When the receiving means receives the first object based on the coordinates input by the input means, the display control means displays a list of objects stored in an object storage unit; The display device according to claim 2 or 3, wherein the display device accepts the second object selected from the list of objects.

5. an object reserved as the second object is set in the object storage unit; When the receiving means receives the first object based on the coordinates input by the input means, the display control means displays a list of only the objects reserved as the second object in the object storage unit; 5. The display device according to claim 4, further comprising area receiving means for receiving the second object selected from the list of objects.

6. When coordinates are input by the input means after the setting means has associated the first object with the second object, The receiving means identifies the first object based on the coordinates; a display control means for displaying a display range including the second object associated with the first object by the setting means; 6. The display device according to claim 1, wherein the display device is a display device having a plurality of light-emitting elements.

7. The display device according to claim 6, characterized in that, when the entire second object does not fit within the display range, the display control means changes the display magnification so that the larger of the width or height of the second object fits within the display range.

8. When the second object associated with the first object is a character string, the display control means determines whether the language of the character string is right-to-left or left-to-right; In the case of right-to-left writing, the second object is placed at the top right of the display range, 7. The display device according to claim 6, wherein the second object is arranged at the top left of the display range in the case of left-to-right writing.

9. The display device according to any one of claims 6 to 8, characterized in that the display control means reduces a display range including the second object associated with the first object identified based on the coordinates and displays it near the first object.

10. when there are a plurality of second objects associated with the first object, the display control means reduces a display range including the second objects and displays the second objects near the first object, 10. The display device according to claim 9, wherein a display range including the second object is displayed in response to a user's selection.

11. when the display control means displays the second object, identification information of the displayed second object is recorded in chronological order; 11. The display device according to claim 6, wherein the display control means changes the display range in response to pressing a preview button or a next button.

12. A display method performed by a display device that displays a handwritten object, comprising: a step in which a receiving means receives a first object displayed on the display device and a second object associated with the first object; a step of associating the first object with the second object by a setting means; A display method having the following.

13. a display device for displaying a handwritten object; a receiving means for receiving a first object displayed on the display device and a second object associated with the first object; a setting means for associating the first object with the second object; A program to function as a

Images