Display device, program, display method, and display system

Abstract

To provide a display device that can grasp an object when the display device displays information.SOLUTION: The present invention relates to a display device 2 for displaying a display component that shows the existence of information and displaying an object, and the display device has display control means for displaying, in a display screen, an information display section for displaying information related to the display component, with the display component and the object, when a selection of the display component is received.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a display device, a program, a display method, and a display system. [Background technology]

[0002] There are known display devices such as electronic whiteboards on which a user draws strokes using an input means such as a dedicated electronic pen or a finger on a touch panel display, and handwritten data formed by several strokes is displayed. Display devices equipped with relatively large touch panels are placed in conference rooms, public facilities, etc., and are used as electronic whiteboards by multiple users.

[0003] Display devices that allow a user to write comments on handwritten data or the like are known (see, for example, Patent Document 1). Patent Document 1 discloses a technique in which a user places an icon at a location where information such as a comment has been written, and the information is displayed by selecting the icon. Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology has a problem that when a display device displays information, objects such as handwritten data are hidden by the information, which makes it difficult to understand the contents of handwritten data hidden by a displayed comment, or to share the contents of handwritten data hidden when multiple users hold a conference from multiple locations.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide a display device that allows a user to grasp an object when the display device displays information. [Means for solving the problem]

[0006] In view of the above problem, the present invention provides a display device that displays, on a display screen, a display component indicating the presence of information together with an object, the display control means for, when a selection of the display component is accepted, displaying, on the display screen, an information display field for displaying the information associated with the display component together with the display component and the object; When the display component for which a selection has been accepted is displayed to the left of the center of the display screen, the display control means displays the information display column at the left edge of the display screen, and when the display component for which a selection has been accepted is displayed to the right of the center of the display screen, the display control means displays the information display column at the right edge of the display screen. . Effect of the Invention

[0007] According to the present invention, it is possible to provide a display device that allows a user to grasp an object when the display device displays information. [Brief description of the drawings]

[0008] [Figure 1] FIG. 13 is a diagram showing a display example of a comment icon. [Diagram 2] FIG. 1 is a diagram showing an example of a schematic perspective view of a display device. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a display device. [Figure 4] FIG. 2 is a diagram showing an example of a functional block diagram for explaining functions of the display device by dividing them into blocks. [Diagram 5] FIG. 13 is a diagram illustrating a method for inputting a comment. [Figure 6] FIG. 13 is a diagram showing a display example of a comment icon. [Figure 7] FIG. 13 is a diagram showing an example of a comment field that is displayed when a comment icon is pressed. [Figure 8] FIG. 13 is a diagram showing an example of a comment icon indicating a comment written by a user. [Figure 9] FIG. 13 is a diagram showing an example of a comment field that is displayed when a comment icon is pressed. [Figure 10] FIG. 8 is a diagram illustrating comment icons added to FIG. 7. [Figure 11] FIG. 13 is a diagram showing a list of consecutive comment numbers displayed by a pull-down menu in response to a user operation. [Figure 12]FIG. 13 is a diagram showing an example of a display of a comment selected from a pull-down menu. [Figure 13] FIG. 2 shows an example of comment icons present on the left and right sides of a display. [Figure 14] FIG. 13 is a diagram showing an example of a comment field in which the contents of a comment are displayed. [Figure 15] FIG. 13 is a diagram showing an example of the content of a comment displayed in a blank space. [Figure 16] FIG. 13 is a diagram showing an example of a display range in which there is no free space. [Figure 17] FIG. 13 is a diagram showing an example of the content of a comment displayed when there is no free space. [Figure 18] FIG. 13 is a diagram illustrating a method for determining whether or not there is free space. [Figure 19] 11 is a flowchart illustrating an example of a procedure for a display device to display a comment field and comment contents. [Figure 20] 13 is a flowchart illustrating an example of a method for displaying the contents of a comment when a serial number of a comment displayed in a pull-down menu is pressed. [Figure 21] FIG. 1 is a schematic diagram illustrating an example of a display system. [Figure 22] FIG. 13 is a diagram showing another configuration example of the display device. [Diagram 23] FIG. 13 is a diagram showing another configuration example of the display device. [Figure 24] FIG. 13 is a diagram showing another configuration example of the display device. [Diagram 25] FIG. 13 is a diagram showing another configuration example of the display device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A display device and a display method in the display device will be described below with reference to examples as embodiments of the present invention. EXAMPLES

[0010] <Outline of operation> Figure 1 shows an example of the display of a comment icon 9. A comment is additional information such as an annotation or memo, and the comment icon 9 is a display component that indicates the presence of additional information. The comment icon 9 is placed where there is additional information. The additional information is also called the content of the comment.

[0011] In Fig. 1(a), a comment icon 9 is already displayed in the display range 12. When a user wants to check the content of a comment, the user presses the comment icon 9 with an input means such as a finger or a pen. The display device 2 determines whether the position of the comment icon 9 is closer to the left or right edge of the display 220 (display screen), and displays the comment section 11 on the closer side.

[0012] 1(b) shows an example of the comment field 11. Since the comment icon 9 is close to the left edge of the display 220, the comment field 11 is displayed on the left side of the display 220. Note that, when the display device 2 can use only a part of the display 220 as the screen area, it is determined whether the comment icon 9 is closer to the left edge or the right edge of the screen area.

[0013] In this way, the display device 2 can display the comment contents in an area (comment field 11) dedicated to the comment contents, without hiding the handwritten data and the like in the display area 12.

[0014] Furthermore, by displaying the comment field 11 on the display device 2 in a location close to where the user was standing when he pressed the comment icon 9, the user can edit the comment in the comment field 11 without changing his / her position.

[0015] <Terminology> The display device of this embodiment has an inputtable area that is larger than the display area shown on the display. The display area is a part of the inputtable area. The display area refers to the area currently displayed on the display.

[0016] The input means may be any means capable of specifying coordinates on a touch panel and writing by hand, such as a pen, a human finger or hand, or a rod-shaped member.

[0017] A stroke refers to a series of operations in which a user presses an input means against a display, moves it continuously, and then releases it from the display. Stroke data is information displayed on the display based on a trajectory of coordinates input by the input means. Stroke data may be appropriately interpolated. Handwritten data is data having one or more stroke data. Handwritten input refers to handwritten data being input by a user.

[0018] Character strings converted from handwritten data through character recognition may include, in addition to text data, data displayed based on user operations, such as stamps displayed as fixed characters such as "Done," marks, shapes such as circles, rectangles, lines, and stars, speech bubbles, and sticky notes. A character string is one or more characters handled by a computer. The actual substance of a character string is a character code. Characters include numbers, alphabets, and symbols. Character strings are also called text data.

[0019] An object displayed on a display is called an object. An object means a target, but in this embodiment, it means a display target. In this embodiment, handwritten data, figures (circles, rectangles, arrows, etc.), stamps, speech bubbles, sticky notes, images, etc. are also objects.

[0020] The display area 12 refers to the area displayed on the display 220. The inputtable area is the area into which the user can input handwritten data, etc. The display area 12 is a part of the inputtable area.

[0021] A display component is a display of a color, shape, etc. that forms a screen. It is not something that can be disassembled from the display device 2 as a component, but it is sufficient that a human can visually distinguish and recognize it from others. For example, a comment icon 9 is a display component.

[0022] The information may be in any form, such as handwritten data, character strings, figures, or images, as long as the information can be displayed by the user on the display device 2. In this embodiment, the information is described using the term "comment." A display component that indicates the presence of information indicates the presence of some information that is not displayed in its entirety. For example, in Fig. 1(a), a comment icon 9 is displayed in display range 12, but the display content is only "(1)", and the content of the comment is not displayed. By selecting comment icon 9, the content of the comment is displayed in Fig. 1(b).

[0023] The information display field is a field in which information corresponding to a display part is displayed, and in this embodiment, is described as a comment field.

[0024] <Configuration example> 2 shows an example of a schematic perspective view of the display device 2. A user (user) uses the display device 2. In the display device 2 of this embodiment, the user can handwrite on the display 220 using an input means such as a hand H or an electronic pen 2500.

[0025] 2 is oriented horizontally, but it can also be oriented vertically. The user can switch between horizontal and vertical orientations by rotating the display device 2 around the center of the display 220.

[0026] <Hardware configuration example> 3 is a diagram showing an example of a hardware configuration of the display device 2. The display device 2 of this embodiment includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an SSD (Solid State Drive) 204, a network controller 205, and an external device connection I / F 206 (Interface), and is a shared terminal for sharing information among a plurality of users.

[0027] Of these, the CPU 201 controls the overall operation of the display device 2. The ROM 202 stores the CPU 201 and programs used to drive the CPU 201, such as an IPL (Initial Program Loader). The RAM 203 is used as a work area for the CPU 201.

[0028] The SSD 204 stores various data such as programs for the display device 2. This program may be an application that runs on an information processing device equipped with a general-purpose OS (Windows (registered trademark), Mac OS (registered trademark), Android (registered trademark), iOS (registered trademark), etc.). In this case, the display device 2 is usually used as a general-purpose information processing terminal, but when a user executes an installed application program, the user can write by hand, etc., just as with a dedicated device as a display device.

[0029] A network controller 205 controls communication with a network. An external device connection I / F 206 controls communication with a USB (Universal Serial Bus) memory 2600 and external devices (a camera 2900, a speaker 2800, and a microphone 2700).

[0030] The display device 2 also includes a capture device 211, a GPU 212, a display controller 213, a contact sensor 214, a sensor controller 215, an electronic pen controller 216, a short-range communication unit 219, and an antenna 219a of the short-range communication unit 219.

[0031] Of these, the capture device 211 displays video information as still images or moving images on a display 220 of the PC 10. The GPU (Graphics Processing Unit) 212 is a semiconductor chip that specializes in graphics. The display controller 213 controls and manages screen display to output an output image from the GPU 212 to the display 220 etc.

[0032] The contact sensor 214 detects that the electronic pen 2500, the user's hand H, or the like has come into contact with the display 220. When the electronic pen 2500 and the hand H are not distinguished from each other, they are referred to as an input means 291.

[0033] The sensor controller 215 controls the processing of the contact sensor 214. The contact sensor 214 inputs and detects coordinates by an infrared blocking method. The method of inputting and detecting coordinates may be a method in which two light receiving and emitting devices installed at both ends of the upper side of the display 220 emit a plurality of infrared rays parallel to the display 220, and receive light that is reflected by a reflecting member installed around the display 220 and returns along the same optical path as the optical path of the light emitted by the light receiving element.

[0034] The contact sensor 214 outputs the ID of the infrared rays radiated by the two light receiving and emitting devices that are blocked by the object to the sensor controller 215, and the sensor controller 215 specifies the coordinate position that is the contact position of the object. That is, the contact sensor 214 forms a touch panel. The electronic pen controller 216 communicates with the electronic pen 2500 to determine whether the pen tip or the pen tail has touched the display 220. The near field communication unit 219 is a communication circuit such as NFC or Bluetooth (registered trademark).

[0035] Furthermore, the display device 2 includes a bus line 210. The bus line 210 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 201.

[0036] The contact sensor 214 is not limited to an infrared blocking type, and may be a capacitive touch panel that identifies the contact position by detecting a change in capacitance. The contact sensor 214 may be a resistive film type touch panel that identifies the contact position by a voltage change in two opposing resistive films. The contact sensor 214 may be an electromagnetic induction type touch panel that identifies the contact position by detecting electromagnetic induction caused by a contact object touching the display unit. The contact sensor 214 may use these various detection means. The electronic pen controller 216 may determine whether or not a touch has been made on not only the tip and end of the electronic pen 2500, but also on a part of the electronic pen 2500 that is held by the user or on other parts of the electronic pen.

[0037] <About the function> Fig. 4 is a diagram showing an example of a functional block diagram illustrating functions of the display device 2. The display device 2 shown in Fig. 4 has a reception unit 21, a drawing data generation unit 22, a display control unit 23, a comment control unit 24, a data recording unit 25, and an operation reception unit 26.

[0038] Each function of the display device 2 is a function or means realized when any of the components shown in FIG. 3 operates according to an instruction from the CPU 201 in accordance with an application loaded on the RAM 203.

[0039] The reception unit 21 detects a contact position based on coordinates of a point where the input means 291 (a finger or an electronic pen 2500) touches the display 220. The reception unit 21 detects the contact position at predetermined time intervals and acquires a sequence of coordinate points.

[0040] The drawing data generating unit 22 acquires the coordinates of the touched point of the input means 291 from the receiving unit 21. The drawing data generating unit 22 generates continuous stroke data by interpolating this sequence of coordinate points.

[0041] The display control unit 23 displays on the display 220 handwritten data, character strings, figures, images converted from the handwritten data, menus for user operations, and the like.

[0042] The comment control unit 24 manages the comments. The comment control unit 24 manages the comment ID numbered for each comment, the position of each comment (X coordinate, Y coordinate), and the content of the comment.

[0043] The data recording unit 25 stores handwritten data, character strings, figures, images, and other objects handwritten on the display device 2 in the storage unit 29. The above-mentioned comments are also saved in the storage unit 29 by the data recording unit 25. An example of comment information stored in the storage unit 29 is shown in Table 1.

[0044] The operation reception unit 26 receives operations for a menu displayed on the display device 2 for the user to operate.

[0045] [Table 1] Table 1 shows comment information related to comments among the information stored in the storage unit 29. The storage unit 29 stores a comment ID, a position (X coordinate, Y coordinate), and the content of the comment in association with each other. The comment ID is identification information that identifies a comment in the inputtable area. Since the inputtable area is larger than the display range 12, the comment ID is unique to the inputtable area. The X coordinate is the X coordinate of the comment icon 9 in the input area. The Y coordinate is the Y coordinate of the comment icon 9 in the input area. The content of the comment is the comment text. The content of the comment may be handwritten data or a character string converted by character recognition. The content of the comment may also be a figure, an icon, an image, or audio data.

[0046] <Example of comment display> An example of displaying the contents of a comment will be described with reference to Figs. 5 to 9. First, Fig. 5(a) is a diagram for explaining a method of inputting a comment. For example, a user inputs a character string 301 during a meeting. The character string 301 may be handwritten data. Furthermore, the user highlights the character string 301 with a marker 302. A marker is a semi-transparent colored stroke that is used to make an arbitrary object stand out. With this input using the marker 302, the comment control unit 24 causes the display control unit 23 to display a comment dialog 303.

[0047] 5(b) shows an example of the comment dialogue 303. Here, the user has entered "Limit access permissions to important information" into the comment dialogue 303. In other words, the comment is entered in response to the input of predetermined handwritten data (marker 302).

[0048] The method of inputting a comment shall follow the operation method of the display device 2. As another input method, the user presses a comment button and specifies the input position. Similarly, the display device 2 displays a comment dialog 303, and the user can input any comment.

[0049] After inputting a comment, when the user closes comment dialog 303, comment icon 9 will be displayed at the upper left or the like of marker 302. Therefore, according to the method of inputting a comment using marker 302, comment icon 9 will be positioned around marker 302. The position of comment icon 9 may also be at the upper right, lower left, lower right, etc. of marker 302.

[0050] 6 shows an example of the display of the comment icon 9. Note that the user can move the position of the comment icon 9 by pressing and holding the comment icon 9 with the input means 291 and then dragging it.

[0051] The comment information for the state in Figure 6 is shown in Table 2.

[0052] [Table 2] In Table 2, one comment corresponding to the comment icon 9 in FIG. 6 is registered.

[0053] Next, a comment field 11 that is displayed when the user presses the comment icon 9 in the state shown in FIG. 6 will be described.

[0054] 7 is an example of the comment field 11 that is displayed when the comment icon 9 is pressed. When the user presses the comment icon 9, the operation reception unit 26 receives the press, refers to the comment information in Table 2, and identifies the comment ID based on the contact position of the input means 291. The operation reception unit 26 notifies the comment control unit 24 that the comment icon 9 has been pressed, along with the comment ID.

[0055] The comment control unit 24 determines whether the comment field 11 is being displayed. If it is not being displayed, the comment control unit 24 refers to the position associated with the comment ID of the pressed comment icon 9 in the comment information in Table 2, and determines whether the position of the comment icon 9 is to the right or left of the center of the display 220. This determination is made based on whether the position of the comment icon 9 is to the right or left of half of the resolution pixel count of the size (width) of the screen of the display 220. For example, when the screen resolution is 1280×1024 pixels (SXGA: Super eXtended Graphics Array), if the position of the comment icon 9 (pointing to the lower left point of the square surrounding the comment icon 9) is in the range of 0 to 640 pixels counting from the left edge of the screen, it is determined to be on the left side of the screen, and if it is in the range of 641 to 1280 pixels, it is determined to be on the right side of the screen. If the resolution of the screen is changed, the left or right side is determined based on the numerical value of the changed resolution.

[0056] In Fig. 7, since it is determined that the comment icon 9 is on the left side, the display control unit 23 displays the comment section 11 on the left side (left edge) of the display 220. In other words, the display control unit 23 displays the comment section 11 in an area close to the comment icon 9 that has been selected. This process corresponds to steps S2 and S4 in the flowchart of Fig. 19, which will be described later. In the following, the process will be assigned the step numbers of Fig. 19 or Fig. 20.

[0057] The comment control unit 24 prepares a comment field 11 of, for example, a fixed width from the left side of the display 220. The display control unit 23 moves the character string 301 that is originally displayed to the right as a whole by the width of the comment field 11. In other words, the display control unit 23 moves the display range 12 that was displayed before the comment field 11 was displayed in the opposite direction to the comment field 11.

[0058] The comment control unit 24 acquires the content of the comment associated with the comment icon 9 from the storage unit 29, and the display control unit 23 displays the content of the comment in the comment field 11 (FIG. 19: step S6).

[0059] Since the comment field 11 is displayed in this manner, the comment content and the comment field 11 are displayed at separate positions from the character string 301, and inconveniences such as the comment content and the character string 301 overlapping, making it difficult for the user to see, can be prevented.

[0060] Furthermore, the comment field 11 has a pull-down menu 13. When there are multiple comment icons 9 in the display area 12, the pull-down menu 13 allows the user to select the content of the comment to be displayed in the comment field 11.

[0061] Furthermore, the comment field 11 has a close button 14. The close button 14 accepts an operation to close the comment field 11 by the user.

[0062] The width of the comment field 11 may be determined in advance, or may be optimized according to the width of the comment. The process for displaying the contents of the comment differs depending on whether the comment is handwritten data or a character string.

[0063] When the content of the comment is handwritten data, the comment control unit 24 judges whether the handwritten data can be displayed in a predetermined minimum width of the comment field 11 (for example, 1 / 4 of the width of the display 220). When the handwritten data cannot be displayed in the predetermined minimum width of the comment field 11, the comment control unit 24 judges whether the handwritten data can be displayed in a predetermined maximum width of the comment field 11 (for example, 1 / 3 of the width of the display 220). When the handwritten data fits into the predetermined maximum width of the comment field 11, the width of the comment field 11 is determined according to the width of the handwritten data. In this case, the width of the comment field 11 is approximately equal to the width of the handwritten data. When the handwritten data cannot fit into the predetermined maximum width of the comment field 11, the comment control unit 24 reduces the handwritten data so that it fits into the predetermined maximum width of the comment field 11. When the content of the comment is small and difficult to read, it is preferable that the user can manually change the width of the comment field 11.

[0064] When the content of the comment is a character string, the comment control unit 24 simply fills the content of the comment into the predetermined minimum width of the comment field 11. In other words, the content of the comment (character string) is displayed on a new line within the width of the comment field 11.

[0065] 7, the display range 12 naturally becomes narrower. When the display range 12 becomes narrower and the character string 301 cannot be fully displayed, the display control unit 23 may, for example, reduce the character string 301. When the comment field 11 is displayed on the left side of the display 220, the beginning of the character string 301 remains in the display range 12 (in the case of horizontal writing from left to right), so it is not necessarily necessary to reduce the character string. In this case, the character string 301 that extends beyond the display range 12 is temporarily hidden.

[0066] Also, instead of reducing the character string 301, if there is no handwritten data or the like in the inputtable range to the right of the display range 12, the comment control unit 24 can temporarily use it. In this case, the entire character string 301 is not displayed, but the character string 301 that extends beyond the inputtable range continues to exist as hidden.

[0067] Next, a description will be given of a case where the comment field 11 is displayed on the right side of the display 220. When the comment field 11 is displayed on the right side of the display 220, the method of displaying the comment contents may be the same.

[0068] Fig. 8 shows an example of a comment icon 9 indicating a comment written by a user. In the explanation of Fig. 8, differences from Fig. 7 will be mainly explained.

[0069] For example, a user inputs a character string 301 during a meeting. The character string 301 may be handwritten data. Furthermore, the user inputs a comment near the character string 301 (using a marker 302) in the same manner as in FIG. 5. In FIG. 8, a comment icon 9 is displayed in the upper left of the marker 302.

[0070] The comment information for the state in Figure 8 is shown in Table 3.

[0071] [Table 3] The comment information in Table 3 is the same as Table 2 except for the X coordinate.

[0072] Next, a comment field 11 that is displayed when the user presses the comment icon 9 in the state shown in FIG. 8 will be described.

[0073] 9 is an example of the comment field 11 that is displayed when the comment icon 9 is pressed. When the user presses the comment icon 9, the operation reception unit 26 receives the press, refers to the comment information in Table 3, and identifies the comment ID based on the contact position of the input means 291. The operation reception unit 26 notifies the comment control unit 24 that the comment icon 9 has been pressed, along with the comment ID.

[0074] The comment control unit 24 determines whether the comment field 11 is being displayed. If it is not being displayed, the comment control unit 24 refers to the position (X coordinate) associated with the comment ID of the pressed comment icon 9 in the comment information in Table 3, and determines whether the position of the comment icon 9 is to the right or left of the center of the display 220. The method of determination may be the same as in FIG. 7.

[0075] In Fig. 9, since it is determined that the comment icon 9 is on the right side, the display control unit 23 displays the comment field 11 on the right side of the display 220 (Fig. 19: steps S2 and S4). The comment control unit 24 prepares the comment field 11 of, for example, a fixed width from the right side (right end) of the display 220. The display control unit 23 moves the character string 301 that is originally displayed overall to the left by the width of the comment field 11.

[0076] The comment control unit 24 acquires the content of the comment associated with the comment icon 9 from the comment information in Table 3, and the display control unit 23 displays the content of the comment in the comment field 11 (FIG. 19: step S5).

[0077] Since the comment field 11 is displayed in this manner, the comment content and the comment field 11 are displayed at separate positions from the character string 301, and inconveniences such as the comment content and the character string 301 overlapping, making it difficult for the user to see, can be prevented.

[0078] <Add comment> Next, a case where a user adds a comment will be described.

[0079] Fig. 10 is a diagram for explaining comment icons 9 added to the display device 2 of Fig. 7. The method for entering a comment may be the same as that of Fig. 5. To distinguish between the comment icons 9, the alphabets a and b are added to the comment icons 9 for explanation. Comment icon 9a corresponds to a comment input by handwriting with marker 302. Comment icon 9b corresponds to a comment input by handwriting with marker 304.

[0080] By adding a comment, one new comment (one record) is also registered in the comment information in the storage unit 29.

[0081] [Table 4] Table 4 shows the comment information in the state of Fig. 10. In Table 4, two comments are registered.

[0082] When multiple comments are entered, the comment control unit 24 displays a selectable serial number or the like for identifying the comments in the pull-down menu 13. This serial number is also displayed on the comment icon 9, allowing the user to understand the correspondence between the comment icon 9 and the serial number displayed on the pull-down menu 13. The serial number may be the last digit of the comment ID, or may be assigned separately to each line of the comment information.

[0083] Fig. 11 shows the list 15 of comment serial numbers displayed by the pull-down menu 13 in response to a user operation. In Fig. 11, the serial numbers ((1), (2)) of two comment icons entered on the display 220 are displayed. The user selects a comment serial number in the pull-down menu 13 with the input means 291. The operation reception unit 26 receives the selected comment serial number, and the display control unit 23 changes the brightness and color of the selected serial number (Fig. 20: step S101).

[0084] However, the sequential numbers of comments displayed in the pull-down menu 13 are limited to those within the display range 12 (the range of the entire display 220 that the user can see). This is because it is considered that in many cases, what a user wants to display in the comment field 11 are comments within the display range 12. However, in response to a user's operation, the comment control unit 24 may display the sequential numbers of all comments within the inputtable range in the pull-down menu 13.

[0085] Fig. 12 shows an example of the display of a comment selected in the pull-down menu 13. The operation reception unit 26 notifies the comment control unit 24 of the serial number of the selected comment, and the comment control unit 24 acquires the content of the comment identified by the serial number of the comment from the storage unit 29. The display control unit 23 displays the content of the comment in the comment field 11 (Fig. 20: steps S102 and S103).

[0086] 13, when comment icons 9 are present on the left and right sides of the display 220, the comment field 11 is displayed on the left or right side of the display 220 according to the comment icon 9 that is initially selected. In other words, no matter which comment serial number in FIG. 13 the user selects in the pull-down menu 13, the comment field 11 does not move left or right each time a selection is made.

[0087] <Simultaneous display of multiple comments> There is a limit to the number of comments that the comment field 11 can display at one time (for example, one). Therefore, if a user wants to simultaneously display more than the limit of comments, the display device 2 displays the contents of the comments outside the comment field 11. For example, the display device 2 displays the contents of the comments in an empty space in the display range 12.

[0088] Fig. 14 shows the comment field 11 in a state where the content of the comment is displayed. The content of the comment in the comment field 11 in Fig. 14 is that of the comment icon 9b. When the user presses the comment icon 9a, the operation reception unit 26 receives the notification that the comment icon 9a has been pressed. The operation reception unit 26 notifies the comment control unit 24 that the comment icon 9a has been pressed, and the comment control unit 24 determines whether the comment field 11 is being displayed (Fig. 19: steps S1 and S2).

[0089] The comment control unit 24 determines whether the comment ID of the pressed comment icon 9a matches the comment ID of the comment displayed in the comment field 11, and if they do not match, retrieves the content of the comment of the pressed comment icon 9a from the memory unit 29 (Figure 19: step S3).

[0090] Then, the comment control unit 24 determines whether the content of the comment can be displayed in the available space on the display 220. If the content of the comment can be displayed in the available space, the display control unit 23 displays the content of the comment in the available space (FIG. 19: steps S7 and S8), as shown in FIG. 15. A method of determining whether or not there is available space will be described with reference to FIG. 18.

[0091] In this way, the display device 2 can simultaneously display a plurality of comments in the comment field 11 and the display area 12.

[0092] Fig. 16 shows the same screen as Fig. 14, but in the case where there is no free space in the display range 12. When there is no free space on the display 220 to display the content of the comment, the comment control unit 24 determines the area below the pressed comment icon 9a as the display position for the content of the comment.

[0093] As shown in Fig. 17, the comment control unit 24 increases the density of the object in area 308 for displaying the comment content, and displays the comment content, for example, semi-transparently (Fig. 19: steps S7, S9, S10). Although it is difficult to see in Fig. 17 due to drawing conditions, the comment content 307 corresponding to the comment icon 9a is displayed semi-transparently, and the character string 301 can be seen through it. Note that density is also called saturation, and increasing the density means, for example, decreasing the saturation. The display control unit 23 may simply increase the RGB values ​​respectively.

[0094] The area 308 in which the comment content is displayed is preferably determined according to the size of the comment content. When the comment content is handwritten data, for example, the area 308 is a circumscribing rectangle of the comment content. When the comment content is a character string, for example, the width of the area 308 may be the same as the display range 12, and the height is determined according to the number of characters.

[0095] In this way, even if there is no free space in the display range 12, the display device 2 can display the contents of the comment while allowing the objects in the display range 12 to be viewed.

[0096] It should be noted that the content of the comment displayed in the display area 12 is erased by the display control unit 23 when the comment icon 9a is pressed.

[0097] 18 is a diagram for explaining a method for determining whether or not there is free space. The comment control unit 24 compares the distance L from the edge of the display 220 to the edge of the object with the width W of the comment content. When the comment field 11 is on the left side of the display 220, the edge of the display 220 is the right edge, and when the comment field 11 is on the right side of the display 220, the edge of the display 220 is the left edge. When the comment field 11 is on the left side of the display 220, the edge of the object is the right edge, and when the comment field 11 is on the right side of the display 220, the edge of the object is the left edge.

[0098] When the comment content is handwritten data, the comment control unit 24 determines that there is free space if the distance L is equal to or greater than the width W. If not, the comment control unit 24 determines that there is no free space. Note that even if the distance L is not equal to or greater than the width W, the comment control unit 24 may reduce the comment content and determine whether there is free space. It is preferable to set a limit on the reduction rate.

[0099] When the content of a comment is a character string, the character string often fits into the free space by line breaks. For this reason, the comment control unit 24 determines that there is free space if the distance L is equal to or greater than a threshold. The reason for setting a threshold is that too many line breaks make the comment difficult to read.

[0100] <Operation procedure> Fig. 19 is an example of a flow chart showing a procedure in which the display device 2 displays the comment field 11 and the comment contents. The process in Fig. 19 starts when the user presses the comment icon 9 with the input means 291 (S1). The operation reception unit 26 identifies the pressed comment icon 9 based on the coordinates where the input means 291 touches the display 220.

[0101] The comment control unit 24 judges whether or not the comment field 11 is already displayed (S2). Since the comment control unit 24 displays the comment field 11, the comment control unit 24 manages whether or not the comment field 11 is displayed.

[0102] If the determination in step S2 is Yes, the comment control unit 24 determines (S3) whether or not the content of the pressed comment icon 9 matches the content of the comment displayed in the comment field 11. The comment ID displayed in the comment field 11 is managed by the comment control unit 24.

[0103] If the determination in step S3 is Yes, the comment content has already been displayed, so the comment control unit 24 ends the process without doing anything.

[0104] If the determination in step S3 is No, the user has attempted to display a comment other than the comment currently being displayed in the comment field 11. In other words, the user has attempted to display the contents of the comment in the display area 12 as well. Therefore, the comment control unit 24 determines whether or not there is free space on the display 220 to display the comment (S7).

[0105] If the determination in step S7 is Yes, the comment control unit 24 displays the content of the comment in the empty space (S8).

[0106] If the determination in step S7 is No, the comment control unit 24 makes the comment semi-transparent (S9). This is to make the object easier to see even if the comment overlaps with an object other than the comment. The comment control unit 24 may further increase the density of the object that overlaps with the area where the content of the comment is displayed (the area below the comment icon 9). Alternatively, the comment control unit 24 may increase the density of the entire display range 12.

[0107] The display control unit 23 displays the content of the comment, for example, at the lower right of the comment icon 9 (S10).

[0108] If the determination in step S2 is No, the comment control unit 24 determines whether the position of the pressed comment icon 9 is to the right of the center of the display 220 (S4).

[0109] If the determination in step S4 is Yes, the comment control unit 24 determines that the comment section 11 is to be displayed on the right side of the display 220, and the display control unit 23 displays the comment section 11 on the right side of the display 220 (S5). The display control unit 23 displays, in the comment section 11, the content of the comment determined based on the position of the pressed comment icon 9.

[0110] If the determination in step S4 is No, the comment control unit 24 determines that the comment field 11 is to be displayed on the left side of the display 220, and the display control unit 23 displays the comment field 11 on the left side of the display 220 (S6). The display control unit 23 displays, in the comment field 11, the content of the comment determined based on the position of the pressed comment icon 9.

[0111] In steps S5 and S6, the comment control unit 24 refers to the storage unit 29 to identify the comment icon 9 in the display range 12. Since the coordinates of the display range 12 in the input possible range are known, the comment control unit 24 only needs to identify the comment icon 9 in this coordinate range. The comment control unit 24 sets the serial number of the comment in the display range 12 in the pull-down menu 13. The processing when a serial number in the pull-down menu 13 is pressed will be described with reference to FIG. 20.

[0112] <<Display the comment contents by pressing the serial number in the pull-down menu>> FIG. 20 is an example of a flow chart illustrating a method for displaying the contents of a comment when the serial number of the comment displayed in the pull-down menu 13 is pressed.

[0113] The user presses a serial number on the pull-down menu 13 (S101). The operation reception unit 26 receives the serial number and notifies the serial number to the comment control unit 24. The comment control unit 24 identifies a comment ID based on the serial number.

[0114] The comment control unit 24 determines whether or not the comment currently being displayed in the comment field 11 has been changed (S102). Since the comment control unit 24 manages the comments currently being displayed in the comment field 11, it can easily determine whether or not the comment currently being displayed in the comment field 11 has been changed.

[0115] If the determination in step S102 is No, the comment control unit 24 does nothing and the process ends.

[0116] If the determination in step S102 is Yes, the comment control unit 24 switches the content of the comment displayed in the comment field 11 to the content of the comment selected by the user (S103).

[0117] <Major Effects> As described above, the display device 2 can display the comment contents without hiding the display range 12 by displaying the comment contents in an area (comment field 11) dedicated to the comment contents.

[0118] Furthermore, by displaying the comment section 11 on the display device 2 in a location close to where the user was standing when the user pressed the comment icon 9, the user can operate the comment section 11 without changing his / her position.

[0119] Furthermore, the display device 2 can display comments in a display area 12 separately from the comment field 11. EXAMPLES

[0120] In this embodiment, a display system 19 in which a display device 2 and a terminal device 31 communicate with each other via a server device 32 will be described.

[0121] 21 is an example of a schematic configuration diagram of a display system 19. A display device 2 and a terminal device 31 are each connected to a server device 32 via a network such as the Internet. The terminal device 31 may be, for example, a PC, a smartphone, a tablet terminal, a game machine, a PDA (Personal Digital Assistant), a wearable PC (Personal Computer), a car navigation system, etc. as long as it can run a web browser or a dedicated application.

[0122] The functions of the display device 2 may be the same as those of the first embodiment, except for communicating with the server device 32. The communication control unit 27 of the display device 2 transmits object data (data to be displayed) to the server device 32. The server device 32 transmits a URL for a conference that is commonly accessed by the display device 2 and the terminal device 31 to the display device 2, and the display device 2 transmits the URL by email or the like to the terminal device 31. This enables the terminal device 31 to communicate with the server device 32 and receive the object data transmitted by the display device 2. Note that when the terminal device 31 displays an object using a Web browser, the server device 32 generates screen information using HTML or the like. When the terminal device 31 displays an object using a dedicated app, the server device 32 only needs to transmit mainly the object and its coordinates to the terminal device 31 as screen information.

[0123] Furthermore, if the terminal device 31 has a touch panel, the user can input handwritten data into the table and clean up the data by using the terminal device 31 in the same way as the display device 2. The cleaned up object data is transmitted to the display device 2 via the server device 32.

[0124] In this way, the display system 19 allows the display device 2 and the terminal device 31 to process objects interactively.

[0125] 21, the server device 32 may perform processing of the display device 2. In this case, the display device 2 displays stroke data and transmits various events to the server device 32. The server device 32 performs a clean copy or the like to generate screen information for the display device 2 and transmits it to the display device 2. EXAMPLES

[0126] In the following embodiment, another configuration example of the display device 2 will be described.

[0127] <<Another configuration example 1 of the display device>> Although the display device 2 of the present embodiment has been described as having a large touch panel, the display device 2 is not limited to having a touch panel.

[0128] Fig. 22 is a diagram showing another example of the configuration of the display device 2. In Fig. 22, a projector 411 is installed on the upper side of a normal whiteboard 413. This projector 411 corresponds to the display device 2. The normal whiteboard 413 is not a flat panel display integrated with a touch panel, but a whiteboard on which a user directly writes by hand with a marker. The whiteboard may be a blackboard, and may be any flat surface large enough to project an image.

[0129] The projector 411 has an ultra-short focal length optical system and can project an image with little distortion from a distance of about 10 cm onto the whiteboard 413. This image may be transmitted from a PC connected wirelessly or via a wire, or may be stored in the projector 411.

[0130] A user writes handwriting on the whiteboard 413 using a dedicated electronic pen 2501. The electronic pen 2501 has a light-emitting part, for example at the tip, that is switched on and emits light when the user presses it against the whiteboard 413 to write. The wavelength of the light is near-infrared or infrared, so it is invisible to the user's eyes. The projector 411 has a camera that captures the light-emitting part, analyzes the image, and identifies the direction of the electronic pen 2501. The electronic pen 2501 also emits sound waves along with the light emission, and the projector 411 calculates the distance based on the arrival time of the sound waves. The projector 411 can identify the position of the electronic pen 2501 based on the direction and distance. Handwritten data is drawn (projected) at the position of the electronic pen 2501.

[0131] Since the projector 411 projects the menu 430, when the user presses a button with the electronic pen 2501, the projector 411 identifies the position of the electronic pen 2501 and the pressed button by the ON signal of the switch. For example, when the save button 431 is pressed, the handwritten data (coordinate point sequence) handwritten by the user is saved by the projector 411. The projector 411 saves the handwritten information in a predetermined server 412 or USB memory 2600 or the like. The handwritten information is saved for each page. Since the information is saved as coordinates rather than as image data, the user can re-edit it. However, in this embodiment, the menu 430 does not need to be displayed because the operation command can be called up by handwriting. EXAMPLES

[0132] <<Another configuration example 2 of the display device>> 23 is a diagram showing another example of the configuration of the display device 2. In the example of FIG.

[0133] The terminal device 600 is connected by wire to the image projection device 700A and the pen motion detection device 810. The image projection device 700A projects image data input by the terminal device 600 onto the screen 800.

[0134] The pen operation detection device 810 communicates with the electronic pen 820 and detects the operation of the electronic pen 820 in the vicinity of the screen 800. Specifically, the electronic pen 820 detects coordinate information indicating a point pointed to by the electronic pen 820 on the screen 800 (the detection method may be the same as that in FIG. 22 ), and transmits the coordinate information to the terminal device 600.

[0135] The terminal device 600 generates image data of handwritten data input by the electronic pen 820 based on the coordinate information received from the pen operation detection device 810, and causes the image projection device 700A to draw an image of the handwritten data on the screen 800.

[0136] Furthermore, the terminal device 600 generates superimposed image data indicating a superimposed image obtained by combining the background image projected by the image projection device 700A and the image of the handwritten data input by the electronic pen 820. EXAMPLES

[0137] <<Another configuration example 3 of the display device>> Fig. 24 is a diagram showing a configuration example of the display device 2. In the example of Fig. 24, the display device 2 includes a terminal device 600, a display 800A, and a pen motion detection device 810A.

[0138] The pen action detection device 810A is disposed near the display 800A, detects coordinate information indicating a point indicated by the electronic pen 820A on the display 800A (the detection method may be the same as in FIG. 22), and transmits the coordinate information to the terminal device 600. In the example of FIG. 24, the electronic pen 820A may be charged by the terminal device 600 via a USB connector.

[0139] Based on the coordinate information received from the pen operation detection device 810A, the terminal device 600 generates image data of the image of the handwritten data input by the electronic pen 820A, and displays the image data on the display 800A. EXAMPLES

[0140] <<Another Configuration Example 4 of the Display Device>> Fig. 25 is a diagram showing a configuration example of the display device 2. In the example of Fig. 25, the display device 2 includes a terminal device 600 and an image projection device 700A.

[0141] The terminal device 600 performs wireless communication (Bluetooth (registered trademark) or the like) with the electronic pen 820B to receive coordinate information of a point indicated by the electronic pen 820B on the screen 800. The coordinate information may be obtained by the electronic pen 820B reading minute position information formed on the screen 800, or the coordinate information may be received from the screen 800.

[0142] Then, based on the received coordinate information, the terminal device 600 generates image data of an image of the handwritten data input by the electronic pen 820B, and causes the image projection device 700A to project the image of the handwritten data.

[0143] Furthermore, the terminal device 600 generates superimposed image data indicating a superimposed image obtained by combining the background image projected by the image projection device 700A and the image of the handwritten data input by the electronic pen 820B.

[0144] As described above, each of the above-described embodiments can be applied to various system configurations.

[0145] <Other application examples> The above describes the best mode for carrying out the present invention using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0146] For example, in this embodiment, an upright display device 2 has been described, but this embodiment can be similarly applied to a flat-laying display device 2. In the flat-laying display device 2, the comment section 11 may be displayed on any one of the four sides, not just on the left and right sides of the display.

[0147] For example, in the present embodiment, an electronic whiteboard has been described as an example, but the electronic whiteboard may be called an electronic whiteboard, an electronic information board, or the like. In addition, the present embodiment can be suitably applied to any information processing device having a touch panel. The information processing device equipped with a touch panel may be, for example, a PJ (Projector), an output device such as digital signage, a HUD (Head Up Display) device, an industrial machine, an imaging device, a sound collecting device, a medical device, a network home appliance, a notebook PC (Personal Computer), a mobile phone, a smartphone, a tablet terminal, a game machine, a PDA (Personal Digital Assistant), a digital camera, a wearable PC, or a desktop PC.

[0148] In addition, the display device 2 of this embodiment detects the coordinates of the pen by detecting the coordinates of the pen tip with a touch panel, but the coordinates of the pen tip may also be detected by ultrasonic waves. The pen emits light and also emits ultrasonic waves, and the display device 2 calculates the distance based on the arrival time of the ultrasonic waves. The display device 2 can identify the position of the pen based on the direction and distance. The projector draws (projects) the trajectory of the pen as stroke data.

[0149] Moreover, the configuration example in FIG. 4 and the like is divided according to main functions in order to facilitate understanding of the processing by the display device 2. The present invention is not limited by the manner in which the processing units are divided or the names of the processing units. The processing by the display device 2 can be divided into more processing units according to the processing contents. Also, it can be divided so that one processing unit includes more processes.

[0150] Furthermore, part of the processing performed by the display device 2 may be performed by a server connected to the display device 2 via a network.

[0151] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above.

[0152] The reception unit 21 is an example of a reception means. The comment control unit 24 is an example of an information display control means. The display control unit 23 is an example of a display control means. [Explanation of symbols]

[0153] 2 Display device [Prior art documents] [Patent documents]

[0154] [Patent Document 1] JP 2019-204431 A

Claims

1. A display device that displays a display component indicating the presence of information together with an object on a display screen, a display control means for displaying, when a selection of the display part is accepted, an information display field for displaying the information associated with the display part together with the display part and the object on the display screen; When the display component for which selection has been accepted is displayed to the left of the center of the display screen, the display control means displays the information display column on the left edge of the display screen; when the display component for which selection has been accepted is displayed to the right of the center of the display screen, the display control means displays the information display column at the right edge of the display screen. Display device.

2. an information display control means for generating, when a selection of the display part is accepted, the information display column for displaying the information associated with the display part; The display device according to claim 1 , further comprising:

3. 2 . The display device according to claim 1 , wherein the display control means displays the information display column in one of the left and right edge areas of the display screen, the information display column being closest to the display component for which a selection has been accepted.

4. The display device of claim 1, wherein the display control means moves the display range in which the object is displayed, which was displayed before the information display column is displayed, to the right in the opposite direction when the information display column is displayed in the leftmost area, and moves it to the left in the opposite direction when the information display column is displayed in the rightmost area.

5. The display device according to claim 4 , wherein the information corresponding to the display parts is input in response to input of predetermined handwritten data.

6. 6. The display device according to claim 1, wherein the display control means displays the information display field having a width determined according to the width of one line of the information, which is a character string.

7. When a plurality of the display components are displayed in the display range, the display control means displays, in the information display field, identification information of the information corresponding to a plurality of display components in a selectable manner, and displays the identification information of the information in the display components in the display range; The display device according to claim 5 , wherein the information selected from the identification information of the information in the information display field is displayed in the information display field.

8. when the information display column is displayed and a selection of a display part other than the display part that caused the information display column to be displayed is accepted, the information display control means determines whether or not there is a space to display the information corresponding to the display part whose selection has been accepted within a display range in which objects are displayed; The display device according to claim 2 , wherein the display control means displays the information in the space if the space is available.

9. The display device according to claim 8 , wherein the display control means, when there is no space, displays the information semi-transparently around the display component that has been selected.

10. A display device that displays a display component indicating the presence of information together with an object on a display screen, when the selection of the display part is accepted, the display control means functions as a display control means for displaying, on the display screen, an information display field for displaying the information associated with the display part together with the display part and the object; When the display component for which selection has been accepted is displayed to the left of the center of the display screen, the display control means displays the information display column at the left edge of the display screen; When the display component for which the selection has been accepted is displayed to the right of the center of the display screen, the display control means displays the information display column on the right edge of the display screen. Program for.

11. A display method performed by a display device that displays a display component indicating the presence of information together with an object on a display screen, comprising the steps of: a step of displaying, when the display control means receives a selection of the display part, an information display field for displaying the information associated with the display part together with the display part and the object on the display screen; When the display component for which selection has been accepted is displayed to the left of the center of the display screen, the display control means displays the information display column at the left edge of the display screen; When the display component for which the selection has been accepted is displayed to the right of the center of the display screen, the display control means displays the information display column on the right edge of the display screen. Display method.

12. A display system in which a server device and a display device communicate with each other via a network, the display device displays a display component indicating the presence of information together with the object on a display screen; a display control means for, when a selection of a display part indicating the presence of information is accepted, displaying on the display screen an information display field for displaying the information associated with the display part together with the display part and the object; When the display component for which selection has been accepted is displayed to the left of the center of the display screen, the display control means displays the information display column at the left edge of the display screen; When the display component for which the selection has been accepted is displayed to the right of the center of the display screen, the display control means displays the information display column on the right edge of the display screen. Display system.

Images