Display device, program, display method, and display system

Abstract

To provide a display device which can restore a desired display state.SOLUTION: A display device displaying an object comprises a display state save processing unit which saves display state data of the displayed object every time a condition is satisfied, an operation receiving unit which receives selection of the display state data, and a restoration processing unit which restores the display state of the object on the basis of the display state data of which the operation receiving unit received the selection.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a display device, a program, a display method, and a display system. [Background technology]

[0002] There are known display devices such as electronic whiteboards that display handwritten data drawn by a user using a special electronic pen or finger on a touch panel display. Unlike conventional whiteboards, handwritten data can be saved as electronic data, and the display device can be connected to an external device such as a PC (Personal Computer) to display images of documents displayed on the external device.

[0003] Some display devices are equipped with an Undo / Redo function (see, for example, Patent Document 1). The Undo / Redo function refers to returning to a state prior to an operation during editing, or advancing an operation. Patent Document 1 discloses a technology for restoring the operation history of the display device at the time of editing when the display device is unexpectedly shut down. Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology has a problem in that in order to restore a desired display state by an Undo / Redo operation, the Undo / Redo operation must be performed as many times as the number of operations performed. That is, since one stroke is input in one operation on a display device, for example, to return handwritten data of "a" (a) which has three strokes to the state before input or to redisplay it, a total of three Undo / Redo operations are required.

[0005] Furthermore, when users use a display device online from multiple locations, the number of Undo / Redo operations required is expected to increase further, as users at each location will be able to write by hand at their own discretion.

[0006] In view of the above-mentioned problems, an object of the present invention is to provide a display device that can restore a desired display state. [Means for solving the problem]

[0007] The present invention is a display device that displays an object, comprising: a display state saving processing unit that saves display state data of the displayed object each time a condition is satisfied; an operation accepting unit that accepts a selection of the display state data; and a restoration processing unit that restores the display state of the object based on the display state data selected by the operation accepting unit. wherein, each time the condition is satisfied, the display state saving processing unit saves a linear list of past operation histories at the time of saving the display state data in association with the display state data, and saves a linear list of operation undo histories that have changed the display state of the object being displayed after the display state data is saved in association with the last saved display state data, and the restoration processing unit returns the restored display state of the object in accordance with the past operation history by the number of times an Undo operation has been accepted, and the restoration processing unit advances the restored display state of the object in accordance with the operation undo history by the number of times a Redo operation has been accepted. It is characterized by: [Effects of the Invention]

[0008] It is possible to provide a display device that can restore a desired display state. [Brief explanation of the drawings]

[0009] [Figure 1] 10A and 10B are diagrams illustrating an outline of an operation for restoring a display state of the display device. [Figure 2] 1 is an example of an overall configuration diagram of a communication system. [Figure 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a display device. [Figure 4] FIG. 1 is an example of a functional block diagram illustrating functions of a display device divided into blocks. [Figure 5] 3A and 3B are diagrams illustrating an example of a data structure of various data managed by the display device. [Figure 6] FIG. 10 is a diagram illustrating an example of a data structure of an object list. [Figure 7] 10A and 10B are diagrams illustrating an example of the data structure of an Undo list and a Redo list. [Figure 8] FIG. 10 is a diagram illustrating a display state list. [Figure 9] 9 is a diagram illustrating in detail the display state data shown in FIG. 8 that is saved at regular intervals or after a certain number of operations. [Figure 10] FIG. 10 is a diagram illustrating a Redo list after display state data is saved. [Figure 11] 10 is a flowchart illustrating an example of processing performed by a display state saving processing unit to save display state data at regular intervals. [Figure 12] 10 is a flowchart illustrating an example of processing performed by a display state saving processing unit to save display state data every time a certain number of operations are performed. [Figure 13] FIG. 10 is a diagram showing an example of a fixed time setting screen on which a user sets the saving frequency by time. [Figure 14] FIG. 10 is a diagram showing an example of an operation count setting screen on which a user sets the saving frequency by the number of operations. [Figure 15] 10A and 10B are diagrams illustrating an example of an object input by a user and a method for starting a time series editing mode. [Figure 16] FIG. 10 is a diagram showing an example of a window of another application displayed on the display. [Figure 17] FIG. 10 is a diagram illustrating a time series editing mode. [Figure 18] 10A and 10B are diagrams illustrating a method for selecting a thumbnail by a gesture operation. [Figure 19] 10A and 10B are diagrams illustrating a process in which a restoration processing unit restores a past display state to a current page using display state data. [Figure 20] 10A and 10B are diagrams illustrating a process in which a restoration processing unit restores a past display state to a new page using display state data. [Figure 21] 10A and 10B are diagrams illustrating a method for restoring a display state in units of one operation using a combination of restoring a display state at a certain point in the past and an Undo / Redo function. [Figure 22] FIG. 22 is a diagram illustrating a method for restoring the display state in units of one operation using a combination of restoring the display state at a certain point in the past and the Undo / Redo function, using a user interface different from that shown in FIG. 21. [Figure 23] FIG. 10 is a flowchart illustrating an example of a process performed by a restoration processing unit to restore a display state. [Figure 24]FIG. 10 is a flowchart illustrating an example of a process performed by a restoration processing unit to restore a display state. [Figure 25] FIG. 1 is a diagram illustrating an example of a schematic configuration of a display system. [Figure 26] FIG. 2 is an example of a functional block diagram of a display device and a server device in the display system. [Figure 27] 10 is an example of a sequence diagram in which the display system saves and restores the display state. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, as an example of an embodiment for carrying out the present invention, a display device and a display method performed by the display device will be described with reference to an embodiment.

[0011] <Outline of the video saving method of this embodiment> First, referring to FIG. 1, an outline of a method by which the display device 2 restores the display state of an object will be described. FIG. 1 is a diagram illustrating an outline of an operation by which the display device 2 of this embodiment restores the display state. FIG. 1(a) shows an example of a screen in time series restoration mode. A user who wants to return to the display state at a certain point in the past presses a button to switch to time series restoration mode. In time series restoration mode, multiple images representing display states at different times in the past are displayed in a slidable manner. These images are hereinafter referred to as thumbnails. When a user slides a thumbnail 101 left or right, a thumbnail 101 of a past or future display state is displayed depending on the sliding direction. The slidably displayed thumbnail 101 is, for example, an image of the display state of an object that is automatically saved at regular intervals.

[0012] When the user selects any thumbnail 101, as shown in Fig. 1(b), the display device 2 displays the object in a display state corresponding to the selected thumbnail 101. In other words, the display device 2 restores the display state at a certain point in time in the past.

[0013] As described above, the display device 2 of the present embodiment can restore a display state to a certain point in time with a small number of operations (at least one operation), whereas conventionally, it requires Undo / Redo depending on the number of operations to restore the display state. In other words, the display device 2 can improve user operability.

[0014] <Terminology> The input means may be any means that allows handwriting by specifying coordinates on a touch panel, such as an electronic pen, a human finger or hand, or a rod-shaped member.

[0015] A stroke is a series of actions in which a user presses an input tool against a display, moves it continuously, and then releases it from the display. A stroke can include tracking a user's movement without touching the display. In this case, the display device may initiate a stroke by a user gesture, for example, using a mouse or pointing device, pressing a button with the user's hand or foot, or in other ways. The user may then terminate the stroke with the same or a different gesture, releasing a button, or using the mouse or pointing device.

[0016] Stroke data is information displayed on a display based on a trajectory of coordinates input by an input means. Stroke data may be interpolated as appropriate. Handwriting data is data having one or more strokes. Handwriting input refers to handwriting data being input by a user. Handwriting input may be performed using a touch interface, a tactile object such as an electronic pen or stylus, or the user's body. Handwriting input may also be performed via other types of input, such as gesture-based input, hand tracking input, or other touch-free input by a user. Although embodiments of the present invention refer to handwriting input and handwriting input data, other forms of handwriting input may be used.

[0017] An object is a display object that is displayed on a display based on stroke data. Object means a target, but in this embodiment, it means a display target. Objects converted from handwritten stroke data may include text, as well as stamps displayed as fixed characters such as "Done," shapes such as circles and stars, and straight lines. Text is primarily a character string (character code) that includes one or more characters, and also includes numbers and symbols. Text may also be referred to as a character string.

[0018] A display state is the appearance of an object as it is displayed on the screen of a display or display device, and may include which objects are displayed, the position of the objects, and the state of the objects (stroke color, thickness, line type, etc.).

[0019] The display state data is data in which the display state is stored.

[0020] Restoring a display state means displaying an object in its original state. The original state is determined by the object list in the saved display state data. The original state may also be determined by a Redo list or an Undo list.

[0021] <System configuration example> Fig. 2 is a diagram showing the overall configuration of the communication system of this embodiment. Note that, for the sake of simplicity, Fig. 2 shows only two display devices 2a and 2b and accompanying electronic pens 4a and 4b, but three or more display devices 2 and electronic pens may be used.

[0022] 2, the communication system 1 includes a plurality of display devices 2a, 2b, a plurality of electronic pens 4a, 4b, USB memories 5a, 5b, notebook personal computers (PCs) 6a, 6b, video conference terminals 7a, 7b, and a PC 8. The display devices 2a, 2b, and the PC 8 are communicatively connected via a communication network 9. The display devices 2a, 2b are further provided with displays 3a, 3b, respectively.

[0023] Furthermore, the display device 2a can display an image drawn by an event generated by the electronic pen 4a (touching the display 3a with the tip or bottom of the electronic pen 4a). The image displayed on the display 3a can also be changed based on an event generated not only by the electronic pen 4a but also by the user's hand Ha or the like (gestures such as zooming in, zooming out, turning a page, etc.).

[0024] A USB memory 5a can be connected to the display device 2a, and the display device 2a can read electronic files such as PDFs from the USB memory 5a and record electronic files to the USB memory 5a. A laptop PC 6a is connected to the display device 2a via a cable 10a1 capable of communication according to standards such as DisplayPort (registered trademark), DVI (Digital Visual Interface), HDMI (High-Definition Multimedia Interface, registered trademark), and VGA (Video Graphics Array). The display device 2a generates an event when the display 3a is touched, and transmits event information indicating this event to the laptop PC 6a in the same manner as events from input devices such as a mouse and keyboard. A video (television) conference terminal 7a is also connected to the display device 2a via a cable 10a2 capable of communication according to the above standards. The laptop PC 6a and the video conference terminal 7a may communicate with the display device 2a via wireless communication conforming to various wireless communication protocols such as Bluetooth (registered trademark).

[0025] Meanwhile, at another location where display device 2b is installed, similarly to the above, display device 2b equipped with display 3b, electronic pen 4b, USB memory 5b, notebook PC 6b, video conference terminal 7b, cable 10b1, and cable 10b2 are used. Furthermore, the image displayed on display 3b can be changed based on an event caused by the user's hand Hb or the like.

[0026] As a result, an image drawn on the display 3a of the display device 2a at one location is also displayed on the display 3b of the display device 2b at another location, and conversely, an image drawn on the display 3b of the display device 2b at another location is displayed on the display 3a of the display device 2a at the one location. In this way, the communication system 1 is capable of performing remote sharing processing to share the same image at remote locations, making it very convenient when used for remote conferences, etc.

[0027] In the following, any one of the multiple display devices 2 will be referred to as "display device 2." Any one of the multiple displays will be referred to as "display 3." Any one of the multiple electronic pens will be referred to as "electronic pen 4." Any one of the multiple USB memory sticks will be referred to as "USB memory stick 5." Any one of the multiple notebook PCs will be referred to as "notebook PC 6." Any one of the multiple video conference terminals will be referred to as "video conference terminal 7." Any one of the multiple user's hands will be referred to as "hand H." Any one of the multiple cables will be referred to as "cable 10."

[0028] In addition, in this embodiment, an electronic whiteboard is described as an example of the display device 2, but the present invention is not limited thereto. Other examples of the display device 2 may include an electronic billboard (digital signage), a telestrator used for sports or weather forecasts, or a remote image (video) diagnostic device. In addition, a notebook PC 6 is described as an example of an external device, but the present invention is not limited thereto. Other examples of the external device may include a terminal capable of supplying image frames, such as a desktop PC, a tablet PC, a PDA, a digital video camera, a digital camera, or a game console. Furthermore, communication networks include the Internet, a LAN (Local Area Network), a mobile phone communication network, and the like. In addition, in this embodiment, a USB memory is described as an example of a recording medium, but the present invention is not limited thereto. Other examples of the recording medium may include various recording media such as an SD card.

[0029] <Hardware configuration of display device> Next, the hardware configuration of the display device 2 of this embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing the hardware configuration of the display device 2. As shown in Fig. 3, the display device 2 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an SSD (Solid State Drive) 204, a network I / F 205, and an external device connection I / F (Interface) 206.

[0030] Of these, the CPU 201 controls the overall operation of the display device 2. The ROM 202 stores the CPU 201 and programs used to drive the CPU 201, such as an IPL (Initial Program Loader). The RAM 203 is used as a work area for the CPU 201.

[0031] The SSD 204 stores various data such as an OS and programs for the display device 2. This program may be an application program that runs on an information processing device equipped with a general-purpose OS (Windows (registered trademark), Mac OS (registered trademark), Android (registered trademark), iOS (registered trademark), etc.). Therefore, although the display device 2 is normally used as a general-purpose information processing device, when a user executes an application program, the user can write by hand on the display device 2 just as if it were a dedicated device.

[0032] The network I / F 205 controls communication with the communication network 9. The external device connection I / F 206 is an interface for connecting various external devices. In this case, the external devices are, for example, a USB (Universal Serial Bus) memory 5 and external devices (a microphone 240, a speaker 250, and a camera 260).

[0033] The display device 2 also includes a capture device 211, a GPU 212, a display controller 213, a contact sensor 214, a sensor controller 215, an electronic pen controller 216, a short-range communication circuit 219, an antenna 219a of the short-range communication circuit 219, a power switch 222, and selection switches 223.

[0034] Of these, the capture device 211 displays video information as still images or moving images on the display of the external notebook PC 6. The GPU (Graphics Processing Unit) 212 is a semiconductor chip that specializes in graphics. The display controller 213 controls and manages the screen display to output the output image from the GPU 212 to the display 3, etc. The contact sensor 214 detects contact of the electronic pen 4, the user's hand H, etc. on the display 3. The sensor controller 215 controls the processing of the contact sensor 214. The contact sensor 214 inputs and detects coordinates using an infrared blocking method. This coordinate input and coordinate detection method involves two light receiving and emitting devices installed at both ends of the upper side of the display 3 emitting multiple infrared rays parallel to the display 3, which are reflected by reflecting members installed around the display 3 and return along the same optical path as the light emitted by the light receiving element. The contact sensor 214 outputs the ID of the infrared light emitted by the two light receiving and emitting devices that has been blocked by an object to the sensor controller 215, and the sensor controller 215 identifies the coordinate position of the contact position of the object. The electronic pen controller 216 determines whether the pen tip or the pen tail has touched the display 3 by communicating with the electronic pen 4. The short-range communication circuit 219 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The power switch 222 is a switch for switching the power of the display device 2 on and off. The selection switches 223 are a group of switches for adjusting, for example, the brightness and color of the display 3.

[0035] The display device 2 further includes a bus line 210. The bus line 210 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 201 shown in FIG.

[0036] The contact sensor 214 is not limited to an infrared blocking type, and may be a capacitive touch panel that identifies the contact position by detecting changes in capacitance. The contact sensor 214 may be a resistive film type touch panel that identifies the contact position by voltage changes across two opposing resistive films. The contact sensor 214 may be an electromagnetic induction type touch panel that identifies the contact position by detecting electromagnetic induction caused by an object touching the display unit, or various other detection means may be used. Furthermore, the electronic pen controller 216 may determine whether or not the part of the electronic pen 4 that the user grips or other parts of the electronic pen 4 are touched, in addition to the pen tip and pen butt.

[0037] <About the function> Fig. 4 is an example of a functional block diagram illustrating functions of the display device 2. The display device 2 shown in Fig. 4 includes a contact position detection unit 12, a drawing data generation unit 13, a conversion unit 14, a display control unit 15, a restoration processing unit 16, a network communication unit 17, a display state saving processing unit 18, an operation reception unit 19, and a storage unit 20.

[0038] Each function of the display device 2 is a function or means realized by any of the components shown in Figure 3 operating in accordance with instructions from the CPU 201 in accordance with an application deployed on the RAM 203.

[0039] The contact position detection unit 12 detects the coordinates of the position where the input means 291 (a finger, an electronic pen, etc.) touches the touch panel. If the number of phototransistors that cannot detect light (the width where light is blocked) is equal to or less than a predetermined number, the contact position detection unit 12 determines that the electronic pen 4 has made contact, and if the number is greater than the predetermined number, it determines that another object (for example, a finger) has made contact.

[0040] The drawing data generating unit 13 generates strokes by interpolating the sequence of coordinate points that are handwritten by the user and detected by the contact position detecting unit 12 .

[0041] The conversion unit 14 performs character recognition processing on the handwritten data to convert it into text, which includes letters, numbers, alphabets, symbols, and shapes such as triangles and squares.

[0042] The display control unit 15 controls the display of objects such as handwritten data, text, application windows, and PC capture images on the display 3. The display control unit 15 also displays menus and the like that are operated by the user.

[0043] The network communication unit 17 is connected to a network such as a LAN or the Internet, and transmits and receives data to and from other display devices 2 and PCs via the network.

[0044] The display state saving processing unit 18 saves the display state data for restoring the object being displayed individually in the storage unit 20 each time a condition is satisfied. The condition may be, for example, every certain time period or every certain number of operations, and the details will be described later.

[0045] The restoration processing unit 16 restores the display state at a certain time to the display 3 based on the display state data at that time stored in the storage unit 20. The specified intervals may be, for example, every fixed time or every fixed number of operations. The display state data may be saved by a user operation or may be saved irregularly.

[0046] The operation receiving unit 19 receives a user operation on the display device 2. For example, the operation receiving unit 19 receives a selection of a thumbnail 101 representing a display state.

[0047] <Data stored in the memory> Next, various types of data stored in the storage unit 20 will be described with reference to Figs. 5 to 7. Fig. 5 shows the data structure of various types of data managed by the display device 2. The display device 2 manages display state data for each conference. A conference refers to the period from when the display device 2 is turned on or returns from an energy saving mode to when it is turned off or goes into an energy saving mode. Instead of a division by conference, a period until a user designation may be considered as a division.

[0048] The various data stored in the storage unit 20 is managed by page. The data held by each page is as follows: A page is a single, planar display area. A page may be the same size as the display, or it may be larger or smaller than the page.

[0049] The object list 111 is a list in which object data is stored in chronological order for each object. The object data is data for displaying an object, including the object type (stroke, text, stamp, etc.), object size (width, height), and object position information (top, left, right, bottom), etc.

[0050] The Undo list 112 is a list in which operation histories are saved in chronological order.

[0051] The Redo list 113 is a list in which the operation cancellation history is stored in chronological order.

[0052] The object list 111, the Undo list 112, and the Redo list 113 are referred to as a current display state 115. With the object list 111, the display device 2 can restore the display state at a certain point in time.

[0053] The display state list 114 is a list in which display state data, which will be described later, is stored in chronological order. Details will be explained in FIG.

[0054] FIG. 6 shows the data structure of the object list 111. The object list 111 is a list of object data 121 to 124 included in a page. As shown in FIG. 6, the list is a linear list. A linear list is a list in which elements called nodes, each containing data and a pointer, are connected by pointers. Therefore, the order in which the objects were input is also recorded. The object data has the following items. ·id is the object's identification information. type is the type of object. The position is the position information of the object. For example, the position information is the coordinates of one of the four vertices of the bounding rectangle of the object. · size is the size (height and width) of the object.

[0055] 7 shows the data structures of the Undo list 112 and the Redo list 113. The Undo list 112 is a history of operations 131 to 134 included in a page. The Redo list 113 is a history of operations undone included in a page, but has the same data structure as the Undo list 112. As shown in FIG. 7, the Undo list 112 and the Redo list 113 are linear lists. The Undo list 112 and the Redo list 113 have the following items. ·operate is the operation content. ·Target_id is the id of the operated object.

[0056] By referring to and manipulating this information, the display device 2 stores the display state of the display device 2 and also realizes Undo / Redo operations. For example, the operation content of operation history 134 in Fig. 7 is the deletion of the object with Target_id = 5, so when the user performs an Undo operation, the display device 2 displays the deleted object with Target_id = 5 again. The object with Target_id = 5 is included in object list 111. Operation history 134 is deleted from Undo list 112, and operation history 134 is added to Redo list 113 as an operation cancellation history.

[0057] 7 is the addition of an object with Target_id = 5, so when the user performs an Undo operation, the display device 2 hides the added object with Target_id = 5. The operation history 133 is deleted from the Undo list 112, and is added to the Redo list 113 as an operation cancellation history.

[0058] In this state, if the user performs a Redo operation, the display device 2 re-executes the add of the object with Target_id = 5 based on the most recent operation undo history (operation history 133) in the Redo list 113, and displays the object with Target_id = 5. The operation undo history (operation history 133) is deleted from the Redo list 113, and the operation history 133 is added to the Undo list 112.

[0059] <Storage of display status data> Next, the display state list 114 will be described with reference to Fig. 8. Fig. 8 is a diagram illustrating the display state list 114. The display state list 114 is a list of display state data 141 to 143 that stores display state data at regular intervals or after a certain number of operations. The display state data is data for restoring a display state at a certain point in time. Each of the display state data 141 to 143 has the following items. The time is the time when the display status data was saved. The time can be absolute time or the time elapsed since the power was turned on or the device returned from power saving mode.

[0060] ·objectlist is the object list 111 at a certain point in time (at the above time).

[0061] The undolist is the Undo list 112 at a certain point in time (at the above time). By including the Undo list 112 in the display state data, it becomes possible to perform a Redo operation on a per-operation basis after restoring the display state at a certain point in time. ·redolist is the Redo list 113 at a certain point in time (at the above time). By including the Redo list 113 in the display state data, it becomes possible to perform a Redo operation in units of one operation after restoring the display state at a certain point in time.

[0062] The display state save processing unit 18 uses a timer to add the current display state data at regular intervals to the display state list 114. In the case of every certain number of operations, the display state save processing unit 18 uses a counter that stores the number of operations, and adds the current display state data to the display state list 114 every time the corresponding number of operations is exceeded (at this time, the counter is also initialized).

[0063] In the object list 111, past objects are saved in chronological order, so the display state data at time 10 includes overlapping display state data at time 5. The display state data at time 15 includes overlapping display state data at time 5 and time 10. The same is true for the undo list 112. In the redo list 113, the operations performed after the display state data was saved are saved as an operation undo history. Details will be explained in FIG. 10.

[0064] The operations that are the target of the counter are those that cause changes to the object list 111, such as adding, moving, or deleting a stroke.

[0065] <Image of display status data saved> Fig. 9 is a diagram illustrating in detail the display state data saved at regular intervals or after a certain number of operations, as shown in Fig. 8. Fig. 9(a) is an example of an object display at a certain point in time. The user has handwritten "aiu."

[0066] FIG. 9(b) shows the object list 111, the undo list 112, and the redo list 113 (current display state 115) at the time when the user handwrote "aiu." The undo list 112 and the redo list 113 are associated and saved in the display state data. As described in FIG. 8, the object list 111, the undo list 112, and the redo list 113 represent the current display state 115. An operation history is added to the undo list 112 for each operation by the user. The redo list 113 is generated when the user performs an undo operation, and the operation history of the undo operation is moved from the undo list 112 to the redo list 113. In FIG. 9(b), the redo list 113 is empty, as it is assumed that the user has not performed an undo operation.

[0067] The display state saving processing unit 18 saves display state data at regular intervals or after a certain number of operations. The display state data is an object list 111, an Undo list 112, and a Redo list 113 at a certain point in time. FIG. 9(c) shows an example of a saved display state list 114. As shown in FIG. 9(c), the object list 111, the Undo list 112, and the Redo list 113 at a certain point in time in FIG. 9(b) are saved as a single display state data 116. Therefore, the object list 111, the Undo list 112, and the Redo list 113 are included in a single display state data 116. Note that the time is omitted.

[0068] Next, updating of the Redo list 113 after saving the display state data will be described with reference to Fig. 10. Fig. 10 is a diagram illustrating the Redo list 113 after saving the display state data 116. If the user edits an object (adding, moving, deleting, etc. an object) after the display state saving processing unit 18 saves the display state data 116, the operation history of these edits is added to the Redo list 113 of the latest display state data 116.

[0069] As shown in FIG. 10(a), the user handwrote handwritten data 109 of "e" following "aiu." "e" has two strokes. FIG. 10(b) shows the object list 111, the undo list 112, and the redo list 113 at the time when "e" was added. As shown in FIG. 10(b), object data 117 corresponding to the added "e" has been added to the object list 111, and the operation history 118 corresponding to the added "e" has been added to the undo list 112. The redo list 113 may be empty.

[0070] On the other hand, as shown in Fig. 10(c), an operation undo history 119 corresponding to the added "e" has been added to the saved display state data 116. No object data is added to the object list 111 of the display state data 116, and no operation history is added to the Undo list 112. By adding the operation history only to the Undo list 112, if the user selects to return to the display state of Fig. 9(a) for which the display state data is saved, the user can further undo an operation with an Undo operation (for example, to delete "u") or proceed with an operation with a Redo operation (for example, to display "e").

[0071] <Saving display status data> 11 is a flowchart showing the process of saving display state data at regular intervals by the display state saving processor 18. The process in FIG. 11 is executed at regular intervals when a timer times out.

[0072] The display state save processing unit 18 determines whether there has been a change in the display state (a change to the object list 111) since the display state data was last saved (S1). One method of determination is for the display state save processing unit 18 to clear a flag each time the display state data is saved, and to turn on the flag when the user changes the display state.

[0073] If the display state has changed (Yes in S1), the display state save processing unit 18 adds the current object list 111, Undo list 112, and Redo list 113 to the display state list 114 as display state data 116 (S2).

[0074] Next, after saving the display state data 116, the display state saving processing unit 18 adds an operation cancellation history to the Redo list 113 of the last saved display state data each time the user changes the display state (S3).

[0075] 12 is a flowchart showing the process of saving display state data every time a certain number of operations are performed by the display state saving processor 18. The process of FIG. 12 is executed every time the user changes the display state.

[0076] The display state save processing unit 18 increments the counter by one (S11).

[0077] The display state save processing unit 18 determines whether the count is equal to or greater than a certain number of times (S12). If the count is not equal to or greater than the certain number of times (No in S12), the processing in FIG.

[0078] If the count is equal to or greater than a certain number of times (Yes in S12), the display state save processing unit 18 adds the current object list 111, Undo list 112, and Redo list 113 to the display state list 114 as display state data 116 (S13). Note that Fig. 12 is executed every time the user changes the display state, so changes have occurred to the last saved display state data 116.

[0079] Moreover, the display state save processing unit 18 initializes a counter (S14).

[0080] Next, after saving the display state data 116, the display state save processing unit 18 adds an operation cancellation history to the Redo list 113 of the last saved display state data 116 each time the user changes the display state (S15).

[0081] <Example of setting a certain number of operations for a certain period of time> The user may be able to set the interval or number of operations at which the display state is saved. Increasing the saving frequency makes it easier to save more display state data, but the more frequently the saving is done, the more memory is used and the smaller the difference between the Undo / Redo operations becomes.

[0082] 13 shows a fixed time setting screen 170 on which the user sets the saving frequency in terms of time. Fixed time setting screen 170 has a message 171 saying "You can set the saving time interval," a time input field 172, an OK button 173, and a cancel button 174. The time input field 172 is, for example, a pull-down menu from which the user can select the desired saving frequency.

[0083] 14 shows an operation count setting screen 180 on which the user sets the save frequency by the number of operations. The operation count setting screen 180 has a message 181 saying "You can set the number of save operations," an operation count input field 182, an OK button 183, and a cancel button 184. The operation count input field 182 is, for example, a pull-down menu, and the user can select the desired save frequency from the pull-down menu.

[0084] <Restore the display state at a certain point in time by operating the slider> Next, the process for restoring the display state to a certain point in time will be described with reference to Figs. 15 to 17. Fig. 15 is a diagram illustrating an example of an object input by a user and a method for starting the time-series editing mode. Figs. 15(a) to 15(d) show an example of an object input by a user. In Figs. 15(a) to 15(d), the user inputs handwritten data 151, displays a captured image 152, displays a stamp 153, and inputs handwritten data 154.

[0085] The display device 2 also has the function of simultaneously executing multiple applications. The functions of the display device 2 described in this embodiment are realized by one application (hereinafter simply referred to as an app). In addition, the display device 2 can also execute a web browser, a remote conference app, and the like. When multiple apps are executed, the windows of the other apps are displayed on the display.

[0086] FIG. 16 shows a window 160 of another application displayed on the display 3 of the display device 2. In FIG. 16, an object 161 and a window 160 are displayed on the display 3. The display state save processing unit 18 can include the window 160 (image data) and display position (coordinates) of the other application in the information saved in the display state data. Therefore, when restoring the display state, the display device 2 can restore the window 160 of the other application. The display state save processing unit 18 can acquire the window of the other application from the OS.

[0087] Returning to Fig. 15, the user presses the time series editing mode button 150 as shown in Fig. 15(e) in order to return to the display state at a certain point in the past. The display device 2 transitions to the time series editing mode as shown in Fig. 17. Note that Figs. 17(a) and (b) are almost the same as Fig. 1, but are shown again for the sake of explanation.

[0088] 17(a) shows an example screen in the time series restoration mode. In the time series restoration mode, thumbnails 101 of display states at different points in the past are displayed in a slidable manner along a straight line. The thumbnails 101 are reduced images of display states restored based on the object list 111 of the display state data 116. When the user touches the thumbnail 101 with a finger or the electronic pen 4 and slides the thumbnail 101 left or right, a thumbnail 101 of a past or future display state is displayed depending on the sliding direction. When the user presses the thumbnail 101 with a finger or the electronic pen 4, the display state corresponding to the pressed thumbnail 101 is selected.

[0089] When the user selects an arbitrary display state, the display device 2 displays the object that was displayed in the selected display state, as shown in Fig. 17(b). This object is obtained by the restoration processing unit 16 from the object list 111 of the display state data corresponding to the selected display state and displayed on the screen. Details of the restoration processing will be explained with reference to Figs. 19 and 20.

[0090] The restored display state is displayed not as image data but as an object state. If the object is a stroke, the object state means that it is displayed based on a sequence of coordinate points. In Figure 17(c), this is utilized to allow the user to move object 163 by swiping with a finger.

[0091] In this way, when the user selects display state data and the display device 2 restores the display state, the original object is displayed rather than as image data, allowing the user to re-edit the object (move, delete, etc.) and undo / redo a single operation.

[0092] 10, the display state data stores an Undo list 112 and a Redo list 113, so the user can undo or redo for each operation. That is, the user can restore the display state at regular time intervals or for a certain number of operations, and then undo or redo each operation using the conventional Undo and Redo operations.

[0093] Furthermore, as shown in FIG. 18, the user can also slide the display state left and right using a gesture operation. FIG. 18 is a diagram illustrating a method for selecting a thumbnail using a gesture operation. A gesture operation refers to a process in which an operation content is assigned according to the number of fingers that the user uses to touch the screen, and the display device 2 accepts an operation according to the number of fingers touching the screen. In the explanation of FIG. 18, an operation using three fingers is an operation for executing the time series restoration mode. Note that the number of fingers used in the gesture operation may be one, two, four or more.

[0094] Therefore, when the user touches any location on the screen with three fingers, the display device 2 transitions to a time-series editing mode. The display device 2 displays thumbnails 101 of display states at different points in time in the past in a linearly slidable manner. When the user slides the thumbnails 101 left or right with three fingers, thumbnails 101 corresponding to past or future display states are displayed depending on the sliding direction. When the user presses a thumbnail 101 with a finger or the electronic pen 4, the display state corresponding to the pressed thumbnail 101 is selected.

[0095] In this way, with gesture operations, the user does not need to press the time series editing mode button 150 to execute the time series restoration mode. Instead of selecting a thumbnail 101 by pressing it with a finger or the electronic pen 4, the thumbnail 101 that was pressed down when the user releases the finger or the electronic pen 4 from the thumbnail 101 may be selected.

[0096] When the user is using the electronic pen 4, the display device 2 can be shifted to the time series restoration mode by touching the butt end of the pen, pressing and holding the tip of the pen, or pressing a button provided on the electronic pen 4.

[0097] <Display state restoration process> 19 is a diagram illustrating the process in which the restoration processing unit 16 restores a past display state to the current page using display state data. Fig. 19 shows an example of restoring a display state at a certain point in time in the past on the current page. The display device 2 stores the current display state 115 (object list 111, Undo list 112, and Redo list 113) in a fixed address (memory area) in the RAM 203.

[0098] In the time-series restoration mode, the user slides the thumbnail 101 left or right to select the display state to be restored, which allows the restoration processing unit 16 to identify the display state data in the display state list 114.

[0099] The restoration processing unit 16 overwrites the currently displayed object list 111, Undo list 112, and Redo list 113 with the specified display state data. That is, the restoration processing unit 16 overwrites the object list 111 of the current display state with the object list 111 of the selected display state data, overwrites the Undo list 112 of the current display state with the Undo list 112 of the selected display state, and overwrites the Redo list 113 of the current display state with the Redo list 113 of the selected display state.

[0100] In this way, the restoration processing unit 16 can restore the previous display state while any page is being displayed.

[0101] Furthermore, as shown in FIG. 20, the restoration processing unit 16 can restore a display state at a certain point in the past to a new page. FIG. 20 is a diagram illustrating the process in which the restoration processing unit 16 uses display state data to restore a past display state to a new page. In the time-series restoration mode, the user slides the display state left or right to select the display state to be restored. This allows the restoration processing unit 16 to identify the display state data 116 in the display state list 114.

[0102] The restoration processing unit 16 copies the identified display state data to the display state of the new page. If a display state area can be secured in RAM 203 for each page, the restoration processing unit 16 simply copies the display state data to an address corresponding to the page. If only the capacity of the current page can be secured in RAM 203, the restoration processing unit 16 simply copies the display state data to an address corresponding to the current page (in this case, the process is equivalent to that of FIG. 19).

[0103] When restoring to a new page, the restoration processing unit 16 copies the object list 111 of the display state data to the object list 111 of the display state of the new page, copies the Undo list 112 of the selected display state data to the Undo list 112 of the display state of the new page, and copies the Redo list 113 of the selected display state data to the Redo list 113 of the display state of the new page.

[0104] In this way, the restoration processing unit 16 can restore the previous display state to the new page.

[0105] <Combination of restoring the display state at a certain point in the past and the Undo / Redo function> The display state restoration method of this embodiment can restore the display state to a certain point in time in the past with fewer operations than conventional Undo / Redo operations, but it is difficult to restore the display state in units of one operation. However, if the user uses the display state restoration method of this embodiment and also uses the Undo / Redo function, the display state can be restored in units of one operation.

[0106] Figure 21 is a diagram explaining a method for restoring the display state for one operation unit using a combination of restoring the display state at a certain point in the past and the Undo / Redo function. The user wants to return the current display state A in Figure 21(a) to display state B, which is "aiueokakikuke."

[0107] 21(b), the user slides the display state thumbnail 101 left and right in the time-series restoration mode to select the display state C that is closest to the display state B (not necessarily the closest). The display device 2 also displays an Undo button 191 and a Redo button 192.

[0108] 21(c), the display state C selected by the user is "aiueokakikukeko." Compared to display state B, display state C has an extra "ko."

[0109] Therefore, the user uses the Undo function to return to the display state before "ko" was input, as shown in Figure 21(d). In other words, since the number of operations on the stroke data for "ko" is two, the user presses the Undo button 191 twice.

[0110] This allows the user to return from display state A to display state B with far fewer operations than if the user were to return from display state A to display state B using only the Undo function.

[0111] FIG. 22 is a diagram illustrating a method for restoring the display state in units of one operation using a combination of restoring the display state at a certain point in the past and the Undo / Redo function, using a user interface different from that shown in FIG.

[0112] The user wants to return the current display state A in FIG. 22(a) to display state B, which is "aiueokakikuke."

[0113] 22(b)(c), in the time-series restoration mode, the user slides the display state thumbnail 101 left and right using a gesture operation to select display state C that is closest to display state B (not necessarily the closest). The user presses display state C with his finger.

[0114] When the user stops sliding left and right, as shown in FIG. 22(d), thumbnails 102 (an example of a second display state) are displayed vertically above and below the thumbnail 101 in the display state selected by the user. The vertical thumbnails 102 scroll vertically. Starting from the display state selected by the user, the vertical thumbnails 102 indicate a display state one operation earlier as they move upward, and indicate a display state one operation later as they move downward. In other words, the display state in the upward direction is the display state when N Undo operations are performed on the display state selected by sliding left and right, and the display state in the downward direction is the display state when N Redo operations are performed on the display state selected by sliding left and right. N times is the number of thumbnails 102 starting from the selected thumbnail 101.

[0115] As shown in FIGS. 22(e) and 22(f), the user can undo / redo one operation at a time by vertically scrolling the display.

[0116] As shown in Fig. 22(g), when the user presses a thumbnail 102 on the touch panel with his / her finger, the restoration processing unit 16 restores the display state corresponding to the thumbnail 102 pressed by the user on the display 3. In this way, with a user interface such as that shown in Fig. 22, the display state can be restored in units of one operation by the user changing the sliding direction. Instead of selecting a thumbnail 102 by pressing it with his / her finger or electronic pen 4, the thumbnail 102 that was pressed when the user releases his / her finger or electronic pen 4 from the thumbnail 102 may be selected.

[0117] 22, the display state for a certain period of time or for a certain number of operations is displayed by sliding left and right, and the display state for one operation unit is displayed by sliding up and down, but the sliding direction of left and right and up and down may be reversed. Also, the display control unit 15 may display the thumbnail 102 corresponding to the Redo operation above the thumbnail 101, and the thumbnail 102 corresponding to the Undo operation below the thumbnail 101.

[0118] <Restore process procedure or operation> Fig. 23 is a flowchart illustrating the process of restoring the display state by the restoration processing unit 16. Fig. 23 illustrates a case where the display state is restored in units of one button operation, as described in Fig. 21.

[0119] First, in response to pressing of the time-series editing mode button 150, the restoration processing unit 16 restores the display state based on the display state data, and the display control unit 15 displays the thumbnails 101 in the display state in a slidable manner (S21). The user slides the thumbnails 101 by moving the finger or electronic pen 4 touching the display 3 left and right.

[0120] The restoration processing unit 16 determines whether the operation receiving unit 19 has received the selection of the thumbnail 101 by a finger or the electronic pen 4 (S22).

[0121] If a thumbnail 101 is selected (Yes in S22), the restoration processing unit 16 overwrites the currently displayed object list 111, Undo list 112, and Redo list 113 with the display state data corresponding to the selected thumbnail 101 (S23).

[0122] Next, the operation receiving unit 19 determines whether the Undo button or the Redo button has been pressed (S24).

[0123] If the Undo button or Redo button is pressed (Yes in S24), the restoration processing unit 16 reverses or advances one operation starting from the display state data in step S23 (S25).

[0124] With the above steps, the user can restore the display state to a certain point in time in the past, and restore the display state in units of one operation.

[0125] Fig. 24 is a flowchart illustrating the process of restoring the display state by the restoration processing unit 16. Fig. 24 illustrates a case where the display state is restored by left / right and up / down sliding operations using gesture operations as described in Fig. 22.

[0126] First, the restoration processing unit 16 restores the display state based on the display state data 116 in response to the gesture operation, and the display control unit 15 displays the thumbnails 101 in the display state in a slidable manner (S31).

[0127] The restoration processing unit 16 determines whether the sliding (gesture operation) in the left-right direction has stopped (S32).

[0128] When the sliding in the left-right direction stops (Yes in S32), the restoration processing unit 16 displays the display state returned by one operation and the display state forward by one operation vertically, starting from (centered on) the display state corresponding to the thumbnail 101 when the three fingers were released (S33).

[0129] That is, the restoration processing unit 16 restores the display state, which is restored by one operation back according to the operation history in the Undo list 112 included in the display state data corresponding to the selected thumbnail 101, for the number of operation histories. The display control unit 15 arranges the thumbnails 102 in the restored display state in chronological order above the thumbnail 101 selected by sliding horizontally. Similarly, the restoration processing unit 16 creates display states, which are advanced by one operation according to the operation undo history in the Redo list 113 included in the selected display state data, for the number of operation undo histories. The display control unit 15 arranges the thumbnails 102 in the restored display state in chronological order below the thumbnail 101 selected by sliding horizontally.

[0130] Next, the restoration processing unit 16 determines whether the operation receiving unit 19 has received the selection of the thumbnail 102 by the finger or the electronic pen 4 (S34).

[0131] If a thumbnail 102 is selected (Yes in S34), the restoration processing unit 16 overwrites the currently displayed object list 111, Undo list 112, and Redo list 113 with the display state data (one operation back or forward) corresponding to the selected thumbnail 102 (S35).

[0132] Now, the user can restore the display state at a certain point in time using gesture operations.

[0133] <Application to server-client systems> The functions of the display device 2 can also be realized in the form of a server-client system as shown in Fig. 25. Fig. 25 is an example of a schematic configuration diagram of a display system 310. The display device 2 and a server device 300 are connected via a network such as the Internet. The hardware block diagram of the server device 300 may be similar to that of a general-purpose information processing device or computer such as a PC.

[0134] 26 is a functional block diagram of the display device 2 and the server device 300 in the display system 310. In the case of the display system 310, the display device 2 has the contact position detection unit 12, the drawing data generation unit 13, the display control unit 15, the network communication unit 17a, and the operation reception unit 19 shown in FIG. 4. The storage unit 20a of the display device 2 only needs to store an object list 111 indicating the display state of the current page, an undo list 112, and a redo list 113. The data in the storage unit 20a of the display device 2 may be synchronized with the storage unit 20b of the server device 300 so as to be in the same state.

[0135] On the other hand, the server device 300 has a conversion unit 14, a restoration processing unit 16, a display state saving processing unit 18, and a network communication unit 17b (an example of a second communication unit). An object list 111, an Undo list 112, a Redo list 113, and a display state list 114 for each page are saved in a storage unit 20b of the server device 300.

[0136] The network communication unit 17a (an example of a first communication unit) of the display device 2 transmits to the server device 300 the operation content for each operation that changes the object list 111. The server device 300 performs the same processing as the flowcharts in Figures 11, 12, 23, and 24, saves the display state data, and performs restoration processing.

[0137] FIG. 27 is an example of a sequence diagram showing how the display system 310 saves and restores the display state.

[0138] S101: The user performs an operation to change the display state of an object on the display device 2. The operation acceptance unit 19 accepts the operation.

[0139] S102: The network communication unit 17a of the display device 2 transmits to the server device 300 the operation content of one operation unit related to the object.

[0140] S103: The network communication unit 17b of the server device 300 receives the operation content for one operation unit, and the display state saving processing unit 18 performs the process of FIG. 11 or FIG. 12 and saves the display state data.

[0141] Note that steps S101 to S103 are repeatedly executed.

[0142] S104: Next, the user who wants to restore the display state at a certain point in the past performs an operation to switch to the time-series editing mode. This operation may be performed by pressing the time-series editing mode button 150 or by a gesture operation.

[0143] S105: The operation accepting unit 19 of the display device 2 accepts the operation, and the network communication unit 17a transmits to the server device 300 a request to switch to the time-series editing mode.

[0144] S106: The network communication unit 17b of the server device 300 receives the request to switch to the time-series editing mode, and the restoration processing unit 16 restores the display state based on the saved display state data 116.

[0145] S107: The network communication unit 17b of the server device 300 transmits the thumbnails 101 of each display state to the display device 2. The network communication unit 17a of the display device 2 receives the thumbnails 101 of each display state, and the display control unit 15 displays the thumbnails 101 in a slidable manner in the left-right direction.

[0146] S108: The user selects an arbitrary thumbnail 101.

[0147] S109: The operation accepting unit 19 of the display device 2 accepts the selection, and the network communication unit 17a transmits a restoration request to the server device 300, specifying the identification information (for example, the page number) of the display state data.

[0148] S110: The network communication unit 17b of the server device 300 receives the restoration request, and the network communication unit 17b transmits to the display device 2 the display state data identified by the identification information.

[0149] S111: The network communication unit 17a of the display device 2 receives the display state data, and overwrites the currently displayed object list 111, Undo list 112, and Redo list 113 with the received display state data.

[0150] Thereafter, the user can perform an Undo / Redo operation for one operation unit by pressing the Undo button or the Redo button. Since the display device 2 has already received the Undo list 112 and the Redo list 113, the display control unit 15 reverses or advances the display state of the object displayed in step S111 by one operation in response to pressing the Undo button or the Redo button.

[0151] 22, the server device 300 transmits the thumbnail 102 along with the thumbnail 101 in step S107. The display device 2 displays the user interface as shown in FIG. 22 and requests the server device 300 for display state data corresponding to the thumbnail 102 selected by the user. Alternatively, the server device 300 may transmit each piece of display state data along with the thumbnail 101 in step S107. The display device 2 uses the display state data to display the user interface as shown in FIG. 22 and displays the display state data corresponding to the thumbnail 102 selected by the user.

[0152] In this way, the display system 310 allows the display device 2 and the server device 300 to interactively display objects. In other words, the user can use a general-purpose information processing device with a touch panel to utilize the same functions as the dedicated display device 2, without using a dedicated display device 2.

[0153] In addition, since the object list 111, the Undo list 112, the Redo list 113, and the display state list 114 are stored in the server device 300, a display device 2 or a PC in a remote location can connect to the server device 300 and share the display state in real time.

[0154] <Major Effects> As described above, the display device 2 of the present embodiment can restore a display state to a certain point in time with a small number of operations (at least one operation), whereas conventionally, it requires Undo / Redo depending on the number of operations to restore the display state. In other words, the display device 2 can improve user operability.

[0155] <Other application examples> The best mode for carrying out the present invention has been described above using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0156] For example, in the present embodiment, an example has been described in which the display state of stroke data is saved, but the object may also be text. In the case of text, one operation corresponds to one character or multiple characters converted at once.

[0157] Furthermore, although an electronic whiteboard has been described as an example in the present embodiment, the electronic whiteboard may also be called an electronic whiteboard, an electronic information board, or the like. Furthermore, the present embodiment can be suitably applied to any information processing device having a touch panel. Examples of information processing devices equipped with a touch panel include output devices such as PJs (Projectors), digital signage, HUDs (Head Up Display) devices, industrial machinery, imaging devices, sound collection devices, medical equipment, network appliances, notebook PCs, mobile phones, smartphones, tablet terminals, game consoles, PDAs (Personal Digital Assistants), digital cameras, wearable PCs, and desktop PCs.

[0158] In this embodiment, the coordinates of the electronic pen 4 are detected by detecting the coordinates of the pen tip using a touch panel, but the display device 2 may also detect the coordinates of the pen tip using ultrasonic waves. The electronic pen 4 emits ultrasonic waves along with light emission, and the display device 2 calculates the distance based on the arrival time of the ultrasonic waves. The display device 2 can identify the position of the electronic pen 4 based on the direction and distance, and the projector draws (projects) the trajectory of the electronic pen 4 as stroke data.

[0159] In addition, the configuration examples in Figure 4 and the like are divided according to main functions to make it easier to understand the processing by the display device 2. The method of dividing the processing units and their names do not limit the present invention. The processing by the display device 2 can be divided into even more processing units depending on the processing content. Also, it can be divided so that one processing unit includes even more processes.

[0160] Furthermore, each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions.

[0161] Embodiments of the present invention provide significant improvements in computer power and functionality. These improvements allow users to utilize computers that provide more efficient and robust interaction with tables, which are ways of storing and presenting information in information processing devices. Furthermore, embodiments of the present invention provide a better user experience through the use of more efficient, powerful, and robust user interfaces. Such user interfaces provide better interaction between humans and machines.

[0162] <Appendix to the Claims> [Claim 1] A display device for displaying an object, a display state saving processing unit that saves display state data of the object being displayed each time a condition is satisfied; an operation receiving unit that receives a selection of the display state data; a restoration processing unit that restores a display state of the object based on the display state data selected by the operation receiving unit; A display device comprising: [Claim 2] The display device according to claim 1, characterized in that, after saving the display state data, the display state saving processing unit saves the display state data only if the condition is satisfied and there is a change in the display state of the displayed object. [Claim 3] The display device according to claim 1, characterized in that the display state saving processing unit saves a linear list of past operation history at the time the display state data is saved, in association with the display state data, each time the condition is satisfied. [Claim 4] The display device according to claim 3, characterized in that the display state saving processing unit saves a linear list of operation undo history that changed the display state of the object displayed after the display state data was saved, in association with the last saved display state data. [Claim 5] a display control unit that displays a plurality of images representing the display state of the object restored by the restoration processing unit in a straight line; the display control unit slides the plurality of images in a direction in which the input unit operates the touch panel; 5. The display device according to claim 4, wherein the restoration processing unit restores, on the display, a display state of the object corresponding to the image selected by the input means. [Claim 6] the restoration processing unit restores the display state of the restored object according to the past operation history by the number of times that Undo operations have been accepted; 6. The display device according to claim 5, wherein the restoration processing unit advances the restored display state of the object by the number of times a Redo operation has been accepted, in accordance with the operation cancellation history. [Claim 7] a display control unit that displays a plurality of images representing the display state of the object restored by the restoration processing unit in a straight line; the display control unit slides the plurality of images in a direction in which the input unit operates the touch panel; when the input means is stopped, the display control unit displays, above the image touched by the input means, an image in a second display state returned in accordance with the past operation history, starting from the display state corresponding to the image; The display device according to claim 4, characterized in that an image of a second display state, which is advanced from the display state corresponding to the image touched by the input means according to the operation cancellation history, is displayed below the image touched by the input means. [Claim 8] 8. The display device according to claim 1, wherein the condition is that a certain period of time elapses, and the certain period of time is accepted. [Claim 9] 8. The display device according to claim 1, wherein the condition is that a certain number of operations are performed, and the display device accepts the certain number of operations. [Claim 10] 8. The display device according to claim 5, wherein the display control unit displays the display state of the object restored by the restoration processing unit on a new page. [Claim 11] an application that displays a window is running on the display device; 11. The display device according to claim 1, wherein the display state saving processing unit includes a screen of a window of the application in the display state data. [Explanation of symbols]

[0163] 2 Display device [Prior art documents] [Patent documents]

[0164] [Patent Document 1] Patent Publication No. 2021-022041

Claims

1. A display device for displaying an object, a display state saving processing unit that saves display state data of the object being displayed each time a condition is satisfied; an operation receiving unit that receives a selection of the display state data; a restoration processing unit that restores a display state of the object based on the display state data selected by the operation receiving unit, the display state saving processing unit saves a linear list of past operation histories at the time when the display state data is saved, in association with the display state data, each time the condition is satisfied; storing a linear list of operation undo histories that changed the display state of the object displayed after the display state data was saved, in association with the display state data that was last saved; the restoration processing unit restores the display state of the restored object according to the past operation history by the number of times that Undo operations have been accepted; The display device, wherein the restoration processing unit advances the restored display state of the object by the number of times a Redo operation has been accepted, in accordance with the operation cancellation history.

2. a display control unit that displays a plurality of images representing the display state of the object restored by the restoration processing unit in a straight line; the display control unit slides the plurality of images in a direction in which the input unit operates the touch panel; 2. The display device according to claim 1, wherein the restoration processing unit restores, on the display, a display state of the object corresponding to the image selected by the input means.

3. A display device for displaying an object, a display state saving processing unit that saves display state data of the object being displayed each time a condition is satisfied; an operation receiving unit that receives a selection of the display state data; a restoration processing unit that restores a display state of the object based on the display state data selected by the operation receiving unit; a display control unit that displays a plurality of images representing the display state of the object restored by the restoration processing unit in a straight line, the display state saving processing unit saves a linear list of past operation histories at the time when the display state data is saved, in association with the display state data, each time the condition is satisfied; storing a linear list of operation undo histories that changed the display state of the object displayed after the display state data was saved, in association with the display state data that was last saved; the display control unit slides the plurality of images in a direction in which the input unit operates the touch panel; when the input means is stopped, the display control unit displays, above the image touched by the input means, an image in a second display state returned in accordance with the past operation history, starting from the display state corresponding to the image; A display device characterized in that an image of a second display state, which is advanced according to the operation cancellation history starting from the display state corresponding to the image, is displayed below the image touched by the input means.

4. 4. The display device according to claim 2, wherein the display state saving processing unit saves the display state data only if, after saving the display state data, the condition is satisfied and there is a change in the display state of the displayed object.

5. 4. The display device according to claim 2, wherein the condition is that a certain period of time elapses, and the certain period of time is accepted.

6. 4. The display device according to claim 2, wherein the condition is a certain number of operations, and the certain number of operations is accepted.

7. 4. The display device according to claim 2, wherein the display control unit displays the display state of the object restored by the restoration processing unit on a new page.

8. an application that displays a window is running on the display device; 4. The display device according to claim 2, wherein the display state saving processing section includes a screen of the window of the application in the display state data.

9. A display device for displaying the object, a display state saving processing unit that saves display state data of the object being displayed each time a condition is satisfied; an operation receiving unit that receives a selection of the display state data; a restoration processing unit that restores the display state of the object based on the display state data selected by the operation receiving unit; the display state saving processing unit saves a linear list of past operation histories at the time when the display state data is saved, in association with the display state data, each time the condition is satisfied; storing a linear list of operation undo histories that changed the display state of the object displayed after the display state data was saved, in association with the display state data that was last saved; the restoration processing unit restores the display state of the restored object according to the past operation history by the number of times that Undo operations have been accepted; The restoration processing unit is a program for advancing the restored display state of the object by the number of times a Redo operation has been accepted, in accordance with the operation cancellation history.

10. A display device for displaying the object, a display state saving processing unit that saves display state data of the object being displayed each time a condition is satisfied; an operation receiving unit that receives a selection of the display state data; a restoration processing unit that restores a display state of the object based on the display state data selected by the operation receiving unit; a display control unit that displays, in a straight line, a plurality of images representing the display state of the object restored by the restoration processing unit; the display state saving processing unit saves a linear list of past operation histories at the time when the display state data is saved, in association with the display state data, each time the condition is satisfied; storing a linear list of operation undo histories that changed the display state of the object displayed after the display state data was saved, in association with the display state data that was last saved; the display control unit slides the plurality of images in a direction in which the input unit operates the touch panel; when the input means is stopped, the display control unit displays, above the image touched by the input means, an image in a second display state returned in accordance with the past operation history, starting from the display state corresponding to the image; a program for displaying, below the image touched by the input means, an image in a second display state advanced from the display state corresponding to the image as a starting point in accordance with the operation cancellation history;

11. A display method performed by a display device that displays an object, a display state saving processing unit saving display state data of the object being displayed each time a condition is satisfied; an operation receiving unit receiving a selection of the display state data; a restoration processing unit restoring the display state of the object based on the display state data selected by the operation receiving unit; a step in which the display state saving processing unit saves a linear list of past operation histories at the time when the display state data is saved, in association with the display state data, each time the condition is satisfied; storing a linear list of undo operations that changed the display state of the displayed object after the display state data was saved, in association with the display state data that was last saved; the restoration processing unit restores the display state of the restored object according to the past operation history by the number of times the Undo operation has been accepted; the restoration processing unit advances the restored display state of the object by the number of times a Redo operation has been accepted in accordance with the operation undo history; A display method comprising:

12. A display method performed by a display device that displays an object, a display state saving processing unit saving display state data of the object being displayed each time a condition is satisfied; an operation receiving unit receiving a selection of the display state data; a restoration processing unit restoring the display state of the object based on the display state data selected by the operation receiving unit; a display control unit displaying, in a straight line, a plurality of images representing the display state of the object restored by the restoration processing unit; a step in which the display state saving processing unit saves a linear list of past operation histories at the time when the display state data is saved, in association with the display state data, each time the condition is satisfied; storing a linear list of undo operations that changed the display state of the displayed object after the display state data was saved, in association with the display state data that was last saved; the display control unit slides the plurality of images in a direction in which the input unit operates the touch panel; when the input means is stopped, the display control unit displays, above the image touched by the input means, an image in a second display state returned in accordance with the past operation history, starting from the display state corresponding to the image; a step of displaying an image of a second display state advanced from the display state corresponding to the image touched by the input means as a starting point in accordance with the operation cancellation history below the image; A display method comprising:

13. A display system in which a display device that displays an object and a server device communicate with each other via a network, The display device includes: an operation receiving unit that receives an operation to change the display state of an object; a first communication unit that transmits operation content related to the object to the server device; The server device a display state saving processing unit that saves display state data of the object displayed on the display device each time a condition is satisfied; a restoration processing unit that restores a display state of the object based on the display state data in response to a request from the display device, the display state saving processing unit saves a linear list of past operation histories at the time when the display state data is saved, in association with the display state data, each time the condition is satisfied; storing a linear list of operation undo histories that changed the display state of the object displayed after the display state data was saved, in association with the display state data that was last saved; a second communication unit that transmits to the display device a plurality of images representing the display state of the object restored by the restoration processing unit, a linear list of past operation histories, and a linear list of operation cancellation histories; the display device has a display control unit that displays the plurality of images in a straight line, the display control unit returns the restored display state of the object according to the past operation history by the number of times that Undo operations have been accepted; A display system characterized in that the restored display state of the object is advanced in accordance with the operation cancellation history by the number of times a Redo operation is accepted.

14. A display system in which a display device that displays an object and a server device communicate with each other via a network, The display device includes: an operation receiving unit that receives an operation to change the display state of an object; a first communication unit that transmits operation content related to the object to the server device; The server device a display state saving processing unit that saves display state data of the object being displayed on the display device each time a condition is satisfied; a restoration processing unit that restores a display state of the object based on the display state data in response to a request from the display device, the display state saving processing unit saves a linear list of past operation histories at the time when the display state data is saved, in association with the display state data, each time the condition is satisfied; storing a linear list of operation undo histories that changed the display state of the object displayed after the display state data was saved, in association with the display state data that was last saved; a second communication unit that transmits to the display device a plurality of images representing the display state of the object restored by the restoration processing unit, a linear list of past operation histories, and a linear list of operation cancellation histories; The display device includes: a display control unit that displays the plurality of images in a straight line; the display control unit slides the plurality of images in a direction in which the input unit operates the touch panel; when the input means is stopped, the display control unit displays, above the image touched by the input means, an image in a second display state returned in accordance with the past operation history, starting from the display state corresponding to the image; A display system characterized by displaying an image of a second display state, which is advanced according to the operation cancellation history starting from the display state corresponding to the image, below the image touched by the input means.

Images