Information processing apparatus, log data generation method, program, and information processing system

Abstract

To suppress increase in processing load of an information processing apparatus when receiving handwritten input in parallel with generation of log data.SOLUTION: An information processing apparatus for periodically generating log data includes: an input receiving unit which receives a handwritten input from a user; a determination unit which determines whether the input receiving unit is continuing the handwritten input when periodically generating log data; and a log data generation unit which does not generate, when the determination unit determines that the input receiving unit is continuing the handwritten input, the log data until the input receiving unit completes receiving the hand written input, or generates the log data when determining that the input receiving unit is not continuing the handwritten input.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates to an information processing device, a log data generating method, a program, and an information processing system. [Background technology]

[0002] There are known information processing devices that display handwritten data that has been handwritten. Some of the information processing devices communicate via a communication network, and some of them transmit log data to a server device using this function. The log data is data that records the current state, operation history, execution results of processing, and the presence or absence of abnormalities. By analyzing the log data, a person in charge can identify, for example, the state of the information processing device, the user's operation, the network status, or the cause of a failure.

[0003] There is known a technique for adjusting the retransmission waiting time until the next log data transmission depending on the network situation (for example, see Patent Document 1). Patent Document 1 discloses a technique for making the next retransmission waiting time longer than the previous retransmission waiting time when the transmission of a log file fails. Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, there is a risk that the generation of log data increases the processing load of the information processing device. For example, when the generation of log data and a user operation such as handwriting are performed in parallel, the processing load of the information processing device increases, and the processing may become slow or some log data may be lost.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide a technique for suppressing an increase in the processing load of an information processing device when handwritten inputs are accepted at the same time when log data is generated. [Means for solving the problem]

[0006] In view of the above problems, the present invention is an information processing device that periodically generates log data, comprising: an input accepting unit that accepts handwritten input from a user; a judgment unit that, when periodically generating the log data, judges whether the input accepting unit is continuing the handwritten input; and a log data generation unit that, when the judgment unit determines that the input accepting unit is continuing the handwritten input, does not generate the log data until the input accepting unit has finished accepting the handwritten input, and generates the log data when the judgment unit determines that the input accepting unit is not continuing the handwritten input. Effect of the Invention

[0007] The present invention can provide a technique for suppressing an increase in the processing load of an information processing device when handwritten inputs are received at the same time when log data is generated. [Brief description of the drawings]

[0008] [Figure 1] 11A and 11B are diagrams illustrating a process of transmitting operation history information on a display device as log data. [Diagram 2] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system having a display device and a server device. [Diagram 3] FIG. 1 is a diagram showing an example of the arrangement of display devices. [Figure 4A] FIG. 2 is a diagram illustrating an example of a hardware configuration of a display device. [Figure 4B] FIG. 2 is a diagram illustrating an example of a hardware configuration of a server device. [Diagram 5] FIG. 2 is an example of a functional block diagram illustrating functions of a display device and a server device in blocks. [Figure 6] 4A and 4B are diagrams illustrating object data stored in an object data storage unit. [Figure 7] FIG. 4 is a diagram illustrating an example of log data stored in a log data storage unit. [Figure 8] 10A and 10B are diagrams illustrating a process in which the display device periodically transmits log data. [Figure 9]FIG. 10 is a diagram illustrating a process for generating log data shown as a comparative technique. [Figure 10] 5A to 5C are diagrams illustrating a process in which the display device of the present embodiment generates log data. [Figure 11] 1 is an example of a flowchart illustrating a process in which the display device generates log data and transmits it to the server device 10. FIG. [Figure 12] 11A and 11B are diagrams illustrating a process of transmitting log data when the communication network is congested at the time of transmitting the log data. [Figure 13] 11 is a flowchart illustrating an example of a process in which the display device generates log data and transmits the log data to the server device when the communication network is not congested. FIG. [Figure 14] 11 is a flowchart illustrating an example of a process in which the display device generates compressed log data and transmits the compressed log data to the server device. [Figure 15] 11 is a flowchart illustrating an example of a process in which the display device generates partially thinned log data and transmits the thinned log data to the server device. [Figure 16] 11 is a flowchart illustrating an example of a process in which the display device generates log data and transmits the log data to the server device when the CPU utilization rate is less than a threshold value. FIG. [Figure 17] 11 is a flowchart illustrating an example of a process in which the display device generates log data based on the priority of a transmission task of the log data and transmits the log data to the server device. FIG. [Figure 18] 11 is a flowchart illustrating an example of a process in which the display device generates log data and transmits it to the server device when the display device is not connected to a pay-per-use network. FIG. [Figure 19] FIG. 11 is a flowchart illustrating an example of a process for generating log data and transmitting the log data to a server device when the movement of the display device is stopped. [Figure 20] FIG. 11 is a flowchart illustrating an example of a process in which a display device connected to a communication network that does not require handover generates log data and transmits the log data to a server device. [Figure 21]10 is a diagram illustrating a process in which the display device generates log data in parallel with a screen switching process. FIG. [Figure 22] 11 is a flowchart illustrating an example of a process in which the display device generates log data in parallel with a screen switching process and transmits the log data to a server device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A display device as an example of an embodiment of the present invention and a log data generating method performed by the display device will be described below with reference to the drawings.

[0010] [First embodiment] <Problems when sending log data> First, a problem that may occur when the display device 2 transmits log data will be described with reference to Fig. 1. Fig. 1 is a diagram for explaining a process of transmitting operation history information on the display device 2 as log data.

[0011] First, the display device 2 periodically generates log data including operation history information (for example, every 10 minutes). This log data is generated regardless of whether or not the user performs an operation. Therefore, even if the user continues to input handwriting into the display device 2, operation history information up to the point at which the generation of log data begins is generated and transmitted to the server device. In FIG. 1(a), the generation of log data begins when the user finishes writing "aiu." Even if the user continues by writing "e-o" by hand, the display device 2 generates and transmits log data including operation history information up to "aiu."

[0012] The operation history information of the handwritten contents and operations performed after the generation of the log data has started is sent to the server device as the next log data. In FIG. 1(b), the user handwrites "e-o" after "ai-u." The log data for this "e-o" is generated and sent 10 minutes after the transmission of the log data for "ai-u." In this way, there are cases where the log data for handwritten input that is to be analyzed together is fragmented, or part of the log data is sent later in time.

[0013] In such a case, since there is a time gap between when the user writes "e o" by hand and when the display device 2 generates and transmits the next log data, the power to the display device 2 may be turned off for an unexpected reason (including forced termination of an application, interruption of execution, etc.). Even if the power is turned off, the log data is retained, but if the power is turned off due to processing load or the like, operation history information including, for example, the handwriting of "e o" may be lost. The person in charge would like to analyze the cause of the power outage, but since the next log data is not transmitted to the server device 10, in this case the cause of the power outage cannot be analyzed.

[0014] Therefore, it is conceivable that the display device 2 collectively generates log data related to a series of handwritten inputs (for example, "aiueo") that are to be analyzed collectively. However, because the log data is generated periodically, the process of generating the log data may be executed when the processing load of the display device 2 is high, which may result in a decrease in usability, such as slow response of the display device, or a loss of part of the log data.

[0015] <Log Data Transmission Process of the Present Embodiment> Therefore, when the display device 2 of this embodiment determines that it has not accepted an operation including handwritten input from the user (when the processing load is not high), it generates log data including operation history information and transmits it to the server device 10. On the other hand, when the display device 2 determines that it has accepted an operation including handwritten input from the user (when the processing load is high), it generates log data including operation history information after the acceptance of the operation including handwritten input is completed, and transmits it to the server device 10.

[0016] In this way, even if a periodic timing arrives, the acceptance of operations including handwritten input takes precedence over the generation of log data, so that the user's operation history information can be transmitted to the server device 10 without interruption. Based on the example of FIG. 1, the timing for generating log data comes when the user writes "aiu" by hand, but since handwriting is in progress, the display device 2 does not generate log data. When the user completes the handwritten input after writing "aiueo," the display device 2 generates log data and transmits it to the server device 10. Even if the power supply of the display device 2 is cut off due to an unexpected cause after the user performs an operation including handwritten input (handwriting "aiueo"), the person in charge can analyze the cause of the power cut because the log data including the user's operation history information has been transmitted to the server device 10. In addition, since log data is not generated when handwritten input is continuing, an increase in the processing load of the display device 2 can be suppressed.

[0017] <Terminology> Log data refers to data that records processes executed by a computer or communication device, events detected, states, and the like. The processes, events, and states that are actually recorded vary depending on the device. In this embodiment, it is preferable that the log data includes operation history information. An example of log data will be described later.

[0018] CPU usage rate refers to the ratio of "non-idle time" to a given time, that is, the time when the CPU is not executing an idle thread. "Non-idle time" may also be called the time when the CPU is busy. The CPU usage rate is monitored by the OS (Operating System), so the display device 2 can obtain the CPU usage rate from the OS.

[0019] Task priority is the priority of each task executed by the CPU. There are many types of tasks, such as tasks that send log data, tasks related to application processing, and tasks related to OS processing. Task priority is determined by the task and is managed by the OS.

[0020] A pay-per-use network refers to a connection using a mobile router or a public line provided by a 4G, 5G, 6G, or other line provider. A public line is a communication line connecting bases that is physically shared and used by unspecified users. An example is a mobile phone network. In contrast, a network that is free or charges a flat rate, such as a wired LAN, wireless LAN, or Bluetooth (registered trademark), is called a flat-rate network.

[0021] Network congestion is an index that is detected by some measurement method as to how much data is flowing through the network bandwidth. In this embodiment, the degree of network congestion is determined by, for example, a ping command.

[0022] The information processing device may be any device that allows handwritten input and generates log data. In this embodiment, the display device 2 will be described as an example.

[0023] <System configuration example> 2 shows an example of the configuration of an information processing system 100 including a display device 2 and a server device 10. The information processing system 100 includes the server device 10 and the display device 2 that are communicatively connected via a communication network N.

[0024] The communication network N may be a LAN constructed in a facility where the display device 2 is installed, a WAN having a plurality of LANs, the Internet, or the like. The communication network N may be any network that allows communication between the display device 2 and the server device 10. The communication network N may be constructed as either a wired or wireless network, or may be a combination of wired and wireless networks. The display device 2 may also communicate with the server device 10 by connecting to a public line such as a mobile phone network.

[0025] The display device 2 is an example of an "information processing device." The display device 2 is a device that allows handwritten input via a touch panel and displays the handwritten content. The display device 2 may have a function of character recognition of handwritten data and converting it into text data. The display device 2 may also have an electronic paper panel that has a function of "remaining the display even when not powered" and a function of "being rewritable." The display device 2 may also be a so-called electronic blackboard, electronic whiteboard, information board, or the like.

[0026] The display device 2 may also be used as digital signage. The display device 2 with an electronic paper panel consumes zero or close to zero power to continue displaying, so it can be used in places where it cannot be connected to an electrical outlet. The display device 2 may output data such as video, documents, and music acquired from an external PC (Personal Computer) or communication network N for advertising purposes.

[0027] The display device 2 may also be an information processing terminal capable of handwriting input (having a touch panel). That is, the display device 2 may be a notebook PC, a smartphone, a tablet terminal, a game console, a PDA (Personal Digital Assistant), a digital camera, a wearable PC, or the like. These information processing terminals are usually used as general-purpose information processing terminals, but when application software for handwriting input is executed, they perform handwriting input, character recognition, and the like, just like the dedicated display device 2.

[0028] The server device 10 is, for example, one or more computers on a communication network N. A server is a computer or software that performs the function of providing information or processing results in response to a request from a client. The server device 10 may also be compatible with cloud computing. Cloud computing refers to a form of utilization in which resources on a communication network N are utilized without awareness of specific hardware resources.

[0029] The server device 10 receives log data from each display device 2 and analyzes the log data to identify, for example, the state of the information processing device, the user's operation, the network status, or the cause of a failure. The analysis may be performed by a person in charge. If any abnormality is found, the server device 10 may send an email to an administrator or contact a seller. The server device 10 may exist on the Internet or on-premise.

[0030] <Example of device configuration> Figure 3 is a diagram showing an example of the arrangement of the display device 2. Figure 3(a) shows, as an example of the display device 2, a display device 2 used as a horizontally long electronic whiteboard hung on a wall.

[0031] 3(a), a display 220 as an example of the display device 2 is provided on the upper part of the display device 2. A user U can handwrite characters or the like on the display 220 using a pen 2500.

[0032] FIG. 3(b) shows a display device 2 used as a vertically long electronic whiteboard suspended on a wall.

[0033] 3(c) shows the display device 2 placed flat on a desk 230. Since the display device 2 is about 1 cm thick, there is no need to adjust the height of the desk even if it is placed flat on a normal desk. In addition, the user U can easily move the display device 2.

[0034] The input method of the coordinates of the pen 2500 includes an electromagnetic induction method, an active electrostatic coupling method, etc. The pen 2500 may also have functions such as pen pressure detection, tilt detection, and a hover function (displaying the cursor before the pen touches the screen).

[0035] <Hardware configuration of the device> Next, the hardware configuration of the display device 2 will be described with reference to Fig. 4A. Fig. 4A is an example of a hardware configuration diagram of the display device 2. As shown in Fig. 4A, the display device 2 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and an SSD (Solid State Drive) 204.

[0036] Of these, the CPU 201 controls the overall operation of the display device 2. The ROM 202 stores the CPU 201 and programs used to drive the CPU 201, such as an IPL (Initial Program Loader). The RAM 203 is used as a work area for the CPU 201.

[0037] The SSD 204 stores various data such as the OS and programs for the display device 2. This program may be an application program that runs on an information processing device equipped with a general-purpose OS (Windows (registered trademark), LINUX (registered trademark), Mac OS (registered trademark), Android (registered trademark), iOS (registered trademark), etc.). In this case, it is usually used as a general-purpose information processing device, but when a user executes an application program, the user can write by hand, just like a dedicated device for the display device 2.

[0038] The display device 2 also includes a display controller 213, a touch sensor controller 215, a touch sensor 216, a display 220, a tilt sensor 217, a serial interface 218, a speaker 219, a microphone 221, a wireless communication device 222, an infrared I / F 223, a power control circuit 224, a power switch 227, an AC adapter 225, and a battery 226.

[0039] The display controller 213 controls and manages the screen display in order to output an output image to the display 220, etc. The touch sensor 216 detects that the pen 2500, the user's hand, etc. (the pen and the user's hand serve as input means) have touched the display 220. The touch sensor 216 also receives a pen ID.

[0040] The touch sensor controller 215 controls the processing of the touch sensor 216. The touch sensor 216 inputs and detects coordinates. The method of inputting and detecting coordinates will be described. For example, in the case of an optical type, two light receiving and emitting devices installed at both ends of the upper side of the display 220 emit a plurality of infrared rays in parallel to the display 220. The two light receiving and emitting devices receive light that is reflected by a reflecting member installed around the display 220 and returns along the same optical path as the optical path of the light emitted by the light receiving element.

[0041] The touch sensor 216 outputs position information of the infrared rays emitted by the two light receiving and emitting devices that are blocked by the object to the touch sensor controller 215, and the touch sensor controller 215 identifies the coordinate position that is the contact position of the object. The touch sensor controller 215 also has a communication unit 215a and can wirelessly communicate with the pen 2500. For example, when communication is performed according to a standard such as Bluetooth (registered trademark), a commercially available pen can be used. If one or more pens 2500 are registered in advance in the communication unit 215a, the user can communicate with the pen 2500 without performing connection settings to communicate with the display device 2.

[0042] The power switch 227 is a switch for switching ON / OFF the power of the display device 2. The tilt sensor 217 is a sensor for detecting the tilt angle of the display device 2. An acceleration sensor may be used as the tilt sensor 217. It is mainly used to detect whether the display device 2 is being used in the installation state of FIG. 3(a), FIG. 3(b), or FIG. 3(c), and can automatically change the thickness of characters, etc. depending on the installation state.

[0043] The serial interface 218 is an external communication interface such as a USB. The serial interface 218 is used for inputting information from the outside. The speaker 219 is used for outputting audio, and the microphone 221 is used for inputting audio. The wireless communication device 222 communicates with a terminal carried by the user, and relays a connection to the Internet, for example. The wireless communication device 222 communicates using Wi-Fi, Bluetooth (registered trademark), or the like, but any communication standard may be used. The wireless communication device 222 forms an access point, and when the user sets an SSID (Service Set Identifier) ​​and password obtained by the user in the terminal carried by the user, the user can connect to the access point.

[0044] It is preferable that the wireless communication device 222 is provided with two access points. a. Access point → Internet b. Access point → Internal network → Internet Access point a is for users outside the company, and users cannot access the company network but can use the Internet. Access point b is for users inside the company, and users can use the company network and the Internet.

[0045] The infrared I / F 223 detects the display device 2 arranged adjacently. The infrared I / F 223 can detect only the display device 2 arranged adjacently by utilizing the linearity of infrared rays. It is preferable that one infrared I / F 223 is provided on each side, and it can detect in which direction of the display device 2 another display device 2 is arranged. This expands the screen, and it is possible to display handwritten information that was previously handwritten on the adjacent display device 2 (handwritten information on another page, with the area of ​​one display 220 being one page) and the like.

[0046] A power supply control circuit 224 controls an AC adapter 225 and a battery 226 which are power sources for the display device 2. The AC adapter 225 converts AC common to commercial power sources into DC.

[0047] If the display 220 is so-called electronic paper, it consumes little or no power to maintain the image display, and therefore can be driven by a battery 226. This makes it possible to use the display device 2 for purposes such as digital signage even in places where it is difficult to connect to a power source, such as outdoors.

[0048] Furthermore, the display device 2 includes a bus line 210. The bus line 210 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 201 shown in Fig. 4A.

[0049] The touch sensor 216 is not limited to an optical type, and may be a capacitive touch panel that identifies the contact position by detecting a change in capacitance. The touch sensor 216 may be a resistive film type touch panel that identifies the contact position by a voltage change in two opposing resistive films. The touch sensor 216 may be an electromagnetic induction type touch panel that identifies the contact position by detecting electromagnetic induction caused by a contact object touching the display unit, and various detection means may be used. The touch sensor 216 may be of a type that does not require an electronic pen to detect the presence or absence of a touch with the tip of the pen. In this case, a fingertip or a pen-shaped stick can be used to perform a touch operation. The pen 2500 does not need to be a long, thin pen type.

[0050] <<Server device>> Fig. 4B is a diagram showing an example of a hardware configuration of the server device 10 according to the present embodiment. As shown in Fig. 4B, the server device 10 is constructed by a computer 500, and includes a CPU 501, a ROM 502, a RAM 503, a HD (Hard Disk) 504, a HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a bus line 510, a keyboard 511, a pointing device 512, an optical drive 514, and a media I / F 516.

[0051] Among these, the CPU 501 controls the operation of the entire computer 500. The ROM 502 stores programs such as an IPL used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls reading or writing of various data from the HD 504 under the control of the CPU 501. The display 506 displays various information such as a cursor, a menu, a window, a character, or an image. The external device connection I / F 508 is an interface for connecting various external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory or a printer. The network I / F 509 is an interface for data communication using the communication network N. The bus line 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501 shown in FIG. 4B.

[0052] Moreover, the keyboard 511 is a type of input means having a plurality of keys used for inputting characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The optical drive 514 controls reading and writing of various data from an optical storage medium 513, which is an example of a removable storage medium. The optical storage medium 513 may be a CD, a DVD, Blu-ray (registered trademark), etc. The media I / F 516 controls reading and writing (storing) of data from a storage medium 515 such as a flash memory.

[0053] <About the function> Next, functions of the display device 2 and the server device 10 will be described with reference to Fig. 5. Fig. 5 is an example of a functional block diagram in which functions of the display device 2 and the server device 10 are divided into blocks and described.

[0054] <<Display device>> First, the display device 2 has an input reception unit 21, a drawing data generation unit 22, a conversion unit 23, a display control unit 24, a data storage unit 25, a network communication unit 26, an operation reception unit 27, a determination unit 28, a log data generation unit 29, and a log data transmission unit 30. Each function of the display device 2 is a function or means realized by any of the components shown in Fig. 4A operating in response to an instruction from the CPU 201 in accordance with a program loaded from the SSD 204 onto the RAM 203.

[0055] The input reception unit 21 receives input of a coordinate trajectory (coordinate point sequence, handwritten data) by detecting the coordinates of the position where an input means such as a pen 2500 touches the touch sensor 216. The drawing data generation unit 22 acquires the coordinates where the tip of the pen 2500 touches from the input reception unit 21. The drawing data generation unit 22 generates stroke data by connecting the coordinate point sequence by interpolating.

[0056] The conversion unit 23 performs character recognition processing on stroke data (handwritten data) handwritten by the user, and converts it into text (character code). In parallel with the user's pen operation, the conversion unit 23 recognizes and converts characters (not only Japanese but also multiple languages ​​such as English), numbers, and symbols (%, $, &, etc.). The conversion unit 23 also performs graphic recognition processing on one or more stroke data (handwritten data) handwritten by the user, and converts them into graphics (lines, circles, triangles, etc.). Various algorithms have been devised for the recognition method, but details will be omitted since known technology can be used in this embodiment.

[0057] The display control unit 24 displays the handwritten data, text converted from the handwritten data, an operation menu for user operation, and the like on the display 220. The data storage unit 25 stores the handwritten data handwritten on the display device 2, the converted text, the PC screen, files, and the like in the object data storage unit 41. The network communication unit 26 connects to the communication network N and transmits and receives data to and from the server device 10 and other devices via the communication network N.

[0058] The operation acceptance unit 27 accepts the selection of any character from a plurality of conversion candidates that have been character-recognized based on the coordinates touched by the pen 2500 or the like, or accepts the pressing of a menu.

[0059] The determination unit 28 performs various determinations. For example, the determination unit 28 determines whether or not the user is continuing handwriting input, that is, whether or not pen down (the pen 2500 touching the touch panel) is detected within a certain time after pen up (the pen 2500 is separated from the touch panel). Pen down and pen up are detected based on whether the input receiving unit 21 detects and no longer detects the contact of an input means such as the pen 2500.

[0060] The log data generating unit 29 collects data to be recorded by the display device 2 (handwritten data within the regular time, processes executed during the regular time, events and conditions detected during the regular time, etc.) and generates log data. The regular time is a design matter and may be, for example, 10 minutes. The regular time does not have to be a fixed time, and may be shorter when the device is in use by a user (e.g., when the user is nearby), or longer when the device is simply powered on. The log data will be explained together with the functions of the server device 10.

[0061] The log data transmission unit 30 controls the transmission of the generated log data via the network communication unit 26. When the log data transmission unit 30 does not receive a notification of completion of log data reception from the server device 10, it resends the log data. In principle, the log data is transmitted immediately after the generation of the log data in order to transmit operation history information up to the present to the server device 10 as soon as possible. However, in cases where the processing load increases after the generation of the log data, the transmission of the log data may be postponed until the processing load decreases or for a certain period of time.

[0062] <<Object data>> FIG. 6 is a diagram illustrating the object data stored in the object data storage unit 41. As shown in FIG.

[0063] The object ID item is identification information for identifying the display data.

[0064] The Type item is the type of object data, and can be handwriting, text, figure, image, table, etc. Handwriting is stroke data (a sequence of coordinate points). Text is one or more characters or symbols (character code) converted from handwritten data. Figures are geometric shapes such as triangles and squares converted from handwritten data. Images are image data such as Jpeg, Png, and Tiff imported from a PC or the Internet. Tables are one-dimensional or two-dimensional table-like objects.

[0065] One screen of the display device 2 is called a page. The item on a page is its page number.

[0066] The coordinates item indicates the position of the object data relative to a predetermined origin of the display device 2. The position of the object data is, for example, the upper left vertex of the bounding rectangle of the object data. The coordinates are expressed, for example, in pixel units of the display.

[0067] The size item is the width and height of the bounding rectangle of the object data.

[0068] The table input item indicates the correspondence between the table and the object that has been input into the table. For example, the text of object ID=2 and 6 is associated with the table of table ID=001. Also, the text of object ID=2 and 6 is shown to have been input into the table (corresponds to the table) in cells (1,1) and (2,1) of the table of table ID=001, respectively. Note that handwriting and figures may also be input into the table.

[0069] <<Server device>> Returning to Fig. 5, the server device 10 has a log data receiving unit 11, a log data analyzing unit 12, and a log data storage unit 13. These functions of the server device 10 are functions or means realized by the CPU 501 in Fig. 4B executing a program installed in the server device 10.

[0070] The log data receiving unit 11 receives log data from each display device 2 and stores the log data in the log data storage unit 13. The log data analysis unit 12 analyzes the log data using a model created by machine learning, for example. The log data analysis unit 12 may further detect a sign of a malfunction and estimate the correspondence between time and remaining battery power.

[0071] Fig. 7 shows an example of log data stored in the log data storage unit 13. Note that since the log data can include a wide variety of information, Fig. 7 shows only a portion of the log data related to this embodiment. The date and time items are the start and end of the operation history information included in the log data. The end is also the date and time when the generation of the log data started. The operation details item includes a chronological history of operations such as handwriting, button pressing, and power off performed by the user since the previous log data was generated. If the operation content includes handwritten input, the coordinate item is a sequence of coordinate points that constitute the handwritten data. The item of CPU utilization rate is the CPU utilization rate of the CPU 201 of the display device 2. The CPU utilization rate may be, for example, an average at regular intervals when log data is generated. The network status item is the congestion level of the communication network N. The network status may be, for example, the turnaround time obtained by a ping command or a value obtained by converting the turnaround time into a five-level scale. The Mobile Network item indicates whether the display device 2 is connected to a mobile router. The handover item indicates whether or not the access point to which the display device 2 is connected has been switched from one access point to another, or whether or not handover is possible. The item of remote conference indicates whether or not the display device 2 communicated with another base via the communication network N (whether or not a remote conference was held). If a remote conference was held, the actual duration of the conference may be recorded. The OS task priority field indicates the priority of a task generated by the OS (e.g., network communication processing). In general, the OS task priority is higher than a task that processes handwritten input. The OS task priority may be the average, maximum, or minimum value at the regular intervals at which log data is generated.

[0072] <Log data transmission control> Next, the transmission control of log data will be described with reference to Fig. 8. Fig. 8 is a diagram that shows a schematic diagram of a process in which the display device 2 periodically transmits log data. The log data generating unit 29 periodically generates log data, and the log data transmitting unit 30 transmits the log data to the server device 10 (uploads the data in Fig. 8). Fig. 8 shows an example in which the data is transmitted every 10 minutes.

[0073] On the other hand, as shown in FIG. 9, there are cases where the log data generating unit 29 starts generating log data while the user is still writing by hand. FIG. 9 shows a schematic diagram of a process for generating log data shown as a comparative technique. In FIG. 9, the time for periodic generation of log data arrives while the user is still writing by hand. Even if the user continues to write by hand on the display device 2, operation history information up to the point when the generation of log data starts is generated and transmitted to the server device. For example, the generation of log data starts when the user finishes writing "aiu". Even if the user continues to write "e o" by hand, the log data generating unit 29 generates and transmits log data including operation history information up to "aiu".

[0074] The operation history information of the handwritten contents and operations performed after the generation of the log data has started is transmitted to the server device as the next log data. Suppose a user writes "e-o" after "ai-u." The log data for "e-o" is generated and transmitted 10 minutes after the transmission of the log data for "ai-u." In this way, there are cases where the log data for handwritten input that is to be analyzed collectively is fragmented, or part of the log data is transmitted later in time.

[0075] In such a case, since there is a time gap between when the user writes "e o" by hand and when the display device 2 generates and transmits the next log data, the power to the display device 2 may be turned off for an unexpected reason (including forced termination of an application, interruption of execution, etc.). Even if the power is turned off, the log data is retained, but if the power is turned off due to processing load or the like, operation history information including, for example, the handwriting of "e o" may be lost. The person in charge would like to analyze the cause of the power outage, but since the next log data is not transmitted to the server device 10, in this case the cause of the power outage cannot be analyzed.

[0076] Therefore, it is conceivable that the log data generating unit 29 collectively generates log data related to a series of handwritten inputs (for example, "aiueo") that are to be analyzed collectively. However, because the log data is generated periodically, the log data generation process may be executed when the processing load of the display device 2 is high, which may result in a decrease in usability, such as a slow response of the display device, or a loss of part of the log data.

[0077] For this reason, in this embodiment, as shown in Fig. 10, if the timing for periodic generation of log data arrives while the user is continuing handwritten input, the log data generation unit 29 delays the generation of log data until the handwritten input is completed. Fig. 10 shows a schematic diagram of a process in which the display device 2 of this embodiment generates log data. In Fig. 10, the timing for periodic generation of log data arrives while the user is continuing handwritten input. However, the log data generation unit 29 does not generate log data at the periodic timing, but generates log data when the handwritten input is completed.

[0078] Whether or not the user is continuing handwriting input is determined by the determination unit 28 based on whether the pen 2500 has contacted the touch panel within a certain period of time since the pen 2500 last contacted the touch panel. "When handwriting input is completed" means that the determination unit 28 has determined that the pen 2500 has not contacted the touch panel within a certain period of time since the pen 2500 last contacted the touch panel.

[0079] 11 is a flowchart illustrating a process in which the display device 2 generates log data and transmits it to the server device 10. FIG 11 is a flowchart illustrating the transmission process of FIG 10.

[0080] The log data generation unit 29 counts down or up the timer to monitor whether or not a regular time T has elapsed since the previous log data generation (S11). Until the regular time T has elapsed (No in S11), the log data generation unit 29 repeats the process of step S11.

[0081] When the regular time T has elapsed (Yes in S11), the judgment unit 28 judges whether or not the user is continuing the handwritten input (S12). When the judgment unit 28 judges that the user is continuing the handwritten input (Yes in S12), the judgment unit 28 waits until the handwritten input is completed. In other words, the log data generation unit 29 does not generate log data until the handwritten input is completed.

[0082] When the determination unit 28 determines that the user is not continuing the handwritten input, that is, when the user completes the handwritten input (No in S12), the log data generation unit 29 generates log data (S13).

[0083] The log data transmission unit 30 transmits the generated log data to the server device 10 (S14). Note that the log data transmission unit 30 resets the timer.

[0084] Through the above process, the display device 2 can transmit a series of operation history information of the user, such as "aiueo," to the server device 10 without interruption. Even if the power supply to the display device 2 is cut off for an unexpected reason after the user performs an operation including handwritten input, the server device 10 can analyze the cause of the power cut because the log data including the user's operation history information has been transmitted to the server device 10. In addition, since the generation of log data and handwritten input are not performed in parallel, the processing load is unlikely to increase, and it is possible to prevent a decrease in usability and a loss of part of the log data.

[0085] <Delay sending log data when network is congested> The log data transmission unit 30 can delay the generation and transmission of log data when the communication network N is congested. This will be described with reference to FIG.

[0086] 12 is a diagram for explaining the transmission process of log data when the communication network N is congested at the time of transmitting the log data. As in FIG. 10, the timing for periodic generation of log data arrived while the user was continuing handwritten input, so the log data generation unit 29 did not generate log data at the periodic timing. Also, at the timing when the handwritten input was completed, the communication network N was congested, so the log data generation unit 29 did not generate log data. After the congestion of the communication network N is alleviated, the log data generation unit 29 generates log data, and the log data transmission unit 30 transmits the log data to the server device 10.

[0087] 13 is a flowchart for explaining the process in which the display device 2 generates log data and transmits it to the server device 10 when the communication network is not congested. FIG. 13 is a flowchart showing the transmission process of FIG. 12. The explanation of FIG. 13 will mainly focus on the differences from FIG. 11. The processes of steps S21 and S22 may be similar to steps S11 and S12 of FIG. 11.

[0088] Furthermore, when the handwritten input is completed, the determination unit 28 determines whether the communication network N is congested (S23). One method of determining whether the communication network N is congested is, for example, to refer to the round trip time obtained by a ping command. The round trip time refers to the length of time it takes for a data packet to be sent to the destination and for an acknowledgment response to that packet to be received by the source. The determination unit 28 determines that the communication network N is congested if the round trip time to the default gateway set in the network settings is equal to or greater than a threshold value. Alternatively, the determination unit 28 may determine that the communication network N is congested if there are many packets passing through the sub-network.

[0089] When the communication network N is congested, the log data generating unit 29 waits until the congestion is alleviated. When the handwriting input is resumed, the log data generating unit 29 waits until the handwriting input is completed.

[0090] When the congestion of the communication network N is alleviated (No in S23), the log data generation unit 29 generates log data (S24). The log data transmission unit 30 transmits the generated log data to the server device 10 (S25). The log data transmission unit 30 resets the timer.

[0091] When the communication network N is congested, the generation and transmission of log data is delayed, which reduces the retransmission control of the log data, thereby suppressing an increase in the processing load and enabling the log data to be transmitted reliably.

[0092] <Processing of log data when communication network is congested> When the communication network N is congested, instead of delaying the generation of log data, the log data generating unit 29 may increase the compression rate of the log data (to compress the log data so that the file size becomes smaller) or thin out the log data (to reduce the file size). In this way, the log data transmitting unit 30 can transmit the log data even if the congestion of the communication network N is not resolved.

[0093] Fig. 14 is a flow chart for explaining a process in which the display device 2 generates compressed log data and transmits it to the server device 10. In the explanation of Fig. 14, the differences from Fig. 13 will be mainly explained. In Fig. 14, the order of generating the log data and determining whether the communication network N is congested is reversed compared to Fig. 13, and further, step S35 is added.

[0094] That is, when it is determined in step S32 that the handwritten input is completed, the log data generating unit 29 generates log data (S33).

[0095] Then, the judgment unit 28 judges whether the communication network N is congested (S34). If the communication network N is congested (Yes in S34), the log data generation unit 29 increases the compression rate of the log data (S35). The compression rate can be adjusted by a compression method such as XZ. Increasing the compression rate increases the processing load, but the degree of increase in the processing load is smaller than that in the case of handwritten input.

[0096] In step S36, the log data transmission unit 30 transmits the log data with the high compression rate to the server device 10 (S36). When the communication network N is not congested, the log data transmission unit 30 transmits the log data with a normal compression rate (a compression rate lower than when the communication network N is congested) to the server device 10.

[0097] Fig. 15 is a flow chart for explaining a process in which the display device 2 generates partially thinned log data and transmits it to the server device 10. In the explanation of Fig. 15, the differences from Fig. 13 will be mainly explained. In Fig. 15, the order of generating log data and determining whether the communication network N is congested is reversed compared to Fig. 13, and further, step S45 is added.

[0098] That is, when it is determined in step S42 that the handwritten input is completed, the log data generating unit 29 generates log data (S43).

[0099] Then, the judgment unit 28 judges whether the communication network N is congested (S44). If the communication network N is congested (Yes in S44), the log data generation unit 29 thins out the log data (S45). Thinning out the log data means deleting log data with a low priority from among the log data as shown in FIG. 7. The priority of the log data is determined by the item of the log data. The priority level of the log data to be deleted may be adjusted depending on the degree of congestion.

[0100] The log data transmission unit 30 transmits the log data, some of which was thinned out in step S45, to the server device 10 (S46). If the communication network N is not congested, the log data transmission unit 30 transmits the unthinned log data to the server device 10.

[0101] In this way, even if congestion in the communication network N is difficult to alleviate (log data cannot be transmitted for a long time), the log data transmission unit 30 can transmit the log data. Note that the processes in Fig. 14 and Fig. 15 may be executed when the communication network N is congested for a certain period of time or more.

[0102] <Major Effects> The display device 2 of this embodiment can transmit the user's operation history information to the server device 10 without interruption. Even if the power supply of the display device 2 is cut off for an unexpected reason after the user performs an operation including handwritten input, the server device 10 can analyze the cause of the power cut because the log data including the user's operation history information has been transmitted to the server device 10. In addition, since the generation of log data and handwritten input are not performed in parallel, the processing load is unlikely to increase, and it is possible to suppress deterioration of usability and loss of part of the log data.

[0103] [Second embodiment] In the first embodiment, the continuation of handwritten input is detected, but an increase in the processing load can also be detected by another indicator. Therefore, in this embodiment, a display device 2 that, when the timing for periodic generation of log data arrives, delays the generation of log data until the CPU usage falls below a threshold will be described.

[0104] In this embodiment, the hardware configuration diagrams of FIG. 4A and FIG. 4B and the functional block diagram of FIG. 5 described in the above embodiment can be used.

[0105] Fig. 16 is a flow chart for explaining the process in which the display device 2 generates log data and transmits it to the server device 10 when the CPU utilization rate is below the threshold. In the explanation of Fig. 16, the differences from Fig. 11 will be mainly explained. In Fig. 16, the process of step S52 differs from step S12 in Fig. 11. In step S52, the determination unit 28 determines whether the CPU utilization rate is equal to or higher than the threshold (S52). If the CPU utilization rate is equal to or higher than the threshold (Yes in S52), the determination unit 28 waits until the CPU utilization rate becomes less than the threshold.

[0106] When the CPU utilization rate falls below the threshold value (No in S52), the log data generator 29 generates log data (S53). The process in step S54 may be the same as that in step S14 in FIG.

[0107] When the CPU usage rate is high, if the log data generating unit 29 generates log data, the processing load of the CPU 201 and the like increases further due to the generation of the log data. This may have adverse effects such as making it difficult for the user to input by handwriting (response is reduced) and causing defects in the log data generated by the log data generating unit 29. In addition to the effects of the first embodiment, the present embodiment can suppress these inconveniences.

[0108] In this embodiment as well, when the communication network N is congested, the generation of log data may be delayed, the compression rate of the log data may be increased, or the log data may be thinned out.

[0109] [Third embodiment] In this embodiment, a display device 2 will be described that, when the time for periodic generation of log data arrives, delays generation of log data until the priority of a log data transmission task becomes higher than other tasks.

[0110] In this embodiment, the hardware configuration diagrams of FIG. 4A and FIG. 4B and the functional block diagram of FIG. 5 described in the above embodiment can be used.

[0111] FIG. 17 is a flow chart for explaining a process in which the display device 2 generates log data based on the priority of the log data transmission task and transmits it to the server device 10. In the explanation of FIG. 17, the difference from FIG. 11 will be mainly explained. In FIG. 17, the process of step S62 is different from step S12 of FIG. 11. In step S62, the judgment unit 28 judges whether the priority of the log data transmission task is lower than other tasks (S62). A task refers to a process executed by a CPU. The OS executes the tasks registered in the queue in order, but performs task scheduling such as changing the execution order according to the priority. For this reason, if the priority of the log data transmission task is lower than other tasks, it becomes difficult to transmit the log data. In addition, the processing load of the display device 2 also increases. The priority of the tasks registered in the queue can be obtained from the OS.

[0112] If the priority of the transmission task is lower than the other tasks (Yes in S62), the decision unit 28 waits until the priority of the transmission task becomes higher than the other tasks.

[0113] When the priority of the transmission task becomes the same as or higher than other tasks (No in S62), the log data generator 29 generates log data (S63). The process in step S64 may be the same as that in step S14 in FIG.

[0114] When the priority of the transmission task is lower than other tasks, if the log data generating unit 29 generates log data, the processing load of the CPU 201 and the like increases further due to the generation of the log data. This may have adverse effects such as making it difficult for the user to input by handwriting (response is reduced) and causing defects in the log data generated by the log data generating unit 29. In addition to the effects of the first embodiment, the present embodiment can suppress these inconveniences.

[0115] In this embodiment as well, when the communication network N is congested, the generation of log data may be delayed, the compression rate of the log data may be increased, or the log data may be thinned out.

[0116] [Fourth embodiment] In this embodiment, a display device 2 is described that, when the time for periodic generation of log data arrives, delays the generation of log data until the display device 2 is connected to a flat-rate charging network instead of a metered-rate charging network.

[0117] In this embodiment, the hardware configuration diagrams of FIG. 4A and FIG. 4B and the functional block diagram of FIG. 5 described in the above embodiment can be used.

[0118] FIG. 18 is a flow chart for explaining a process in which the display device 2 generates log data and transmits it to the server device 10 when not connected to a pay-per-use network. In the explanation of FIG. 18, the difference from FIG. 11 will be mainly explained. In FIG. 18, the process of step S72 is different from step S12 of FIG. 11. In step S72, the judgment unit 28 judges whether the display device 2 is connected to a pay-per-use network (S72). Pay-per-use refers to a network in which the network usage fee increases according to the amount of data transmitted and the connection time. Pay-per-use networks include, for example, a network connected via a mobile router, or a network connected via a mobile phone network such as 4G, 5G, or 6G. A method for judging whether a mobile router is connected to a mobile router includes a method for inquiring of the mobile router (access point). Another method for judging whether a mobile router is connected to a mobile phone network includes a method for comparing the MAC address of the mobile router with a list (prepared in advance) of MAC addresses of mobile routers circulating for pay-per-use networks. Since the OS detects whether a mobile phone network is connected, the judgment unit 28 inquires of the OS.

[0119] If connected to the pay-per-use network (Yes in S72), the decision unit 28 waits until connected to the flat-rate charging network.

[0120] When connected to the flat-rate charging network (No in S72), the log data generating unit 29 generates log data (S73). The process in step S74 may be the same as that in step S14 in FIG.

[0121] When connected to a pay-per-use network, if the log data generating unit 29 generates log data, the processing load may increase (retransmission control) due to switching to a low-speed line and interrupting communication, etc. This may have adverse effects such as making it difficult for the user to input handwritten characters (response is reduced) and causing defects in the log data generated by the log data generating unit 29. In addition to the effects of the first embodiment, the present embodiment can suppress these inconveniences.

[0122] In this embodiment as well, when the communication network N is congested, the generation of log data may be delayed, the compression rate of the log data may be increased, or the log data may be thinned out.

[0123] [Fifth embodiment] In this embodiment, a display device 2 that delays the generation of log data until the movement of the display device 2 stops when the timing for periodic generation of log data arrives will be described.

[0124] In this embodiment, the hardware configuration diagrams of FIG. 4A and FIG. 4B and the functional block diagram of FIG. 5 described in the above embodiment can be used.

[0125] FIG. 19 is a flow chart for explaining a process of generating log data and transmitting it to the server device 10 when the movement of the display device 2 stops. In the explanation of FIG. 19, the difference from FIG. 11 will be mainly explained. In FIG. 19, the process of step S82 is different from step S12 of FIG. 11. In step S82, the judgment unit 28 judges whether the display device 2 is moving (S82). The display device 2 is easily portable. However, when the display device 2 is moving, communication may not be stable, and communication may become unstable due to handover or the like. Whether the display device 2 is moving can be judged by a fluctuation in the signal of the acceleration sensor (tilt sensor 217). If the display device 2 acquires position information by GPS or the like, it can be judged by a change in the position information.

[0126] If the display device 2 is moving (Yes in S82), the determination unit 28 waits until the movement stops.

[0127] When the movement of the display device 2 has stopped (No in S82), the log data generating unit 29 generates log data (S83). The process in step S84 may be the same as that in step S14 in FIG.

[0128] When the display device 2 is moving, if the log data generating unit 29 generates log data, the processing load may increase (retransmission control) due to interruption of communication, etc. This may have adverse effects such as making it difficult for the user to input handwritten data (decreased response) and causing defects in the log data generated by the log data generating unit 29. In addition to the effects of the first embodiment, the present embodiment can suppress these inconveniences.

[0129] In this embodiment as well, when the communication network N is congested, the generation of log data may be delayed, the compression rate of the log data may be increased, or the log data may be thinned out.

[0130] [Sixth embodiment] In this embodiment, a display device 2 that, when the timing for periodic generation of log data arrives, delays the generation of log data until the display device 2 is connected to a communication network N that does not require handover will be described.

[0131] In this embodiment, the hardware configuration diagrams of FIG. 4A and FIG. 4B and the functional block diagram of FIG. 5 described in the above embodiment can be used.

[0132] FIG. 20 is a flow chart for explaining the process in which the display device 2 connected to a communication network N that does not allow handover generates log data and transmits it to the server device 10. In the explanation of FIG. 20, the differences from FIG. 11 will be mainly explained. In FIG. 20, the process of step S92 is different from step S12 in FIG. 11. In step S92, the judgment unit 28 judges whether or not the display device 2 is connected to a communication network N that allows handover (S92). Handover refers to switching the access point to which the display device 2 is connected during communication. This is because communication may not be stable if the display device 2 is connected to a communication network N that allows handover. Whether or not the display device 2 is connected to a communication network N that allows handover can be determined based on whether multiple access points having the same SSID are detected.

[0133] When the display device 2 is connected to a communication network N that allows handover (Yes in S92), the determination unit 28 waits until the display device 2 is connected to a communication network N that does not allow handover.

[0134] When the display device 2 is connected to the communication network N that does not require handover (No in S92), the log data generating unit 29 generates log data (S93). The process in step S94 may be the same as that in step S14 in FIG.

[0135] When the display device 2 is connected to a communication network N that allows handover, if the log data generating unit 29 generates log data, the processing load may increase (retransmission control) due to communication interruption or the like. This may have adverse effects such as making it difficult for the user to input handwritten data (response is reduced) and causing defects in the log data generated by the log data generating unit 29. In addition to the effects of the first embodiment, the present embodiment can suppress these inconveniences.

[0136] In this embodiment as well, when the communication network N is congested, the generation of log data may be delayed, the compression rate of the log data may be increased, or the log data may be thinned out.

[0137] [Seventh embodiment] In this embodiment, a display device 2 that delays the generation of log data until the start of a screen switching process when the timing for periodic generation of log data arrives will be described.

[0138] If the display device 2 is equipped with an electronic paper panel, the screen display can continue even when the power is turned off. Therefore, when the display device 2 periodically switches screens as a digital signage, the generation of log data can be overlapped with the screen switching process, making use of the characteristics of the electronic paper panel to reduce power consumption. The screen switching process refers to the display device 2 receiving advertising data from an external PC or the Internet, creating drawing data to be drawn on the display 220, and rewriting the display 220. The advertising data may be stored in advance. The processing load of the screen switching process is smaller than that of handwritten input, so the processing load is unlikely to increase.

[0139] Fig. 21 is a schematic diagram showing a process in which the display device 2 of the present embodiment generates log data in parallel with a screen switching process. In Fig. 21(a), the screen switching process is performed at the timing when the user completes handwriting input. Therefore, the log data generating unit 29 generates log data at the timing when the handwriting input is completed.

[0140] 21B, the screen switching process is not performed when the user completes the handwriting input, so log data generator 29 generates log data after the screen switching process starts.

[0141] In this way, for example, when a digital signage function periodically executes a screen switching process, the log data is generated in parallel with the screen switching process, so power consumption can be reduced more than if the screen switching process and the generation of the log data are performed separately. Since the log data is also generated periodically, the frequency of log data generation is unlikely to decrease even if the log data is generated in parallel with the screen switching process.

[0142] 22 is a flowchart for explaining the process in which the display device 2 generates log data in parallel with the screen switching process and transmits the log data to the server device 10. FIG. 22 is a flowchart showing the transmission process of FIG. 21. The explanation of FIG. 22 will mainly focus on the differences from FIG. 11. The processes of steps S101 and S102 may be similar to steps S11 and S12 of FIG.

[0143] When the handwritten input is completed, the determination unit 28 determines whether or not a screen switching process has started (S103). A method for determining whether or not a screen switching process has started is, for example, a method of inquiring of the OS.

[0144] If the screen switching process is not being performed (No in S103), the log data generating unit 29 waits until the screen switching process starts. If the handwriting input is resumed, the log data generating unit 29 waits until the handwriting input is completed.

[0145] When the screen switching process starts (Yes in S103), the log data generation unit 29 generates log data (S104). The log data transmission unit 30 transmits the generated log data to the server device 10 (S105). The log data transmission unit 30 resets the timer.

[0146] When handwritten input is completed, log data is generated in parallel with the screen switching process, so that in addition to the effect of the first embodiment, power consumption can be reduced.

[0147] In this embodiment, too, the generation of log data may be delayed when the communication network N is congested. In this case, even if the screen switching process starts, the log data generation unit 29 delays the generation of log data when the communication network N is congested. Also, when the communication network N is congested, the compression rate of the log data may be increased or the log data may be thinned out.

[0148] [Other application examples] The above describes the best mode for carrying out the present invention using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0149] For example, the display device 2 may store the log data in its own non-volatile memory or storage medium, instead of transmitting the log data to the server device 10.

[0150] In addition, in the present embodiment, the display device has been described as an example of an electronic whiteboard, but the electronic whiteboard may be called an electronic whiteboard, an electronic information board, or the like. In addition, the present embodiment can be suitably applied to any information processing device having a touch panel. The information processing device equipped with a touch panel may be, for example, an output device such as a projector or digital signage, a HUD (Head Up Display) device, an industrial machine, an imaging device, a sound collecting device, a medical device, a network home appliance, a notebook PC, a mobile phone, a smartphone, a tablet terminal, a game machine, a PDA (Personal Digital Assistant), a digital camera, a wearable PC, or a desktop PC.

[0151] In this embodiment, the coordinates of the pen are detected by detecting the coordinates of the pen tip with a touch panel, but the display device 2 may detect the coordinates of the pen tip with ultrasonic waves. The pen also emits ultrasonic waves together with light, and the display device 2 calculates the distance based on the arrival time of the ultrasonic waves. The display device 2 can identify the position of the pen based on the direction and distance, and the projector draws (projects) the trajectory of the pen as stroke data.

[0152] Furthermore, part of the processing executed by the display device 2 in this embodiment may be executed by a server device connected to the display device 2 via a network.

[0153] Moreover, the configuration examples in Fig. 5 and the like are divided according to main functions in order to facilitate understanding of the processing by the display device 2. The present invention is not limited by the manner in which the processing units are divided or the names of the processing units. The processing by the display device 2 can be divided into more processing units according to the processing contents. Also, it can be divided so that one processing unit includes more processes.

[0154] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above.

[0155] Embodiments of the present invention provide significant improvements in computer power and functionality. These improvements allow users to utilize computers that provide more efficient and robust interaction with tables, which are ways of storing and presenting information in information processing devices. Furthermore, embodiments of the present invention provide a better user experience through the use of more efficient, powerful and robust user interfaces. Such user interfaces provide better interaction between humans and machines.

[0156] <Additional Notes> [Appendix 1] An information processing device that periodically generates log data, an input receiving unit that receives handwritten input from a user; a determination unit that determines whether the input receiving unit is continuing the handwritten input when the log data is generated periodically; a log data generating unit that, when the determination unit determines that the input accepting unit is continuing the handwriting input, does not generate the log data until the input accepting unit completes accepting the handwriting input, and generates the log data when the determination unit determines that the input accepting unit is not continuing the handwriting input; An information processing device having the above configuration. [Appendix 2] 2. The information processing device according to claim 1, wherein the log data includes operation history information regarding the handwritten input accepted by the input accepting unit. [Appendix 3] 3. The information processing device according to claim 2, wherein the log data includes, as the operation history information, coordinates that were handwritten input after the log data was generated last time. [Appendix 4] a determination unit that determines whether a CPU usage rate of the information processing device is equal to or greater than a threshold when generating the log data; When the determination unit determines that the CPU utilization rate is less than a threshold value, the log data generation unit generates the log data; The information processing device according to any one of claims 1 to 3, wherein, when it is determined that the CPU utilization rate is equal to or higher than a threshold, the log data is generated after the CPU utilization rate falls below the threshold. [Appendix 5] a determination unit that determines whether the information processing device is moving when generating the log data; When the determination unit determines that the information processing device is not moving, the log data generation unit generates the log data; The information processing device according to any one of appendices 1 to 3, wherein, when it is determined that the information processing device is moving, the log data is generated after the movement of the information processing device stops. [Appendix 6] a determination unit that determines whether the information processing device is connected to a pay-per-use network or a flat-rate network when generating the log data; When the determination unit determines that the information processing device is connected to a flat-rate charging network, the log data generation unit generates the log data; The information processing device according to any one of claims 1 to 3, wherein, when it is determined that the information processing device is connected to a pay-per-use network, the log data is generated after the information processing device is connected to a flat-rate charging network. [Appendix 7] a determination unit that determines whether or not a priority of a task that transmits the log data is lower than a priority of another task when the log data is generated; When the determination unit determines that the priority of the task that transmits the log data is equal to or higher than the priority of another task, the log data generation unit generates the log data; The information processing device according to any one of claims 1 to 3, wherein, when it is determined that the priority of the task that transmits the log data is lower than the priority of other tasks, the log data is generated after the priority of the task that transmits the log data becomes the same as or higher than the priority of the other tasks. [Appendix 8] a determination unit that determines whether or not the information processing device is connected to a communication network that allows handover when generating the log data; When the determination unit determines that the information processing device is not connected to a communication network that can perform handover, the log data generation unit generates the log data; The information processing device according to any one of claims 1 to 3, wherein, when it is determined that the information processing device is connected to a communication network in which handover is possible, the log data is generated after the information processing device connects to a communication network in which handover is not possible. . [Appendix 9] The determination unit determines whether or not the information processing device is performing a screen switching process when generating the log data; when the determination unit determines that the input reception unit is not continuing handwriting input and that the information processing device is performing screen switching processing, the log data generation unit generates the log data; The information processing device according to any one of claims 1 to 3, wherein when the determination unit determines that the input accepting unit is not continuing handwritten input but that the information processing device is not performing screen switching processing, the log data is generated after the information processing device starts screen switching processing. [Appendix 10] When generating the log data, if a communication network is not congested, the log data generating unit generates the log data; The information processing device according to any one of claims 1 to 9, wherein, when the communication network is congested, the log data is generated after the congestion of the communication network is alleviated. [Appendix 11] When generating the log data, the log data generating unit generates the log data after the input accepting unit completes accepting the handwritten input; The information processing device according to any one of claims 1 to 3, wherein, when the communication network is congested, the log data generation unit increases the compression rate of the log data compared to when the communication network is not congested, or thins out the log data. [Appendix 12] 12. The information processing device according to any one of claims 1 to 11, further comprising a log data transmission unit that transmits the log data generated by the log data generation unit to a server device. [Explanation of symbols]

[0157] 2 Display device 10. Server equipment [Prior art documents] [Patent documents]

[0158] [Patent Document 1] JP 2016-054442 A

Claims

1. An information processing device that periodically generates log data, an input receiving unit that receives handwritten input from a user; a determination unit that determines whether the input receiving unit is continuing the handwritten input when the log data is generated periodically; a log data generating unit that, when the determination unit determines that the input accepting unit is continuing the handwriting input, does not generate the log data until the input accepting unit completes accepting the handwriting input, and generates the log data when the determination unit determines that the input accepting unit is not continuing the handwriting input; An information processing device having the above configuration.

2. The information processing apparatus according to claim 1 , wherein the log data includes operation history information regarding the handwritten input accepted by the input accepting unit.

3. The information processing apparatus according to claim 2 , wherein the log data includes, as the operation history information, coordinates that have been handwritten input after the log data was generated the previous time.

4. a determination unit that determines whether a CPU utilization rate of the information processing device is equal to or greater than a threshold when generating the log data; When the determination unit determines that the CPU utilization rate is less than a threshold value, the log data generation unit generates the log data; 4. The information processing apparatus according to claim 1, wherein, when it is determined that the CPU utilization rate is equal to or higher than a threshold, the log data is generated after the CPU utilization rate falls below the threshold.

5. a determination unit that determines whether the information processing device is moving when generating the log data; When the determination unit determines that the information processing device is not moving, the log data generation unit generates the log data; The information processing device according to claim 1, wherein, when it is determined that the information processing device is moving, the log data is generated after the movement of the information processing device stops.

6. a determination unit that determines whether the information processing device is connected to a pay-per-use network or a flat-rate network when generating the log data; When the determination unit determines that the information processing device is connected to a flat-rate charging network, the log data generation unit generates the log data; 4. The information processing device according to claim 1, wherein, when it is determined that the information processing device is connected to a pay-per-use network, the log data is generated after the information processing device is connected to a flat-rate charging network.

7. a determination unit that determines whether or not a priority of a task that transmits the log data is lower than a priority of another task when the log data is generated; When the determination unit determines that the priority of the task that transmits the log data is equal to or higher than the priority of another task, the log data generation unit generates the log data; An information processing device according to any one of claims 1 to 3, wherein, when it is determined that the priority of a task that transmits the log data is lower than the priority of other tasks, the log data is generated after the priority of the task that transmits the log data becomes the same as or higher than the priority of the other tasks.

8. a determination unit that determines whether or not the information processing device is connected to a communication network that allows handover when generating the log data; When the determination unit determines that the information processing device is not connected to a communication network that can perform handover, the log data generation unit generates the log data; The information processing device according to any one of claims 1 to 3, wherein, when it is determined that the information processing device is connected to a communication network that allows handover, the log data is generated after the information processing device connects to a communication network that does not allow handover.

9. The determination unit determines whether or not the information processing device is performing a screen switching process when generating the log data; when the determination unit determines that the input reception unit is not continuing handwriting input and that the information processing device is performing screen switching processing, the log data generation unit generates the log data; An information processing device according to any one of claims 1 to 3, wherein when the judgment unit determines that the input accepting unit is not continuing handwritten input but that the information processing device is not performing screen switching processing, the log data is generated after the information processing device starts screen switching processing.

10. When generating the log data, if a communication network is not congested, the log data generating unit generates the log data; The information processing apparatus according to claim 1 , wherein, when the communication network is congested, the log data is generated after the congestion of the communication network is alleviated.

11. When generating the log data, the log data generating unit generates the log data after the input accepting unit completes accepting the handwritten input; The information processing device according to any one of claims 1 to 3, wherein, when the communication network is congested, the log data generation unit increases a compression rate of the log data compared to when the communication network is not congested, or thins out the log data.

12. The information processing apparatus according to claim 1 , further comprising a log data transmission unit that transmits the log data generated by the log data generation unit to a server device.

13. A log data generation method performed by an information processing device that periodically generates log data, comprising: The input receiving unit receives handwritten input from a user. a process in which a determination unit determines whether or not the input receiving unit is continuing the handwritten input when the log data is generated periodically; a process in which, when the determination unit determines that the input acceptance unit is continuing the handwriting input, a log data generation unit does not generate the log data until the input acceptance unit has completed acceptance of the handwriting input, and generates the log data when the input acceptance unit determines that the handwriting input is not being continued; A method for generating log data.

14. An information processing device that periodically generates log data, an input receiving unit that receives handwritten input from a user; a determination unit that determines whether the input receiving unit is continuing the handwritten input when the log data is generated periodically; a log data generating unit which, when the determination unit determines that the input accepting unit is continuing the handwriting input, does not generate the log data until the input accepting unit completes accepting the handwriting input, and generates the log data when the determination unit determines that the input accepting unit is not continuing the handwriting input; A program to function as a

15. An information processing system having an information processing device that periodically generates log data and a server device, The information processing device includes: an input receiving unit that receives handwritten input from a user; a determination unit that determines whether the input receiving unit is continuing the handwritten input when the log data is generated periodically; a log data generating unit that, when the determination unit determines that the input accepting unit is continuing the handwriting input, does not generate the log data until the input accepting unit completes accepting the handwriting input, and generates the log data when the determination unit determines that the input accepting unit is not continuing the handwriting input; a log data transmission unit that transmits the log data generated by the log data generation unit to a server device, The server device includes: An information processing system having a log data receiving unit that receives the log data.

Images