Display device, program, conversion method, display system

Abstract

To provide a display device capable of converting handwritten data displayed in a table according to an attribute set for the table.SOLUTION: A display device for displaying a table on a screen is provided, the display device comprising reception means for receiving input of handwritten data in the table, and conversion means configured to convert the handwritten data received by the reception means into a character string according to an attribute associated with the table.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a display device, a program, a conversion method, and a display system.

Background Art

[0002] There are known display devices such as an electronic blackboard in which a user draws a stroke on a touch panel type display with an input means such as a dedicated electronic pen or a finger and displays handwritten data formed by several strokes. A display device equipped with a relatively large touch panel is arranged in a conference room or a public facility and used as an electronic blackboard by a plurality of users.

[0003] The user can also handwrite a table with the input means. There is known a technique for transcribing a handwritten table (see, for example, Patent Document 1). Patent Document 1 discloses an image processing method for detecting intersections and lengths of a handwritten table and transcribing the table and the handwritten data in the table.

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the conventional technique has a problem that it is difficult to convert the handwritten data displayed in the table according to the attributes set in the table.

[0005] In view of the above problems, an object of the present invention is to provide a display device capable of converting handwritten data displayed in a table according to the attributes set in the table.

Means for Solving the Problems

[0006] In view of the above problems, the present invention is a display device that displays a table on a screen, and includes a reception means for receiving an input of handwritten data for the table, and a conversion means for converting the handwritten data received by the reception means into a character string based on the attributes associated with the table. A screen that accepts attribute settings for each cell of the table, and UI image generation means for outputting a screen including the cell and the setting information of the attribute of the cell , Operation reception means for accepting settings of different attributes for each cell of the table, It is characterized by having. [Effect of the Invention]

[0007] It is possible to provide a display device capable of converting handwritten data displayed in a table according to attributes set in the table. [Brief Description of the Drawings]

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Embodiments for Carrying Out the Invention

[0009] Hereinafter, as an example of the embodiments for carrying out the present invention, the display device 2 and the conversion method in the display device 2 will be described with examples.

Examples

[0010] <Outline of Transcription of Handwritten Data in a Table According to Table Attributes> FIG. 1 is a diagram for explaining an outline of a fair copy of a table according to the attributes of the table. As shown in FIG. 1(a), the user has generated a table on the display device 2 in which character strings and handwritten data are mixed.

[0011] In the prior art, there has been a problem that it is difficult to convert handwritten data displayed in a table according to the attributes set for the table. For example, when a plurality of users handwrite on a table, handwritten data with different font sizes and colors exists in one table. In this case, even if the display device simply makes a fair copy of the handwritten data, it can be converted into character strings with different character sizes and colors. As a result, the readability of the table may sometimes hinder the discussion.

[0012] Therefore, in the present embodiment, the user operates a menu or the like to execute a fair copy, or the display device 2 automatically executes a fair copy. The display device 2 makes a fair copy of the table according to the attributes of the table set in advance.

[0013] If there is handwritten data, the display device 2 first performs OCR processing to convert it into a character string. Then, the display device 2 makes a fair copy of the character string according to the attributes (font, size, arrangement, color, etc.) set for the table and displays it. Further, if there is already a character string, the display device 2 makes a fair copy of the character string according to the attributes and displays it.

[0014] FIG. 1(b) shows an example of a table that has been made fair according to the attributes of the table. In FIG. 1(b), the character size of each cell has been made fair according to the attributes, and further, each cell has been made fair and displayed in the same color.

[0015] As described above, since the display device 2 of the present embodiment makes a fair copy of the table according to the attributes, even when a plurality of users handwrite with different handwriting styles, a table with a unified feeling can be created. Since the table becomes easier to view, discussions and the like can be deepened more easily.

[0016] <Regarding terms> The input means may be any means that can handwrite by designating coordinates on the touch panel. For example, there are an electronic pen, a person's finger or hand, a rod-shaped member, and the like.

[0017] A series of operations where a user presses and continuously moves an input means on a display and then releases it from the display is called a stroke. Stroke data is information displayed on the display based on the trajectory of coordinates input by the input means. The stroke data may be interpolated as appropriate. Handwritten data is data having one or more stroke data. Handwritten input indicates that handwritten data is input by the user.

[0018] An object displayed on the display based on stroke data is called an object. Although an object means a target, in this embodiment, it means a display target or the like. For example, there are hand-drawn data, tables, figures, images, characters, and the like.

[0019] The character string obtained by recognizing and converting handwritten data may include, in addition to text data, fixed characters or marks such as "completed" displayed as stamps, figures such as circles and stars, straight lines, etc., and data displayed based on the user's operations.

[0020] A character string is one or more characters handled by a computer. The actual form of a character string is a character code. Characters include numbers, alphabets, symbols, and the like. A character string is also called text data.

[0021] Conversion means converting handwritten data into a character code and displaying the character code in a predetermined font. It also means displaying a character string in a manner according to its attributes. In addition, conversion includes converting handwritten data into figures such as straight lines, curves, rectangles, or tables. In this embodiment, conversion is described as transcription.

[0022] Attributes are the modes of information displayed in a table, such as the font, font size, arrangement, or color of characters in the table, and the modes of the table itself, such as the color and thickness of the table's grid lines.

[0023] <System Configuration Example> FIG. 2 is an example of an overall configuration diagram of a communication system using a display device. In FIG. 2, for the sake of simplicity of explanation, only two display devices 2a and 2b and associated electronic pens 4a and 4b are shown, but three or more display devices 2 and electronic pens or the like may be used.

[0024] As shown in FIG. 2, the communication system 1 includes a plurality of display devices 2a and 2b, a plurality of electronic pens 4a and 4b, USB memories 5a and 5b, notebook PCs (Personal Computers) 6a and 6b, TV (video) conference terminals 7a and 7b, and a PC 8. Also, the display devices 2a and 2b and the PC 8 are communicably connected via a communication network 9. Further, the plurality of display devices 2a and 2b are each provided with a display 3a and 3b.

[0025] In addition, the display device 2a can display on the display 3a an image drawn by an event (touch of the pen tip or pen butt of the electronic pen 4a on the display 3a) generated by the electronic pen 4a. Note that not only the electronic pen 4a but also an image displayed on the display 3a can be changed based on an event (gesture such as zooming in, zooming out, page turning, etc.) generated by the user's hand Ha or the like.

[0026] In addition, a USB memory 5a can be connected to the display device 2a, and the display device 2a can read an electronic file such as a PDF from the USB memory 5a, or the display device 2a can record an electronic file in the USB memory 5a. Also, a notebook PC 6a is connected to the display device 2a via a cable 10a1 capable of communication according to standards such as DisplayPort (registered trademark), DVI (Digital Visual Interface), HDMI (High-Definition Multimedia Interface. Registered trademark), and VGA (Video Graphics Array). Then, the display device 2a generates an event by contact with the display 3a, and transmits event information indicating this event to the notebook PC 6a in the same manner as an event from an input device such as a mouse or a keyboard. Similarly, a TV (video) conference terminal 7a is connected to the display device 2a via a cable 10a2 capable of communication according to the above standards. Note that the notebook PC 6a and the TV conference terminal 7a may communicate with the display device 2a by wireless communication compliant with various wireless communication protocols such as Bluetooth (registered trademark).

[0027] On the other hand, at another site where the display device 2b is installed, a display device 2b equipped with a display 3b, an electronic pen 4b, a USB memory 5b, a notebook PC 6b, a TV conference terminal 7b, a cable 10b1, and a cable 10b2 are used in the same manner as above. Furthermore, based on an event generated by the user's hand Hb or the like, the image displayed on the display 3b can also be changed.

[0028] As a result, the image drawn on the display 3a of the display device 2a at one site is also displayed on the display 3b of the display device 2b at another site. Conversely, the image drawn on the display 3b of the display device 2b at another site is displayed on the display 3a of the display device 2a at one site. Thus, in the communication system 1, since remote sharing processing for sharing the same image at a remote location can be performed, it is very convenient when used for a meeting or the like at a remote location.

[0029] Hereinafter, when any one of the plurality of display devices 2 is shown, it is denoted as "display device 2". When any one of the plurality of displays is shown, it is denoted as "display 3". When any one of the plurality of electronic pens is shown, it is denoted as "electronic pen 4". When any one of the plurality of USB memories is shown, it is denoted as "USB memory 5". When any one of the plurality of notebook PCs is shown, it is denoted as "notebook PC 6". When any one of the plurality of video conferencing terminals is shown, it is denoted as "video conferencing terminal 7". Also, when any one of the hands of a plurality of users is shown, it is denoted as "hand H". When any one of the plurality of cables is shown, it is denoted as "cable 10".

[0030] In addition, in this embodiment, as an example of the display device 2, an electronic blackboard will be described, but it is not limited thereto. As another example of the display device 2, the display device 2 may be an electronic signboard (digital signage), a telestereo used in sports, weather forecasts, etc., or a remote image (video) diagnostic device, etc. Also, an example of the information processing terminal is not limited to the notebook PC 6. As another example of the information processing terminal, the information processing terminal may be a desktop PC, a tablet PC, a PDA (Personal Digital Assistant), a digital video camera, a digital camera, a terminal capable of supplying an image frame such as a game machine, etc. Furthermore, the communication network includes the Internet, a LAN (Local Area Network), a mobile phone communication network, etc. Also, in this embodiment, as an example of the recording medium, a USB memory will be described, but it is not limited thereto, and the recording medium may be various recording media such as an SD card.

[0031] <Hardware Configuration of Display Device> Subsequently, with reference to FIG. 3, the hardware configuration of the display device 2 of this embodiment will be described. Note that FIG. 3 is a hardware configuration diagram of the display device 2.

[0032] As shown in FIG. 3, the display device 2 includes a CPU 101 that controls the operation of the entire display device 2, a ROM 102 that stores programs used for driving the CPU 101 such as IPL, a RAM 103 used as a work area for the CPU 101, an SSD 104 that stores various data such as programs for the display device 2, a network controller 105 that controls communication with the communication network 9, and an external storage controller 106 that controls communication with the USB memory 5.

[0033] The SSD 104 stores various data such as an OS and programs for the display device 2. Note that this program may be an application program that operates on an information processing device equipped with a general-purpose OS (such as Windows (registered trademark), Mac OS (registered trademark), Android (registered trademark), iOS (registered trademark), etc.). That is, the display device 2 may be a general-purpose information processing device.

[0034] In addition, the display device 2 includes a capture device 111 that causes the display of the notebook PC 6 to display video information as a still image or a moving image, a GPU (Graphics Processing Unit) 112 that specializes in graphics, and a display controller 113 that controls and manages screen display in order to output the output image from the GPU to the display 3 or the video conferencing terminal 7.

[0035] Furthermore, the display device 2 includes a sensor controller 114 that controls the processing of the contact sensor 115, and a contact sensor 115 that detects contact of an electronic pen 4, the user's hand H, etc. on the display 3. This contact sensor 115 performs coordinate input and coordinate detection by an infrared blocking method. The method for this coordinate input and coordinate detection is, for example, the infrared blocking method. In this method, two light emitting and receiving devices installed at both upper ends of the display 3 emit a plurality of infrared rays parallel to the display 3. A reflecting member provided around the display 3 reflects the infrared rays, and the light receiving element receives the light that returns on the same optical path as the optical path of the emitted light. The contact sensor 115 outputs the blocking position blocked by the object to the sensor controller 114, and the sensor controller 114 specifies the coordinate position that is the contact position of the object by triangulation.

[0036] Also, the contact sensor 115 is not limited to the infrared blocking method, and various detection means such as a capacitance type touch panel that specifies the contact position by detecting a change in capacitance, a resistive film type touch panel that specifies the contact position by a voltage change between two opposing resistive films, or an electromagnetic induction type touch panel that detects electromagnetic induction generated by contact of a contact object with the display unit and specifies the contact position may be used.

[0037] Also, the display device 2 includes an electronic pen controller 116. This electronic pen controller 116 determines the presence or absence of pen tip touch and pen butt touch on the display 3 by communicating with the electronic pen 4. Note that the electronic pen controller 116 may be configured to determine the presence or absence of touch not only at the pen tip and pen butt of the electronic pen 4 but also at the portion held by the user of the electronic pen 4 and other portions of the electronic pen.

[0038] Furthermore, the display device 2 includes a CPU 101, a ROM 102, a RAM 103, an SSD 104, a network controller 105, an external memory controller 106, a capture device 111, a GPU 112, a sensor controller 114, and an electronic pen controller 116, and has bus lines 120 such as an address bus and a data bus for electrically connecting them as shown in FIG. 3.

[0039] <Functional Configuration of Display Device> Subsequently, the functional configuration of the display device 2 will be described with reference to FIG. 4. FIG. 4 is a functional block diagram of the display device 2.

[0040] The display device 2 has each functional configuration shown in FIG. 4 by the hardware configuration and program shown in FIG. 3.

[0041] The video acquisition unit 21 acquires the video output from the notebook PC 6 connected to the display device via a cable. Further, the video acquisition unit 21 analyzes the acquired video and extracts image information including the resolution and update frequency of the image frame. The image information is output to the image acquisition unit 31 in the image processing unit 30.

[0042] When an electronic pen or the user's hand touches the display, the coordinate detection unit 22 detects the contact position coordinates as the event occurrence position. The detection result is output to the event distribution unit 25.

[0043] The automatic adjustment unit 23 is automatically activated when the power of the display device 2 is turned on, and adjusts the parameters of the contact sensor so that the contact sensor can output appropriate values to the coordinate detection unit 22.

[0044] The contact detection unit 24 detects whether the tip or rear end of the electronic pen is pressed against the display by receiving a signal that the tip or rear end of the electronic pen is pressed against the display. The detection result is output to the event distribution unit 25.

[0045] The event distribution unit 25 determines the type of event based on the position coordinates detected by the coordinate detection unit 22 and the detection result of the contact detection unit 24. Here, the events include "stroke input", "text input", "UI operation", and "table operation". Note that the position coordinates detected by the coordinate detection unit 22 and the detection result of the contact detection unit 24 are also referred to as "event information". The event distribution unit 25 outputs the event information to the stroke processing unit 32, the text processing unit 35, the table processing unit 33, and the operation processing unit 26 according to the detection result of the event.

[0046] The operation processing unit 26 receives the event information determined as a "UI operation" and requests the image processing unit 30 to perform processing according to the UI element where the event occurred. UI is short for User Interface, mainly referring to menus, buttons, etc. UI operations include various operations such as changing the color of handwritten data, zooming in and out, table operations, and navigation of UI menus.

[0047] The communication control unit 27 controls communication with other display devices 2, PCs, or servers via a communication network. The display control unit 28 displays the image generated by the display overlay unit 38 on the display. The image may include UI, character strings, handwritten data, tables, images, and backgrounds.

[0048] The image processing unit 30 further has the following functions. The image processing unit 30 includes an image acquisition unit 31, a stroke processing unit 32, a table processing unit 33, a table attribute holding unit 34, a text processing unit 35, a UI image generation unit 36, a background generation unit 37, a display overlay unit 38, a page processing unit 39, a page data storage unit 40, a file processing unit 41, a license management unit 42, and a layout management unit 43. Details will be described later.

[0049] The image acquisition unit 31 acquires the frame information included in the video acquired by the video acquisition unit 21 as an image and outputs it to the display overlay unit 38 and the page processing unit 39.

[0050] The stroke processing unit 32 receives event information determined as "stroke input" and performs operations such as drawing stroke data, deleting the drawn stroke data, and editing. The results of each operation are output to the display overlay unit 38 and the page processing unit 39.

[0051] The table processing unit 33 receives event information determined as "table operation" and generates a table. Also, the table processing unit 33 deletes the character strings and handwritten data already drawn in the table and generates character strings according to the attributes. The results of each operation are output to the display overlay unit 38 and the page processing unit 39. The cell table attribute holding unit 34 held by 34 holds the attributes of the table. The attributes of the table include the attributes of the entire table and the attributes of the cells. Also, the table attribute holding unit 34 holds the initial values of each attribute. The initial value is an attribute that is set in advance without the user setting it. The attributes of the table will be described in Tables 3 and 4.

[0052] The table processing unit 33 can convert (transcribe) the handwritten data drawn by hand on the table into character strings by OCR. Also, the table processing unit 33 can convert (transcribe) the font etc. of the character strings displayed on the table into another font according to the attributes of the table.

[0053] The text processing unit 35 receives event information determined as "text input" and generates a character string. Text input means that characters are input with a soft keyboard. Also, the text processing unit 35 deletes and edits the character strings already displayed. The results of each operation are output to the display overlay unit 38 and the page processing unit 39.

[0054] The UI image generation unit 36 generates a preset UI image and outputs it to the display overlay unit 38. The UI image is, for example, menus and buttons for the user to perform various settings.

[0055] The background generation unit 37 receives the media data included in the page data called by the page processing unit 39 from the page data storage unit 40 from the page processing unit 39, and outputs it to the display overlay unit 38. The media data serves as the background.

[0056] The layout management unit 43 manages information regarding the layout of each image (the stacking order and display / non-display of each layer) output from the image acquisition unit 31, the stroke processing unit 32, the table processing unit 33, the UI image generation unit 36, and the background generation unit 37. The layout information is output to the display overlay unit 38.

[0057] Based on the layout information from the layout management unit 38, the display overlay unit 38 determines the layout of the images output from the image acquisition unit 31, the stroke processing unit 32, the table processing unit 33, the UI image generation unit 36, and the background generation unit 37.

[0058] The page processing unit 39 combines the stroke data, text data, data related to table data, and video data from the PC into one page and stores it in the page data storage unit 40. The page data storage unit 40 stores the page data. The page data held by the page data storage unit 40 will be described in Tables 1 and 2.

[0059] The file processing unit 41 converts the PDF file read from the USB into an image. The file processing unit 41 generates a PDF file from the displayed image. The license management unit 42 manages the license regarding the use right of the display device 2.

[0060]

Table 1

[0061] · The object ID is identification information for identifying an object within the screen. An object refers to all various display items shown on the screen.

[0062] · The type is the kind of object. For example, there are tables (rows or columns), handwritten, characters, graphics, images, etc. A table indicates a table object. Handwritten is stroke data (a sequence of coordinate points). Characters are character strings (character codes) converted from handwritten data. Character strings are sometimes referred to as text data. Graphics are geometric shapes converted from handwritten data such as triangles and rectangles. Images are image data such as Jpeg, Png, Tiff, etc. imported from a PC, the Internet, etc. The data of each object is managed in association with the object ID.

[0063] · Coordinates indicate the position of the starting point of an object based on a predetermined origin of the screen of the display device 2. The starting point of an object is, for example, the upper left vertex of the circumscribed rectangle of the drawing object. Coordinates are represented, for example, in pixel units of the display.

[0064] · Size is the magnitude (width and height) of each object. The position of the ending point may also be recorded.

[0065]

Table 2

[0066]

Table 3

[0067] · The number of rows is the number of rows.

[0068] · The number of columns is the number of columns.

[0069] · "Clear the screen simultaneously with input" is a setting for whether the display device 2 clears the screen according to the table attributes immediately after the user inputs a character string or handwritten data. When "Clear the screen simultaneously with input" is True, the hand-drawn data or character string is cleared, including the OCR processing of the handwritten data. However, if "Automatically convert the character string by OCR" in Table 4 is off, the OCR processing is not performed automatically.

[0070]

Table 4

[0071] · Font is the font of the character string displayed in the cell.

[0072] · Size is the size of the character string displayed in the cell.

[0073] · Color is the color of the character string displayed in the cell.

[0074] · Background color is the color of the cell.

[0075] · Alignment is the position of the characters in the cell (right-aligned / left-aligned / centered, top-aligned / bottom-aligned / centered).

[0076] · "Automatically convert text using OCR" is a setting for whether to automatically convert handwritten data into text. By "automatically", it means that when the electronic pen 4 is lifted from the display and a certain amount of time has passed, OCR is executed without the user inputting an OCR start operation.

[0077] · "Automatically adjust cells to fit the text" is a setting for whether to automatically adjust the size of cells to match the text size within the cells (number of characters × size of one character).

[0078] · The title is a setting for whether a cell is a title. For cells set as titles, the font, background color, etc. may be set based on the title attributes.

[0079] <Regarding events> Next, referring to FIG. 5, the events detected by the display device 2 will be described. FIG. 5 is a diagram for explaining the operations detected as events.

[0080] FIG. 5(a) is a diagram for explaining UI operation events. UI operation events are events detected when the UI is displayed on the display and the user presses a predetermined position with the electronic pen or finger. The user can change the color and width of the electronic pen by UI operations. Also, the user can switch the input of stroke data, text data, and table data by UI operations.

[0081] FIG. 5(b) is a diagram for explaining stroke input events. Stroke input events are events detected when the user presses the electronic pen against the display, moves the electronic pen in this state, and finally lifts the electronic pen from the display. Also, stroke input events are events where the user deletes or moves strings that have already been placed.

[0082] FIG. 5(c) is a diagram for explaining a text operation event. The text operation event is an event detected when a user inputs data represented by a character code such as a character string using a soft keyboard or the like. Further, the text operation event is an event in which a user deletes or edits a string that has already been arranged.

[0083] FIG. 5(d) is a diagram for explaining a table operation event. The table operation event is an event detected when a user inputs a table, or inputs handwritten data or a character string into a table. When, for example, stroke data is superimposed and hand-drawn on a square figure, the event distribution unit 25 determines it as a table operation event, and the table processing unit 33 divides the square and converts it into table data (details will be described later). Similarly, the event distribution unit 25 determines that it is a handwritten input into the table from the coordinates of the handwritten stroke data, and the table processing unit 33 transcribes the handwritten data according to the attributes. The table operation event also includes an event detected when a user operates (moves, enlarges / reduces, deletes) a table that is already displayed.

[0084] <Input of table> FIGS. 6 to 8 are diagrams for explaining a method of inputting a table. FIG. 6 is a diagram for explaining the UI of the display device 2. The display device 2 has a setting button 201 and a drawing panel 202 as a UI. The setting button 201 will be described later, and it is a button for a user to set an initial value of the table attributes.

[0085] The drawing panel 202 has buttons such as a pen color 202a, a marker 202b, an eraser tool 202c, a ruler 202d, a handwritten input 202e, a table creation 202f, a restore 202g, and a redo 202h.

[0086] FIG. 7 shows an example of a table 211 handwritten by a user on a display. The detection method of the table 211 includes a method of determining the shape (FIG. 7(a)) and a method of detecting the intersection of straight lines (FIG. 7(b)).

[0087] In the method of determining the shape, it is assumed that the on / off slide button 351 of "converting ink into a figure" described later is on. The event distribution unit 25 detects the shape of a square by pattern matching or machine learning (Fig. 7(a-1)), and the display control unit 28 displays the square of the figure (Fig. 7(a-2)). When a straight line is handwritten overlapping the square (Fig. 7(a-3)), the event distribution unit 25 determines it as a table operation event and clears the table (Fig. 7(a-4)). When the user creates a table based on the square, it is not necessary (or it may be turned on) to turn on the table creation 202f.

[0088] When the user wants to convert a plurality of straight lines into a table, the on / off slide button of "converting ink into a table" 352 described later is on. Alternatively, it is preferable to turn on the table creation 202f. This is because the user may sometimes want to leave the stroke data as it is. In the method of detecting the intersection of straight lines, the event distribution unit 25 divides the stroke data into horizontal lines and vertical lines, and there is a method of detecting the intersection. The event distribution unit 25 assigns numbers to the intersections of each horizontal line from the left (or right), and determines whether the intersections with the same number on each horizontal line are approximately the same at their X coordinates. Similarly, the event distribution unit 25 assigns numbers to the intersections of each vertical line from the top (or bottom), and determines whether the intersections with the same number on each vertical line are approximately the same at their Y coordinates. When both are approximately the same, the event distribution unit 25 detects a table operation event.

[0089] Fig. 8 is a diagram showing the cleared table 212 by the table processing unit 33. In the method where the square is displayed first, the table processing unit 33 deletes the stroke data overlapping the table, and divides the square based on the coordinates of the stroke data to create the table 212.

[0090] In the method of detecting a plurality of straight lines, the table processing unit 33 erases the stroke data constituting the table 211 and displays the table 212 transcribed based on the stroke data. For example, the table processing unit 33 obtains the circumscribed rectangle of all the stroke data detected as a table operation event, and divides the circumscribed rectangle by the number of horizontal and vertical lines of the stroke data. The positions of the horizontal and vertical lines are determined based on the above intersections.

[0091] By doing so, the table is transcribed. However, conventionally, when multiple people handwrote on this table, it may have been difficult to view as a table.

[0092] In addition, in FIGS. 7 and 8, the table handwritten by the user is transcribed, but the table may originally have been transcribed. That is, when the user specifies the number of rows and columns to display a table, a table with cells of a certain height and width is displayed. The user can arbitrarily change the height and width of the displayed table.

[0093] <Example of Transcription According to Table Attributes> Next, with reference to FIGS. 9 to 12, the transcription according to the table attributes will be described. FIG. 9 shows the handwritten data 222 or the character string 223 input inside the transcribed table 221. In FIG. 9, the handwritten data 222 and the character string 223 are mixed in one table.

[0094] Regarding the method of inputting the character string 223 into the table, there is a method in which the user designates a cell with the electronic pen 4 and inputs an arbitrary character string 223 with the soft keyboard. Alternatively, there is a method in which the user inputs an arbitrary character string 223 with the soft keyboard and then moves the character string 223 to an arbitrary cell. In FIG. 9, the positions of the character strings 223 differ depending on the cells.

[0095] When the user wants to clear the handwritten data on the table, the electronic pen 4 is long-pressed on the table. The event distribution unit 25 determines that it is a UI operation event based on the coordinates pressed by the electronic pen 4, and notifies the operation processing unit 26. Since the electronic pen 4 has been long-pressed on the table, the operation processing unit 26 requests the UI image generation unit 36 to provide a menu for table operations. The UI image generation unit 36 displays a menu for table operations.

[0096] Figure 10 shows an example of a menu 231 for table operations. The menu 231 for table operations has a clear button 232 for the table. The clear button 232 for the table is a button for the user to instruct the display device 2 to clear the handwritten data 222 and the character string 223 input on the table. That is, the object to be cleared may be either handwritten data or a character string. In the case of handwritten data 222, the clearing includes OCR and conversion according to the attributes of the table. In the case of the character string 223, the clearing is a conversion such as a font according to the attributes of the table.

[0097] Since the pressing of the clear button 232 for the table is a UI operation event, the event distribution unit 25 notifies the operation processing unit 26 that the table is to be "cleared". The operation processing unit 26 requests the table processing unit 33 to clear the table.

[0098] Figure 11 is an example of a flowchart diagram for explaining the processing performed by the display device 2 when the clear button for the table is pressed. In Figure 11, it is executed when "clear simultaneously with input" of the overall table attributes is True, or when the user executes the clear button 232 from the menu.

[0099] The table processing unit 33 obtains the attributes of the table (overall table attributes and cell attributes) selected by the user from the table attribute holding unit 34 (S1).

[0100] The table processing unit 33 determines whether the setting of "automatically convert to character string by OCR" for the cell attributes is True (S2).

[0101] If the determination in step S2 is Yes, the table processing unit 33 determines whether there is handwritten data in the table (S3). The type (stroke data, character string, image) and coordinates of each data input to the display device 2 are stored in the page data storage unit 40.

[0102] If there is handwritten data, the table processing unit 33 converts the handwritten data into a character string using OCR (S4). Note that the timing at which step S4 is executed is the timing when a certain period of time has elapsed after the user separates the electronic pen 4 from the display.

[0103] Then, the table processing unit 33 transcribes the character strings of each cell according to the attributes of the table (S5). That is, the table processing unit 33 changes according to the attributes of the cells for which the font, font size, color, background color, and layout are set. The table processing unit 33 converts the handwritten data into a character string, and further converts the converted character string based on the attributes associated with the table.

[0104] FIG. 12 shows an example of the table after transcription displayed by the above processing. Comparing with FIG. 9, the handwritten data 222 is converted into the character string 241. The character string 223 is converted into the character string 242 after transcription. By transcription, the sizes of the character strings 241 and 242 in each cell are the same, and the horizontal and vertical layouts within the cell are also centered.

[0105] As described above, the display device 2 of the present embodiment can transcribe the handwritten data and character strings input to the table according to the attributes of the table.

[0106] <Setting of Attributes> Subsequently, with reference to FIGS. 13 to 21, the change of the attributes of the table or cells will be described. FIG. 13 is a diagram for explaining the setting of the attributes of the table. As shown in FIG. 13, when the user long-presses the frame of the table with the electronic pen 4, the UI image generation unit 36 displays the table operation menu 231 via the event distribution unit 25 and the operation processing unit 26.

[0107] The table operation menu 231 has a table setting button 251. When the user presses the table setting button 251, a table setting screen 252 is displayed. The table setting screen 252 has an on / off slide button group 260, a list of table designs 270, an on / off slide button 281, an on / off slide button 301, a font setting field 311, a color palette 316, and an arrangement setting 320. Hereinafter, the table setting screen 252 will be described in detail.

[0108] Figure 14 is a diagram for explaining the details (part 1) of the table setting screen. In Figure 14, on / off slide buttons are respectively displayed for the item names. · The title row 261 is an on / off slide button for the user to set whether to set the first row as the title row. · The first column 262 is an on / off slide button for the user to set whether to use the first column as the title column. · The total row 263 is an on / off slide button for the user to set whether to use the last row as the total row. · The last column 264 is an on / off slide button for the user to set whether to use the last column as the total column. · The stripe pattern (row) 265 is an on / off slide button for the user to set whether to use the title row with a stripe pattern. · The stripe pattern (column) 266 is an on / off slide button for the user to set whether to use the title column with a stripe pattern.

[0109] In this way, the user can change the cell attributes related to the title. In accordance with the user's settings in Figure 14, the title of the cell attributes is changed.

[0110] FIG. 15 is a diagram for explaining details (Part 2) of the table setting screen. In FIG. 15(a), a list 270 of table designs is shown. For drawing convenience, FIG. 15(a) is in black and white, but each table design is designed in a color scheme of monochrome, blue, red, gray, yellow, light blue, and green. The list of table designs has a pull-down menu 271. When the user presses the pull-down menu 271, the list of designs in FIGS. 15(b) to 15(e) is displayed. FIG. 15(b) shows a design for documents, FIG. 15(c) shows a light-color design, FIG. 15(d) shows an intermediate (in terms of color intensity) design, and FIG. 15(e) shows a dark-color design, respectively.

[0111] The color and background color of the cell attributes are changed according to the table design selected by the user in FIG. 15.

[0112] FIG. 16 is a diagram for explaining details (Part 3) of the table setting screen. FIG. 16 shows an on / off slide button 281 associated with "Adjust cell size according to text". This on / off slide button 281 is a button for the user to set whether to automatically adjust the cell size according to the text in the cell.

[0113] FIG. 17 is a diagram for explaining the on / off slide button 281 of "Adjust cell size according to text". FIG. 17(a) shows the table before transcription, and FIG. 17(b) shows the table after transcription. When the size of the cell string in FIG. 17(a) is changed according to the cell attributes, all the text cannot fit into the cell.

[0114] In this case, when the on / off slide button 281 of "Automatically adjust cell according to text" is off (False), the table processing unit 33 does not enlarge the cell according to the text size. Enlarging means the horizontal width of the cell. As shown in FIG. 17(b), the character string 291 of "Usage frequency" becomes two lines, and the height of the cell increases so that the two-line character string can be accommodated.

[0115] Figure 17(c) shows a proofreading example when the on / off slide button 281 for "Automatically adjust cells according to text" is on (True). When "Automatically adjust cells according to text" is True, the table processing unit 33 enlarges the cells according to the size of the character string. As shown in Figure 17(c), the horizontal width of the cell has increased so that the character string 291 "Usage frequency" fits on one line.

[0116] Note that since the size (vertical and horizontal) of the characters is determined by the font and character type (Japanese, alphabet, numbers), the table processing unit 33 compares the total size of the character string with the size of the cell and can determine whether line breaks are necessary or size changes are necessary according to the setting of "Automatically adjust cells according to text". When line breaks are necessary, the table processing unit 33 determines the necessary number of lines from the number of characters that can fit on one line and the total number of characters, calculates the height of this cell, and corrects the height of the cell. When not performing line breaks, the table processing unit 33 determines the necessary horizontal length of the cell from the total number of characters and corrects the horizontal length of the cell.

[0117] The content set by the user with the on / off slide button 281 in Figure 16 is set to "Automatically adjust cells according to text" in the cell attributes.

[0118] Figure 18 is a diagram for explaining the details (Part 4) of the table setting screen. Figure 18 shows an on / off slide button 301 associated with "Automatically convert text by OCR". This on / off slide button 301 is a button for the user to set whether the display device 2 automatically performs OCR on the handwritten data in the cell.

[0119] Since OCR may cause misrecognition, some users may not want to use it. In that case, by turning off the on / off slide button 301 associated with "Automatically convert text by OCR", the display device 2 does not perform OCR and only performs position adjustment.

[0120] In Fig. 18, the content set by the user using the on / off slide button 301 is set to "Automatically convert characters by OCR" in the cell attributes.

[0121] Fig. 19 is a diagram for explaining the details (Part 5) of the table setting screen. In Fig. 19, a font setting field 311 is shown. · The pull-down menu 312 can be used for the user to select a font. · The pull-down menu 313 can be used for the user to select a character size. · The size increase button 314 is a button for the user to increase the character size by one step. · The size decrease button 315 is a button for the user to decrease the character size by one step. · The color palette 316 is a list of colors for the user to set the color of the characters. Regarding the selection of colors, as shown in Fig. 20, the user may be able to select from the color palette 317.

[0122] The information set in Fig. 19 or Fig. 20 is set to the font, font size, and color of the cell attributes.

[0123] Fig. 21 is a diagram for explaining the details (Part 6) of the table setting screen. In Fig. 21, a string arrangement setting 320 is displayed. Fig. 21(a) is a list for the user to select the horizontal arrangement of the string within the cell. The user can select left alignment 321, center alignment 322, or right alignment 323.

[0124] Fig. 21(b) is a list for the user to select the vertical arrangement of the string within the cell. The user can select top alignment 324, center alignment 325, or bottom alignment 326. The information set in Fig. 21 is set to the arrangement of the cell attributes.

[0125] <Setting of Cell Attributes> As described above, the user can set attributes common to each cell. And further, the user can set individual attributes for each cell.

[0126] Referring to FIG. 22, the setting of the attributes of individual cells will be described. FIG. 22 shows an example of the attribute setting screen 332 of individual cells. When the user long-presses the electronic pen 4 on an arbitrary cell, the UI image generation unit 36 displays the table operation menu 330. The table operation menu 330 has a cell setting button 331. When the user presses the cell setting button 331, the attribute setting screen 332 of the cell selected by the user is displayed.

[0127] The cell attribute setting screen 332 has items such as "Font", "Resize cell to fit text", and "Text alignment". These setting methods may be the same as those of the table setting screen 252 in FIG. 13. The setting method of the "Background color" 340 may be the same as that of the color palette 316.

[0128] In this way, the user can select a cell and set different attributes for each cell.

[0129] <Initial value setting> The display device 2 has initial values of the table attributes in advance. That is, even if the user does not set the attributes, the display device 2 can write neatly with the same character size for each cell.

[0130] FIG. 23 shows the setting button 201 that the user presses for initial value setting. FIG. 24 is an example of various setting screens 350 displayed by pressing the setting button 201. Each item of the various setting screens 350 will be described. · "Convert ink to shape" 351 is an on / off button for the user to set whether to convert handwritten data to a shape. · "Convert ink to table" 352 is an on / off button for the user to set whether to convert handwritten data to a table. What kind of handwritten data is converted to a table is predetermined. · "Clear the table simultaneously with input" 353 is an on / off button for the user to set "Automatically convert the character string by OCR" of the cell attribute. · "Object Alignment" 354 is an on / off button for the user to set whether to align the arrangement of nearby objects. · "Table Type" 355 is a button for the user to set the initial values of the table attributes. When the user presses "Table Type" 355, the initial setting screen is displayed.

[0131] Figure 25 shows an example of the initial setting screen 360. The initial setting screen 360 in Figure 25 has the same configuration as the table setting screen 252 in Figure 13. The user can set the initial values of the table attributes from the initial setting screen 360.

[0132] The "Clear the table simultaneously with input" 353 on the various setting screens 350 in Figure 24 will be described. When the on / off slide button of "Clear the table simultaneously with input" 353 is set to on, when the following operations are detected, according to the table attributes, the table processing unit 33 immediately clears the data in the table. · Input of handwritten data · Drop of handwritten data · Input of a character string · Drop of a character string That is, the user does not need to display or press the clear button, and the table can be cleared by these operations. However, for the conversion of handwritten data to a character string, "Automatically convert the character string with OCR" needs to be on.

[0133] <Main Effects> As described above, since the display device 2 of the present embodiment clears the table according to the attributes, even when multiple users write in different handwriting styles, the table can be cleared into a unified table. It becomes easier to view as a table, so discussions and the like can be deepened more easily.

Example

[0134] In this example, a display system 19 in which the display device 2 and the terminal device 11 communicate via the server device 12 will be described.

[0135] FIG. 26 is an example of a schematic configuration diagram of the display system 19. The display device 2 and the terminal device 11 are each connected to the server device 12 via a network such as the Internet. The terminal device 11 only needs a Web browser or a dedicated application to operate. For example, it can be a PC, a smartphone, a tablet terminal, a game machine, a PDA (Personal Digital Assistant), a wearable PC (Personal Computer), a car navigation device, etc.

[0136] The function of the display device 2 may be the same as that in the first embodiment except for communicating with the server device 12. The communication control unit 27 of the display device 2 transmits object data (tables and strings in the tables) to the server device 12. Since the server device 12 transmits a URL for a meeting that the display device 2 and the terminal device 11 can commonly access to the display device 2, the display device 2 transmits the URL to the terminal device 11 by email or the like. As a result, the terminal device 11 can communicate with the server device 12 and receive the object data transmitted by the display device 2. When the terminal device 11 displays an object with a Web browser, the server device 12 generates screen information in HTML or the like. When the terminal device 11 displays an object with a dedicated application, the server device 12 mainly needs to transmit the object and coordinates to the terminal device 11 as screen information.

[0137] Also, when the terminal device 11 has a touch panel, the user can use the terminal device 11 in the same way as the display device 2 to input or clear handwritten data in the table. The cleared object data is transmitted to the display device 2 via the server device 12.

[0138] In this way, according to the display system 19, the display device 2 and the terminal device 11 can interactively process the table.

[0139] Also, in the configuration as shown in FIG. 26, the server device 12 may perform the processing of the display device 2. In this case, the display device 2 displays the stroke data and transmits various events to the server device 12. The server device 12 performs redaction and the like to generate the screen information of the display device 2 and transmits it to the display device 2.

Example

[0140] In the following example, another configuration example of the display device 2 will be described.

[0141] <<Another Configuration Example 1 of the Display Device>> Although the display device 2 of the present embodiment is described as having a large touch panel, the display device 2 is not limited to having a touch panel.

[0142] FIG. 27 is a diagram showing another configuration example of the display device 2. In FIG. 27, a projector 411 is installed on the upper side of a normal whiteboard 413. This projector 411 corresponds to the display device 2. The normal whiteboard 413 is not a flat panel display integrated with a touch panel, but a whiteboard on which the user can directly write by hand with a marker. Note that the whiteboard may be a blackboard or any flat surface large enough to project an image.

[0143] The projector 411 has an ultra-short focus optical system and can project an image with little distortion onto the whiteboard 413 from about 10 cm. This image may be transmitted from a PC connected wirelessly or by wire, or may be stored in the projector 411.

[0144] The user writes on the whiteboard 413 using a dedicated electronic pen 2501. The electronic pen 2501 has, for example, a light-emitting part at its tip that emits light when the user presses it against the whiteboard 413 for handwriting, and the switch turns on. Since the wavelength of the light is near-infrared or infrared, it is invisible to the user's eyes. The projector 411 has a camera that images the light-emitting part, analyzes the image, and identifies the direction of the electronic pen 2501. In addition, the electronic pen 2501 emits sound waves along with the light emission, and the projector 411 calculates the distance based on the arrival time of the sound waves. The projector 411 can identify the position of the electronic pen 2501 based on the direction and the distance. The handwritten data is drawn (projected) at the position of the electronic pen 2501.

[0145] Since the projector 411 projects the menu 430, when the user presses a button with the electronic pen 2501, the projector 411 identifies the button pressed based on the position of the electronic pen 2501 and the ON signal of the switch. For example, when the save button 431 is pressed, the handwritten data (coordinate point sequence) written by the user is saved by the projector 411. The projector 411 saves the handwriting information to a predetermined server 412, a USB memory 2600, or the like. The handwriting information is saved page by page. Since it is saved as coordinates rather than image data, the user can re-edit it. However, in this embodiment, since operation commands can be called by handwriting, the menu 430 does not have to be displayed.

Example

[0146] <<Another configuration example 2 of the display device>> Figure 28 is a diagram showing another configuration example of the display device 2. In the example of Figure 28, as the display device 2, the system includes a terminal device 600, an image projection device 700A, and a pen motion detection device 810.

[0147] The terminal device 600 is connected to the image projection device 700A and the pen motion detection device 810 by wire. The image projection device 700A projects the image data input by the terminal device 600 onto the screen 800.

[0148] The pen motion detection device 810 communicates with the electronic pen 820 and detects the motion of the electronic pen 820 in the vicinity of the screen 800. Specifically, on the screen 800, the electronic pen 820 detects the coordinate information indicating the point indicated by the electronic pen 820 (the detection method may be the same as that in FIG. 27) and transmits it to the terminal device 600.

[0149] Based on the coordinate information received from the pen motion detection device 810, the terminal device 600 generates image data of the handwritten data input by the electronic pen 820 and causes the image projection device 700A to draw the image of the handwritten data on the screen 800.

[0150] Also, the terminal device 600 generates superimposed image data indicating a superimposed image obtained by synthesizing the background image projected by the image projection device 700A and the image of the handwritten data input by the electronic pen 820.

Example

[0151] <<Another configuration example 3 of the display device>> FIG. 29 is a diagram showing a configuration example of the display device 2. In the example of FIG. 29, as the display device 2, the system includes a terminal device 600, a display 800A, and a pen motion detection device 810A.

[0152] The pen motion detection device 810A is arranged in the vicinity of the display 800A, detects the coordinate information indicating the point indicated by the electronic pen 820A on the display 800A (the detection method may be the same as that in FIG. 27), and transmits it to the terminal device 600. In the example of FIG. 29, the electronic pen 820A may be charged by the terminal device 600 via a USB connector.

[0153] Based on the coordinate information received from the pen motion detection device 810A, the terminal device 600 generates image data of the image of the handwritten data input by the electronic pen 820A and displays it on the display 800A.

Example

[0154] <<Another configuration example 4 of the display device>> FIG. 30 is a diagram showing a configuration example of the display device 2. In the example of FIG. 30, as the display device 2, the system includes a terminal device 600 and an image projection device 700A.

[0155] The terminal device 600 performs wireless communication (such as Bluetooth (registered trademark)) with the electronic pen 820B and receives the coordinate information of the point indicated by the electronic pen 820B on the screen 800. The coordinate information may be read by the electronic pen 820B from the minute position information formed on the screen 800, or the terminal device 600 may receive the coordinate information from the screen 800.

[0156] Then, based on the received coordinate information, the terminal device 600 generates image data of an image of the handwritten data input by the electronic pen 820B and causes the image projection device 700A to project the image of the handwritten data.

[0157] In addition, the terminal device 600 generates superimposed image data showing a superimposed image obtained by synthesizing the background image projected by the image projection device 700A and the image of the handwritten data input by the electronic pen 820B.

[0158] As described above, each of the above-described embodiments can be applied in various system configurations.

[0159] <Other application examples> As described above, the best mode for carrying out the present invention has been described using examples. However, the present invention is not limited to such examples, and various modifications and substitutions can be made without departing from the gist of the present invention. For example, configurations combining Example 1 and Example 2, Example 1 and Example 3, Example 1 and Example 4, Example 1 and Example 5, Example 2 and Example 3, Example 2 and Example 4, and Example 2 and Example 5 are also possible.

[0160] For example, in this embodiment, a display device that can be used as an electronic blackboard has been described. However, the display device only needs to be able to display an image, and for example, it may be a digital signage or the like. Also, instead of a display, a projector may perform the display. In this case, in this embodiment, the coordinates of the pen tip are detected by the touch panel to detect the coordinates of the pen. However, the display device 2 may detect the coordinates of the pen tip by ultrasonic waves. The pen emits ultrasonic waves together with light emission, and the display device 2 calculates the distance based on the arrival time of the ultrasonic waves. The display device 2 can specify the position of the pen based on the direction and the distance. The projector draws (projects) the trajectory of the pen as a stroke.

[0161] Also, in this embodiment, an electronic blackboard has been described as an example. However, any information processing device having a touch panel can be preferably applied. A device having the same functions as an electronic blackboard is also referred to as an electronic whiteboard, an electronic information board, an interactive board, or the like. Examples of the information processing device equipped with a touch panel include output devices such as a PJ (Projector), a digital signage, etc., a HUD (Head Up Display) device, industrial machines, imaging devices, sound collection devices, medical devices, network home appliances, notebook PCs (Personal Computers), mobile phones, smartphones, tablet terminals, game machines, PDAs, digital cameras, wearable PCs, or desktop PCs, etc.

[0162] Also, the configuration example such as in FIG. 4 is divided according to the main functions in order to facilitate the understanding of the processing by the display device 2. The present invention is not limited by the way and the name of the division of the processing units. The processing of the display device 2 can be further divided into more processing units according to the processing content. Also, it can be divided such that one processing unit includes more processing.

[0163] Moreover, each function of the embodiments described above can be realized by one or more processing circuits. Here, the "processing circuit" in this specification refers to a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, an ASIC (Application Specific Integrated Circuit) designed to execute each function described above, a DSP (digital signal processor), an FPGA (field programmable gate array), and devices such as conventional circuit modules.

[0164] Note that the coordinate detection unit 22 and the stroke processing unit 32 are examples of reception means. The table processing unit 33 is an example of conversion means. The UI image generation unit 36 is an example of setting screen display means. The operation processing unit 26 is an example of operation reception means. The communication control unit 27 is an example of communication control means.

Explanation of Reference Numerals

[0165] 2 Electronic blackboard 3 Display 4 Electronic pen

Prior Art Documents

Patent Documents

[0166]

Patent Document 1

Claims

1. A display device for displaying a table on a screen, comprising: reception means for receiving input of handwritten data for the table; conversion means for converting the handwritten data received by the reception means into a character string based on an attribute associated with the table; UI image generation means for outputting a screen for receiving attribute settings for each cell of the table, the screen including the cell and setting information of the attribute of the cell; operation reception means for receiving attribute settings for each cell of the table; A display device, characterized by comprising the above.

2. The display device according to claim 1, wherein when a certain period of time has elapsed since the electronic pen for the handwritten data has left the touch panel, the conversion means converts the handwritten data received by the reception means into a character string based on an attribute associated with the table.

3. The display device according to claim 1, wherein in response to pressing a clear button displayed on the screen, the conversion means converts the handwritten data received by the reception means into a character string based on an attribute associated with the table.

4. The display device according to claim 3, wherein when a character string is already displayed in the table in addition to the handwritten data received by the reception means, in response to pressing the clear button, the conversion means converts the handwritten data received by the reception means into a character string based on an attribute associated with the table, and displays the character string already displayed in the table based on an attribute associated with the table.

5. The display device according to any one of claims 1 to 4, wherein the conversion means converts the handwritten data received by the reception means into a character string, and further displays the converted character string based on an attribute associated with the table.

6. The display device according to any one of claims 1 to 5, wherein the attribute is a font.

7. The display device according to any one of claims 1 to 6, wherein the attribute is a font size.

8. The display device according to any one of claims 1 to 7, wherein the attribute is a background color.

9. The display device according to any one of claims 1 to 8, wherein the attribute is the arrangement of the character string within the cell of the table.

10. The attribute is a setting as to whether to automatically adjust the horizontal size of the cell of the table according to the horizontal size of the character string within the cell. Compare the horizontal size of the said string with the horizontal size of the table cell, When the said setting for automatic adjustment is False, determine the necessary number of lines from the number of characters that can fit in one line of the cell and the number of characters of the said string, calculate the height of the cell, and correct the height of the cell. When the said setting for automatic adjustment is True, it has a table processing unit that determines the necessary horizontal size for the cell from the horizontal size of the said string and increases the horizontal size of the cell. The display device according to any one of claims 1 to 9.

11. The display device according to any one of claims 1 to 10, characterized in that the said attribute is a setting as to whether the cell is a title or not.

12. The display device according to any one of claims 1 to 11, characterized in that the said attribute is a setting as to whether to automatically convert the handwritten data received by the said reception means into a string when a certain period of time has elapsed since the electronic pen of the said handwritten data left the touch panel.

13. It has a setting screen display means for displaying a setting screen for receiving the setting of the said attribute, The said operation reception means receives the setting for the said setting screen. The display device according to any one of claims 1 to 12.

14. A display device that displays a table on the screen, A reception means for receiving the input of handwritten data for the said table, A conversion means for converting the handwritten data received by the said reception means into a string based on the attribute associated with the said table, A UI image generation means for outputting a screen for receiving the setting of an attribute for each cell of the said table, the screen including the said cell and the setting information of the said attribute of the cell, An operation reception means for receiving the setting of different attributes for each cell of the said table, A program for functioning as such.

15. A conversion method performed by a display device that displays a table on the screen, A step in which a reception means receives the input of handwritten data for the said table, A step in which a conversion means converts the handwritten data received by the said reception means into a string based on the attribute associated with the said table, A step in which a UI image generation means outputs a screen for receiving the setting of an attribute for each cell of the said table, the screen including the said cell and the setting information of the said attribute of the cell, A step in which an operation reception means receives the setting of different attributes for each cell of the said table, A conversion method characterized by having the above.

16. A display system in which a display device and a terminal device communicate via a server device, wherein the display device includes: reception means for receiving input of handwritten data for a table; conversion means for converting the handwritten data received by the reception means into a character string based on an attribute associated with the table; UI image generation means for outputting a screen for receiving setting of an attribute for each cell of the table, the screen including the cell and setting information of the attribute of the cell; operation reception means for receiving setting of different attributes for each cell of the table; communication control means for transmitting the character string converted by the conversion means and data related to the table to the server device; and wherein the terminal device is a display system that displays the table and the character string received from the server device.

Images