Display device, display method, and program

Abstract

To provide a display screen that allows a user to search for a playback part in sound data that the user wants to play back.SOLUTION: In a PC 5, a transmitting / receiving unit 51 receives, from a schedule management server 8, sound data, text data converted from the sound data, and time information related to a time of generation of the text data. A display control unit 54 displays, on a display 508, the received text data in order of the time of generation of the text data, and displays, on the display 508, a playback part specification image 581p that shows a playback part in a total playback time of the received sound data. A receiving unit 52 receives selection for specific text data from the displayed text data. The display control unit 54 displays the playback part specification image 581p at a position of time indicated in time information corresponding to the specific text data selection for which has been received.SELECTED DRAWING: Figure 48

Description

[Technical Field]

[0001] The present disclosure relates to a display terminal, a shared system, a display control method, and a program. [Background technology]

[0002] In recent years, electronic whiteboards have been used in meetings and other events held at companies, educational institutions, government agencies, etc., where a background image is displayed on a display and users can draw stroke images such as letters, numbers, and figures on top of this background image (for example, Patent Document 1).

[0003] Furthermore, when an event such as a conference is held, a technology is known in which a server converts recorded data from the event into text data and a terminal displays images related to the text data, thereby automatically creating minutes of the event (for example, Patent Document 2). Also known is a technology in which a terminal displays minutes including images related to the text data, and plays back recorded data from the event (for example, Patent Document 3). Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional methods, when a user views a displayed image and wants to play back the audio data corresponding to that image, the user must select which portion of the audio data to play out of the total playback time, which makes it difficult for the user to find the playback point in the audio data they want to play. [Means for solving the problem]

[0005] In order to solve the above-mentioned problem, the invention of claim 1 is a display terminal capable of communicating with a management system that manages sound data and text data converted from the sound data, and comprises: a receiving means that receives from the management system the sound data, the text data, and time information regarding the occurrence time of the text data; a display control means that displays the received text data on a display unit in the order of occurrence time and displays on the display unit a first image indicating the playback position in the total playback time of the received sound data; and an acceptance means that accepts a selection of specific text data from the displayed text data, wherein the display control means is a display terminal that displays the first image at the time position indicated in the time information corresponding to the specific text data whose selection has been accepted.

[0006] According to the present invention, an effect is achieved in that a user can find a playback position of sound data that he or she wishes to play. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a schematic diagram of a shared system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a hardware configuration diagram of an electronic whiteboard. [Figure 3] FIG. 2 is a hardware configuration diagram of a video conference terminal. [Figure 4] FIG. 2 is a hardware configuration diagram of a car navigation device. [Figure 5] This is a hardware configuration diagram of a PC and a server. [Figure 6] FIG. 1 is a diagram illustrating the software configuration of an electronic whiteboard. [Figure 7] This is a diagram of the software configuration of a PC. [Figure 8] FIG. 2 is a functional block diagram of a shared system. [Figure 9] 1A is a conceptual diagram of a user authentication management table, FIG. 1B is a conceptual diagram of an access management table, and FIG. 1C is a conceptual diagram of a schedule management table. [Figure 10]1A is a conceptual diagram of an execution event management table, and FIG. 1B is a conceptual diagram of a content management table. [Figure 11] 1A is a conceptual diagram of a user authentication management table, FIG. 1B is a conceptual diagram of a user management table, and FIG. 1C is a conceptual diagram of a union management table. [Figure 12] 1A is a conceptual diagram of a union reservation management table, and FIG. 1B is a conceptual diagram of an event management table. [Figure 13] 1A is a conceptual diagram of a server authentication management table, and FIG. 1B is a conceptual diagram of a project member management table. [Figure 14] 1A is a conceptual diagram of an execution event history management table, and FIG. 1B is a conceptual diagram of an execution event management table. [Figure 15] FIG. 10 is a conceptual diagram of a related information management table. [Figure 16] FIG. 10 is a sequence diagram showing a schedule registration process. [Figure 17] FIG. 10 is a diagram showing a sign-in screen. [Figure 18] This is an example of the initial screen of a PC. [Figure 19] FIG. 10 is a diagram showing a schedule input screen. [Figure 20] FIG. 10 is a sequence diagram illustrating a process for starting an event. [Figure 21] FIG. 10 is a diagram showing a sign-in screen displayed on an interactive whiteboard. [Figure 22] FIG. 10 is a diagram showing a reservation list screen for a union. [Figure 23] FIG. 10 is a sequence diagram illustrating a process for starting an event. [Figure 24] FIG. 10 is a diagram showing a project list screen. [Figure 25] FIG. 10 is a diagram showing a detailed information screen for an event. [Figure 26] FIG. 10 is an image diagram showing a state in which a user uses an electronic whiteboard. [Figure 27] FIG. 10 is a sequence diagram illustrating a process of registering an execution event history. [Figure 28A]10 is a flowchart showing a speech-to-text conversion process. [Figure 28B] FIG. 10 is a sequence diagram showing a speech-to-text conversion process. [Figure 28C] FIG. 10 is a diagram illustrating an example of a data file including text data. [Figure 29] FIG. 10 is a sequence diagram illustrating a process of registering an execution event history. [Figure 30] 10 is a flowchart showing a process of registering an action item. [Figure 31] 10 is an example of a screen that recognizes an action item. [Figure 32] 10 is an example screen showing a list of candidates for carrying out an action item. [Figure 33] 10 is a screenshot showing a calendar for selecting a due date for an action item. [Figure 34] FIG. 10 is a sequence diagram showing an event termination process. [Figure 35] FIG. 10 is a sequence diagram showing an event termination process. [Figure 36] FIG. 10 is a diagram showing an event end screen on an electronic whiteboard. [Figure 37] FIG. 10 is a diagram showing a file data upload screen on an electronic whiteboard. [Figure 38] FIG. 10 is a diagram showing a file data upload completion screen on an electronic whiteboard. [Figure 39] FIG. 10 is a sequence diagram showing a process of viewing an execution event history. [Figure 40] FIG. 10 is a sequence diagram showing a process of viewing an execution event history. [Figure 41] This is a diagram showing a project list screen on a PC. [Figure 42] FIG. 10 is a diagram showing an execution event list screen on a PC. [Figure 43] FIG. 10 is a diagram showing an execution event history screen on a PC. [Figure 44] FIG. 10 is a diagram showing an execution event history screen on a PC. [Figure 45]FIG. 10 is a diagram showing an action item screen on a PC. [Figure 46] FIG. 10 is a sequence diagram illustrating a selection process of an executed event history. [Figure 47] FIG. 10 is a diagram showing an execution event history screen on a PC. [Figure 48] FIG. 10 is a diagram showing an execution event history screen on a PC. [Figure 49] FIG. 10 is a sequence diagram illustrating a selection process of an executed event history. [Figure 50] FIG. 10 is a sequence diagram illustrating a selection process of an executed event history. [Figure 51] FIG. 10 is a sequence diagram illustrating a selection process of an executed event history. DETAILED DESCRIPTION OF THE INVENTION

[0008] The shared system 1 according to this embodiment will be described in detail below with reference to the drawings. In this embodiment, the term "file" refers to an "electronic file."

[0009] <<System configuration overview>> First, a description will be given of an outline of the configuration of the shared system 1. Fig. 1 is a schematic diagram of the shared system according to this embodiment.

[0010] As shown in FIG. 1, the shared system 1 of this embodiment is constructed by an electronic whiteboard 2, a video conferencing terminal 3, a car navigation device 4, a PC (Personal Computer) 5, a sharing support server 6, a schedule management server 8, and a speech-to-text conversion server 9.

[0011] The electronic whiteboard 2, video conference terminal 3, car navigation device 4, PC 5, sharing support server 6, schedule management server 8, and speech-to-text conversion server 9 can communicate with each other via a communication network 10. The communication network 10 is constructed using the Internet, a mobile communication network, a LAN (Local Area Network), etc. The communication network 10 may include not only wired communication networks but also networks using wireless communication networks such as 3G (3rd Generation), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution).

[0012] The electronic whiteboard 2 is used in conference room X. The video conference terminal 3 is used in conference room Y. The union is available for reservation by each user. The car navigation device 4 is used in vehicle α. In this case, vehicle α is a vehicle that is eligible for car sharing. Vehicles include cars, motorbikes, bicycles, wheelchairs, etc.

[0013] A "union" refers to an object, service, space (room), location, or information that is used jointly by multiple people or organizations. Meeting room X, meeting room Y, and vehicle α are examples of unions that are shared by multiple users. An example of information attached to a union is an account. For example, in a specific service provided on the web, the accounts used are all in one place. In such cases, it is restricted.

[0014] The electronic whiteboard 2, the video conference terminal 3, and the car navigation device 4 are examples of communication terminals. A "communication terminal" is, for example, a terminal that becomes available when a user signs in (see S32 described later). Note that the communication terminal used in the vehicle α includes not only the car navigation device 4 but also a smartphone or a smartwatch on which a car navigation application is installed.

[0015] The PC 5 is an example of a display terminal, and an example of a registration device that registers reservations for the use of each union and events that each user plans to carry out with the schedule management server 8. Events include, for example, meetings, get-togethers, get-togethers, consultations, arrangements, driving, riding, or traveling.

[0016] The sharing support server 6 is a computer that remotely supports each communication terminal in sharing a common object.

[0017] The schedule management server 8 is a computer that manages the reservations of each union and the schedule of each user's plans.

[0018] The voice-to-text conversion server 9 is a computer that converts sound (voice) data received from an external computer (for example, the sharing support server 6) into text data.

[0019] Here, the sharing support server 6, the schedule management server 8, and the speech-to-text conversion server 9 are referred to as a management system. The management system may be, for example, a computer that consolidates all or part of the functions of the sharing support server 6, the schedule management server 8, and the speech-to-text conversion server 9. The sharing support server 6, the schedule management server 8, and the speech-to-text conversion server 9 may each be configured to distribute their functions across multiple computers. While the sharing support server 6, the schedule management server 8, and the speech-to-text conversion server 9 are described as server computers located in a cloud environment, the sharing support server 6 and the speech-to-text conversion server 9 may be servers that operate on a communication terminal such as the electronic whiteboard 2, or may be servers located in an on-premise environment. The schedule management server 8 may also be a server located in an on-premise environment.

[0020] <<Hardware configuration>> Next, the hardware configuration of the devices or terminals that constitute the shared system 1 will be described with reference to FIGS.

[0021] <Hardware configuration of the electronic whiteboard> Fig. 2 is a hardware configuration diagram of the electronic whiteboard. As shown in Fig. 2, the electronic whiteboard 2 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an SSD (Solid State Drive) 204, a network I / F 205, and an external device connection I / F (Interface) 206.

[0022] Of these, the CPU 201 controls the overall operation of the electronic whiteboard 2. The ROM 202 stores programs used to drive the CPU 201, such as the CPU 201 and an IPL (Initial Program Loader). The RAM 203 is used as a work area for the CPU 201. The SSD 204 stores various data, such as programs for the electronic whiteboard. The network I / F 205 controls communication with the communication network 10. The external device connection I / F 206 controls communication with a USB (Universal Serial Bus) memory 2600, a PC 2700, and external devices (a microphone 2200, a speaker 2300, and a camera 2400).

[0023] The electronic whiteboard 2 also includes a capture device 211, a GPU 212, a display controller 213, a contact sensor 214, a sensor controller 215, an electronic pen controller 216, a short-range communication circuit 219, an antenna 219a of the short-range communication circuit 219, and a power switch 222.

[0024] Of these, the capture device 211 acquires image data displayed on a display 220, which is an example of a display unit (display means), via a display controller 213 and stores the image data in a RAM 203 or the like. The GPU (Graphics Processing Unit) 212 is a semiconductor chip that specializes in graphics. The display controller 213 controls and manages screen display in order to output an output image from the capture device 211 or the GPU 212 to the display 220 or the like. The contact sensor 214 detects contact of the electronic pen 2500, the user's hand H, or the like on the display 220. The sensor controller 215 controls the processing of the contact sensor 214. The contact sensor 214 inputs and detects coordinates using an infrared blocking method. This coordinate input and coordinate detection method involves two light receiving and emitting devices installed at both ends of the upper side of the display 220 emitting multiple infrared rays parallel to the display 220, which are reflected by reflecting members installed around the periphery of the display 220 and return along the same optical path as the light emitted by the light receiving element. The contact sensor 214 outputs the ID of the infrared light emitted by the two light receiving and emitting devices that has been blocked by the object to the sensor controller 215, and the sensor controller 215 identifies the coordinate position that is the contact position of the object. The electronic pen controller 216 determines whether the pen tip or the pen tail has touched the display 220 by communicating with the electronic pen 2500. The short-range communication circuit 219 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The power switch 222 is a switch for turning the power of the electronic whiteboard 2 ON / OFF.

[0025] Furthermore, the electronic whiteboard 2 includes a bus line 210. The bus line 210 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 201 shown in FIG.

[0026] The contact sensor 214 is not limited to an infrared blocking type, and various detection means may be used, such as a capacitive touch panel that identifies the contact position by detecting a change in capacitance, a resistive film touch panel that identifies the contact position by a change in voltage between two opposing resistive films, or an electromagnetic induction touch panel that identifies the contact position by detecting electromagnetic induction caused by a contacting object coming into contact with the display unit. Also, the electronic pen controller 216 may determine whether or not a part of the electronic pen 2500 that is held by the user or other parts of the electronic pen has been touched, in addition to the pen tip and pen butt.

[0027] <Videoconferencing terminal hardware configuration> FIG. 3 shows the hardware configuration of a videoconference terminal 3. As shown in FIG. 3, the videoconference terminal 3 includes a CPU 301, a ROM 302, a RAM 303, a flash memory 304, an SSD 305, a media I / F 307, operation buttons 308, a power switch 309, a bus line 310, a network I / F 311, a CMOS (Complementary Metal Oxide Semiconductor) sensor 312, an image sensor I / F 313, a microphone 314, a speaker 315, an audio input / output I / F 316, a display I / F 317, an external device connection I / F 318, a short-range communication circuit 319, and an antenna 319a of the short-range communication circuit 319. Of these components, the CPU 301 controls the overall operation of the videoconference terminal 3. The ROM 302 stores programs used to drive the CPU 301, such as an IPL. The RAM 303 is used as a work area for the CPU 301. Flash memory 304 stores various data such as communication programs, image data, and audio data. SSD 305 controls reading and writing of various data from and to flash memory 304 under the control of CPU 301. Note that an HDD may be used instead of an SSD. Media I / F 307 controls reading and writing (storing) of data from and to recording media 306 such as flash memory. Operation button 308 is a button that is operated when selecting a destination for the video conference terminal 3, etc. Power switch 309 is a switch for turning the power of the video conference terminal 3 ON / OFF.

[0028] The network I / F 311 is an interface for data communication using a communication network 10 such as the Internet. The CMOS sensor 312 is a type of built-in imaging means that captures an image of a subject and obtains image data under the control of the CPU 301. The image sensor I / F 313 is a circuit that controls the driving of the CMOS sensor 312. The microphone 314 is a type of built-in sound collection means that inputs sound. The sound input / output I / F 316 is a circuit that processes input and output of sound signals between the microphone 314 and the speaker 315 under the control of the CPU 301. The display I / F 317 is a circuit that transmits image data to an external display 320 under the control of the CPU 301. The external device connection I / F 318 is an interface for connecting various external devices. The short-range communication circuit 319 is a communication circuit such as NFC or Bluetooth.

[0029] 3. The bus line 310 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 301 shown in FIG.

[0030] Display 320 is a type of display unit (display means) configured with liquid crystal or organic electroluminescence (EL) that displays an image of a subject, operation icons, etc. Display 320 is connected to display I / F 317 by cable 320c. This cable 320c may be a cable for analog RGB (VGA) signals, a cable for component video, or a cable for DisplayPort, HDMI (High-Definition Multimedia Interface (registered trademark)), or DVI (Digital Video Interactive) signals.

[0031] Note that an imaging element such as a CCD (Charge Coupled Device) sensor may be used instead of the CMOS sensor 312. External devices such as an external camera, an external microphone, and an external speaker can be connected to the external device connection I / F 318 via a USB cable or the like. When an external camera is connected, the external camera is driven in priority over the built-in CMOS sensor 312 under the control of the CPU 301. Similarly, when an external microphone or an external speaker is connected, the external microphone or the external speaker is driven in priority over the built-in microphone 314 or the built-in speaker 315, respectively, under the control of the CPU 301.

[0032] The recording medium 306 is detachable from the video conference terminal 3. As long as it is a non-volatile memory that reads or writes data under the control of the CPU 301, it is not limited to the flash memory 304, and an EEPROM or the like may also be used.

[0033] <Hardware configuration of car navigation device> 4 is a hardware configuration diagram of a car navigation device. As shown in FIG. 4, the car navigation device 4 includes a CPU 401, a ROM 402, a RAM 403, an EEPROM 404, a power switch 405, an acceleration / direction sensor 406, a media I / F 408, and a GPS receiver 409.

[0034] Of these, the CPU 401 controls the overall operation of the car navigation device 4. The ROM 402 stores programs used to drive the CPU 401, such as an IPL. The RAM 403 is used as a work area for the CPU 401. The EEPROM 404 reads or writes various data, such as programs for the car navigation device 4, under the control of the CPU 401. The power switch 405 is a switch for turning the power of the car navigation device 4 on and off. The acceleration / direction sensor 406 is a variety of sensors, such as an electronic magnetic compass or gyrocompass that detects geomagnetism, and an acceleration sensor. The media I / F 408 controls the reading or writing (storage) of data from or to a recording medium 407, such as a flash memory. The GPS receiver 409 receives GPS signals from GPS satellites.

[0035] The car navigation device 4 also includes a long-distance communication circuit 411, an antenna 411a of the long-distance communication circuit 411, a CMOS sensor 412, an image sensor I / F 413, a microphone 414, a speaker 415, an audio input / output I / F 416, a display 417, a display I / F 418, an external device connection I / F 419, a short-distance communication circuit 420, and an antenna 420a of the short-distance communication circuit 420.

[0036] Among these, the long-distance communication circuit 411 is a circuit for receiving traffic information, road work information, traffic accident information, etc. provided by external infrastructure outside the vehicle, and conversely transmitting the vehicle position, life-saving signals in case of emergency, etc. to the outside. The external infrastructure is, for example, a road information guidance system such as the VICS (registered trademark) (Vehicle Information and Communication System) system. The CMOS sensor 412 is a kind of built-in imaging means for imaging a subject according to the control of the CPU 401 to obtain image data. The imaging device I / F 413 is a circuit for controlling the driving of the CMOS sensor 412. The microphone 414 is a kind of built-in sound collection means for inputting sound. The audio input / output I / F 416 is a circuit for processing the input / output of sound signals between the microphone 414 and the speaker 415 according to the control of the CPU 401. The display 417 is a kind of display unit (display means) such as a liquid crystal or an organic EL for displaying the image of the subject, various icons, etc., and has the function of a touch panel. The touch panel is a kind of input means for the user to operate the car navigation device 4. The display I / F 418 is a circuit for displaying an image on the display 417. The external device connection I / F 419 is an interface for connecting various external devices. The short-distance communication circuit 420 is a communication circuit such as NFC or Bluetooth. Also, the car navigation device 4 includes a bus line 410. The bus line 410 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 401 shown in FIG. 4.

[0037] <Hardware Configuration of PC and Server> FIG. 5 is a hardware configuration diagram of a PC and a server. The PC 5 is constructed by a computer and, as shown in FIG. 5, includes a CPU 501, a ROM 502, a RAM 503, an HD 504, an HDD (Hard Disk Drive) controller 505, a media I / F 507, a display 508, a network I / F 509, a keyboard 511, a mouse 512, a CD-RW drive 514, a speaker 515, and a bus line 510.

[0038] Of these, the CPU 501 controls the overall operation of the PC 5. The ROM 502 stores programs, such as an IPL, used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data, such as programs. The HDD (Hard Disk Drive) controller 505 controls the reading and writing of various data from and to the HD 504 under the control of the CPU 501. The media I / F 507 controls the reading and writing (storage) of data from and to a recording medium 506, such as a flash memory. The display 508 displays various information, such as a cursor, menus, windows, characters, or images. The display 508 is an example of a display unit (display means). The network I / F 509 is an interface for data communication using the communication network 10. The keyboard 511 is a type of input means equipped with multiple keys for inputting characters, numbers, various instructions, etc. The mouse 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving the cursor, etc. A CD-RW drive 514 controls reading and writing of various data from and to a CD-RW (Compact Disc-Rewritable) 513, which is an example of a removable recording medium. A speaker 515 outputs sound signals under the control of the CPU 501.

[0039] The PC 5 also includes a bus line 510. The bus line 510 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 501 shown in FIG.

[0040] 5, the sharing support server 6 is constructed by a computer and includes a CPU 601, a ROM 602, a RAM 603, a HD 604, a HDD controller 605, a recording medium 606, a media I / F 607, a display 608, a network I / F 609, a keyboard 611, a mouse 612, a CD-RW drive 614, and a bus line 610. These components have the same configuration as the CPU 501, ROM 502, RAM 503, HD 504, HDD controller 505, a recording medium 506, a media I / F 507, a display 508, a network I / F 509, a keyboard 511, a mouse 512, a CD-RW drive 514, and a bus line 510 in the PC 5, and therefore a description thereof will be omitted.

[0041] 5, the schedule management server 8 is configured with a computer, and includes a CPU 801, a ROM 802, a RAM 803, a HD 804, a HDD controller 805, a recording medium 806, a media I / F 807, a display 808, a network I / F 809, a keyboard 811, a mouse 812, a CD-RW drive 814, and a bus line 810. These components have the same configuration as the CPU 501, ROM 502, RAM 503, a HD 504, a HDD controller 505, a recording medium 506, a media I / F 507, a display 508, a network I / F 509, a keyboard 511, a mouse 512, a CD-RW drive 514, and a bus line 510 in the PC 5, and therefore description thereof will be omitted.

[0042] 5, the speech-to-text conversion server 9 is constructed by a computer and includes a CPU 901, a ROM 902, a RAM 903, a HD 904, a HDD controller 905, a recording medium 906, a media I / F 907, a display 908, a network I / F 909, a keyboard 911, a mouse 912, a CD-RW drive 914, and a bus line 910. These components have the same configuration as the CPU 501, ROM 502, RAM 503, HD 504, HDD controller 505, a recording medium 506, a media I / F 507, a display 508, a network I / F 509, a keyboard 511, a mouse 512, a CD-RW drive 514, and a bus line 510 in the PC 5, and therefore description thereof will be omitted.

[0043] Each of the above programs may be recorded on a computer-readable recording medium as an installable or executable file and distributed. Examples of recording media include CD-Rs (Compact Disc Recordables), DVDs (Digital Versatile Disks), Blu-ray Discs, SD cards, etc. The recording media may be provided domestically or internationally as a program product.

[0044] Furthermore, the sharing support server 6 may be constructed by a single computer, or may be constructed by multiple computers to which each unit (function, means, or memory unit) is divided and arbitrarily assigned. The same applies to the schedule management server 8 and the speech-to-text conversion server 9.

[0045] <<Electronic whiteboard software configuration>> Next, we will use Figure 6 to explain the computer software installed on the interactive whiteboard 2. Here, computer software (hereinafter referred to as software) refers to programs related to computer operation and other information used for computer processing that is equivalent to a program. A program is a set of instructions for a computer that are combined to produce a single result. Furthermore, equivalents to programs refer to things that cannot be called programs because they are not direct instructions to a computer, but that have properties similar to programs in that they define computer processing. For example, a data structure (the logical structure of data expressed by the interrelationships between data elements) is equivalent to a program.

[0046] Additionally, an application is a general term for software used to perform specific processes. On the other hand, an operating system (hereinafter referred to as OS) is software that controls a computer and allows applications to use computer resources. The OS performs basic management and control of a computer, such as input / output control, management of hardware such as memory and hard disks, and process management. Applications run using the functions provided by the OS.

[0047] 6 is a diagram showing the software configuration of the electronic whiteboard. As shown in FIG. 6, an OS 101, a Launcher 102, a schedule viewer 103a, a file viewer 103b, and a browser application 103c run on a working area 15 of a RAM 203. The OS 101 is basic software that provides the basic functions of the electronic whiteboard 2 and manages the entire electronic whiteboard 2.

[0048] Launcher102 is a launcher application that operates on OS101. Launcher102 manages, for example, the start and end of events such as meetings executed on the electronic blackboard 2, or manages external applications such as the schedule viewer 103a, file viewer 103b, and browser application 103c used during the execution of events.

[0049] The schedule viewer 103a, file viewer 103b, and browser application 103c are external applications that operate on Launcher102 (hereinafter, referred to as external application 103 when there is no need to particularly distinguish). The external application 103 is executed independently of Launcher102 and executes services or functions provided on OS101. Note that FIG. 6 shows an example in which three external applications, the schedule viewer 103a, file viewer 103b, and browser application 103c, are installed on the electronic blackboard 2, but the number of external applications is not limited to this.

[0050] <<Software Configuration of PC>> Subsequently, the computer software installed on PC5 will be described using FIG. 7. FIG. 7 is a diagram showing the software configuration of the PC. As shown in FIG. 7, the OS 5501, minutes application 5502a, and browser 5502b operate on the work area 5500 of the RAM 503. The OS 5501 is basic software that provides the basic functions of PC5 and manages the whole of PC5.

[0051] The minutes application 5502a and browser 5502b are applications for generating and displaying an execution event history screen, which is the minutes of an event executed by the electronic blackboard 2, using various data transmitted from the schedule management server 8. Note that FIG. 7 shows an example in which two external applications, the minutes application 5502a and browser 5502b, are installed on PC5, but the number of external applications is not limited to this.

[0052] <<Functional configuration of the shared system>> Next, the functional configuration of this embodiment will be described with reference to Figures 8 to 15. Figure 8 is a functional block diagram of a shared system. Note that Figure 8 shows the terminals, devices, and servers shown in Figure 1 that are related to the processes or operations described below.

[0053] <Functional configuration of the electronic whiteboard> 8, the interactive whiteboard 2 has a transmitting / receiving unit 21, a receiving unit 22, an image / sound processing unit 23, a display control unit 24, a determining unit 25, a recognizing unit 26, an acquiring / providing unit 28, and a storing / reading processing unit 29. Each of these units is a function or a means for performing the function, which is realized when any of the components shown in FIG. 2 operates in response to an instruction from the CPU 201 in accordance with a program loaded from the SSD 204 onto the RAM 203. The interactive whiteboard 2 also has a memory unit 2000 constructed by the RAM 203, the SSD 204, or the USB memory 2600 shown in FIG. 2.

[0054] (Each function of the electronic whiteboard) Next, we will explain each component of the electronic whiteboard 2. The transmitting / receiving unit 21 is realized by commands from the CPU 201 shown in Fig. 2, as well as the network I / F 205 and the external device connection I / F 206, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 10.

[0055] The reception unit 22 is realized mainly by commands from the CPU 201 shown in FIG. 2, as well as the contact sensor 214 and the electronic pen controller 216, and receives various inputs from the user.

[0056] The image / sound processing unit 23 is realized by instructions from the CPU 201 and the capture device 211 shown in FIG. 2 and stores image data displayed on the display 220. The image / sound processing unit 23 is also realized by instructions from the CPU 201 and the GPU 212 shown in FIG. 2 and performs image processing for displaying an image on the display 220. The image / sound processing unit 23 also performs image processing on image data obtained by the camera 2400 capturing an image of a subject. The image / sound processing unit 23 also performs audio processing on audio data related to the audio signal after the microphone 2200 converts the user's voice into an audio signal. The image / sound processing unit 23 also outputs an audio signal related to the audio data to the speaker 2300, causing the speaker 2300 to output the audio. The image / sound processing unit 23 also performs processing to convert drawing image data obtained by the user drawing on the display 220 with the electronic pen 2500 or their hand H into coordinate data. For example, when an electronic whiteboard (2a) at a given location transmits coordinate data to an electronic whiteboard (2b) at another location, the electronic whiteboard (2b) displays a drawing of the same content on the display 220 of the other electronic whiteboard (2b) based on the coordinate data.

[0057] The display control unit 24 is realized by instructions from the CPU 201 shown in Fig. 2 and the display controller 213 shown in Fig. 2, and causes the display 220 to display drawn images and access the sharing support server 6 using a Web browser to display various screen data. Specifically, the display control unit 24 starts and executes the launcher 102 and external application 103 that run on the OS 101 shown in Fig. 6, thereby causing the display 220 to display various screens drawn by an API (Application Programming Interface) provided by the OS 101.

[0058] The determination unit 25 is realized by instructions from the CPU 201 shown in FIG. 2, and performs various determinations.

[0059] The recognition unit 26 is realized by instructions from the CPU 201 shown in FIG. 2, and as shown in FIG. 31 described later, recognizes a designated area 262 surrounded on the display 220.

[0060] The acquisition / providing unit 28 is executed by instructions from the CPU 201 shown in FIG. 2, as well as the short-distance communication circuit 219 and the antenna 219a, and performs acquisition and provision of data by short-distance communication with a proprietary terminal such as an IC card or a smartphone.

[0061] The storage / readout processing unit 29 is executed by instructions from the CPU 201 shown in FIG. 2 and the SSD 204, and performs processing for storing various data in the storage unit 2000 or reading out various data stored in the storage unit 2000. Further, in the storage unit 2000, image data and audio data received when communicating with other electronic blackboards or video conference terminals are overwritten and stored each time they are received. Among these, an image is displayed on the display 220 by the image data before overwriting, and audio is output from the speaker 2300 by the audio data before overwriting.

[0062] Note that the video conference terminal 3 and the car navigation device 4 have the same functions as the electronic blackboard 2, and thus the description thereof is omitted here.

[0063] <Functional configuration of the PC> The PC 5 has a transmission / reception unit 51, a reception unit 52, a display control unit 54, a generation unit 56, an audio control unit 58, and a storage / readout processing unit 59. Each of these units is a function or a means that functions by operating according to instructions from the CPU 501 in which any of the components shown in FIG. 5 is deployed on the RAM 503 from the HD 504. The PC 5 also has a storage unit 5000 constructed by the HD 504 shown in FIG. 5.

[0064] (Functional configurations of the PC) Next, we will explain each component of the PC 5. The transmitting / receiving unit 51 is realized by commands from the CPU 501 and the network I / F 509 shown in Fig. 5, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 10. The transmitting / receiving unit 51 is an example of a receiving means.

[0065] The reception unit 52 is realized mainly by commands from the CPU 501 shown in Fig. 5, as well as the keyboard 511 and the mouse 512, and receives various inputs from the user. The reception unit 52 is an example of a reception means.

[0066] The display control unit 54 is implemented by instructions from the CPU 501 shown in FIG. 5 , and displays images on the display 508 or accesses the sharing support server 6 using a web browser to display various screen data. Specifically, the display control unit 54 accesses the sharing support server 6 or the schedule management server 8 by, for example, starting and executing a minutes application 5502 a or a browser 5502 b running on the OS 5501 shown in FIG. 7 . The display control unit 54 then downloads a WebAPP (Web Application) including at least HTML (HyperText Markup Language) and CSS (Cascading Style Sheets) or JAVA SCRIPT (registered trademark), and displays various image data generated by the WebAPP on the display 508. The display control unit 54 displays image data generated by HTML5 including data in, for example, Extensible Markup Language (XML), JavaScript Object Notation (JSON), or Simple Object Access Protocol (SOAP) format on the display 508. The display control unit 54 is an example of a display control unit.

[0067] The generation unit 56 is realized by commands from the CPU 501 shown in FIG. 5, and has a function of generating various image data to be displayed on the display 508. The generation unit 56 generates various image data using content data received by the transmission / reception unit 51. For example, the generation unit 56 renders text data, which is content data, and generates image data related to the text data, which is content image data, in order to display the rendered data. Rendering is a process of interpreting data written in a language for describing web pages (HTML, CSS, XML, etc.) and calculating the layout of characters, image data, etc., to be actually displayed on the screen. The generation unit 56 is an example of a generation means.

[0068] 5, and has a function of outputting a sound signal from speaker 515. Sound control unit 58 sets sound data to be output from speaker 515, and plays back the sound data by outputting a sound signal related to the set sound data from speaker 515. Sound control unit 58 is an example of a playback means.

[0069] The storage / readout processing unit 59 is executed by instructions from the CPU 501 shown in FIG. 5 and the HDD controller 505, etc., and performs processing such as storing various data in the memory unit 5000 and reading various data from the memory unit 5000.

[0070] <Functional configuration of the shared support server> The sharing support server 6 has a transmitting / receiving unit 61, an authenticating unit 62, a creating unit 63, a generating unit 64, a determining unit 65, and a storing / reading processing unit 69. Each of these units is a function or a means for performing a function that is realized when any of the components shown in Fig. 5 operates in response to an instruction from the CPU 601 in accordance with the sharing support program loaded from the HD 604 onto the RAM 603. The sharing support server 6 also has a storage unit 6000 constructed by the HD 604 etc. shown in Fig. 5.

[0071] (User authentication management table) Fig. 9(A) is a conceptual diagram showing a user authentication management table. A user authentication management DB 6001 configured with a user authentication management table such as that shown in Fig. 9(A) is constructed in the storage unit 6000. In this user authentication management table, a user ID for identifying a user, a user name, an organization ID for identifying the organization to which the user belongs, and a password are associated and managed. The organization ID also includes a domain name representing a group or organization for managing multiple computers on a communication network.

[0072] (Access control table) FIG. 9(B) is a conceptual diagram showing an access management table. The storage unit 6000 has an access management DB 6002 configured with an access management table such as that shown in FIG. 9(B). This access management table manages an organization ID, an access ID, and an access password required for authentication when accessing the schedule management server 8, in association with each other. The access ID and access password are required for the sharing support server 6 to use the services (functions) provided by the schedule management server 8 via a Web API or the like using the HTTP (Hypertext Transfer Protocol) or HTTPS (Hypertext Transfer Protocol Secure) protocols. The schedule management server 8 manages multiple schedulers, and different organizations may use different schedulers, so management using the access management table is necessary.

[0073] (Schedule management table) Fig. 9(C) is a conceptual diagram showing a schedule management table. A schedule management DB 6003 configured with a schedule management table such as that shown in Fig. 9(C) is constructed in the storage unit 6000. In this schedule management table, for each scheduled event ID and executed event ID, the organization ID, the user ID of the person who made the reservation, whether or not the person who made the reservation will participate, the name of the person who made the reservation, the scheduled start time, the scheduled end time, the event name, the user IDs of other participants, whether or not the other participants will participate, the names of the other participants, and file data are associated and managed.

[0074] Of these, the scheduled event ID is identification information for identifying a reserved event. The scheduled event ID is an example of scheduled event identification information for identifying an event that is scheduled to be executed. The executed event ID is identification information for identifying an event that has actually been executed or is currently being executed among the reserved events. The executed event ID is an example of executed event identification information for identifying an event that has been executed or is currently being executed. The reservation name is the name of the person who reserved the union. If the union is a conference room, for example, it is the name of the organizer, and if the union is a vehicle, it is the name of the driver, for example. The scheduled start time indicates the scheduled start time of use of the union. The scheduled end time indicates the scheduled end time of use of the union. The event name indicates the name of the event that the reservation is scheduled to be executed by the reservationr. The user ID of other participants is identification information for identifying participants other than the reservationr. The other participant names are the names of participants other than the reservationr, and include the union. In other words, in this case, the users include not only the reservationr and other participants, but also the union. The file data is file data of a material file to be used for an event corresponding to a scheduled event ID, which is registered by user A using schedule input screen 550 (see FIG. 19) described later. The file data is data in a predetermined file format created using various applications. The file format of the file data is, for example, a PowerPoint file, an Excel file, etc.

[0075] (Execution event management table) FIG. 10A is a conceptual diagram illustrating an execution event management table. The storage unit 6000 stores an execution event management DB 6004 configured with the execution event management table shown in FIG. 10A. In this execution event management table, an execution event ID is associated with each project ID and managed. The project ID is identification information (an example of project identification information) for identifying a project. A project indicates a group, team, or group composed of multiple members working to achieve a specific goal, plan, or project. Members belonging to the same project can share execution event history, such as minutes of events associated with the project ID. As shown in FIG. 24 (described later), a project ID is assigned to each project, such as next year's policy or customer development. The project ID may also be referred to as a group ID or team ID.

[0076] (Content Management Table) FIG. 10(B) is a conceptual diagram showing a content management table. A content management DB 6005 configured with a content management table such as that shown in FIG. 10(B) is stored in the storage unit 6000. In this content management table, a content processing ID, a type of content processing, the content details, and the start and end dates and times of the content processing are associated and managed for each execution event ID. Here, content refers to the details of an execution event that occurred in an event such as a meeting for a certain project, or materials used in the event. Types of content processing include recording, snapshots, speech-to-text conversion, the generation of action items, and the delivery of materials. The content processing ID is identification information for identifying the content processing that occurred in each event.

[0077] Here, the content includes history information indicating the execution details of the event and action items generated by the executed event. The history information is data such as audio recording data, snapshots, audio text, or materials. A snapshot is a process of acquiring image data of a display screen at a certain point in time during an ongoing event. Note that a snapshot is also called, for example, a capture or image recognition.

[0078] If the type of content processing is "sound recording," the content includes a URL indicating the destination where the recorded sound data is saved. If the type of content processing is "snapshot," the content includes a URL indicating the destination where image data of the screen acquired by snapshot (capture) is saved. Capturing means saving an image (still image, moving image) displayed on display 220 as image data. If the type of content processing is "speech-to-text conversion," the content includes a URL indicating the destination where text data of the received speech text is saved.

[0079] Here, an action item occurs during an event such as a meeting in a certain project, and indicates the action that must be taken by those involved in the event. When the type of content processing is "action item occurrence," the content includes the user ID of the person who will execute the action item, the deadline for completing the action item, and a URL indicating the location where image data representing the action item is saved.

[0080] (Functional configuration of the shared support server) Next, a detailed description will be given of each functional configuration of the sharing support server 6. In the following, when describing each functional configuration of the sharing support server 6, the relationship between each component shown in Fig. 5 and the main components for realizing each functional configuration of the sharing support server 6 will also be described.

[0081] The transmission / reception unit 61 of the sharing support server 6 shown in Figure 8 is realized by instructions from the CPU 601 shown in Figure 5 and the network I / F 609 shown in Figure 5, and transmits and receives various data (or information) with other terminals, devices, or systems via the communication network 10.

[0082] The authentication unit 62 is realized by instructions from the CPU 601 shown in Figure 5, and performs authentication by determining whether the information (user ID, organization ID and password) sent from the communication terminal is information that has been pre-registered in the user authentication management DB 6001.

[0083] The creation unit 63 is realized by commands from the CPU 601 shown in FIG. 5, and creates a reservation list screen 230 as shown in FIG. 22, which will be described later, based on the reservation information and schedule information sent from the schedule management server 8.

[0084] The generating unit 64 is implemented by an instruction from the CPU 601 shown in FIG. 5, and generates an execution event ID, a content processing ID, and a URL of a storage destination.

[0085] The determination unit 65 is implemented by instructions from the CPU 601 shown in Fig. 5, and performs various determinations, which will be described later.

[0086] The storage / readout processing unit 69 is realized by instructions from the CPU 601 shown in FIG. 5 and the HDD controller 605 shown in FIG. 5, and performs processing such as storing various data in the memory unit 6000 and reading out various data stored in the memory unit 6000.

[0087] <Schedule management server functional configuration> The schedule management server 8 has a transmission / reception unit 81, an authentication unit 82, a generation unit 83, and a storage / readout processing unit 89. Each of these units is a function or a means for performing a function that is realized when any of the components shown in Fig. 5 operates in response to an instruction from the CPU 801 in accordance with a schedule management program loaded from the HD 804 onto the RAM 803. The schedule management server 8 also has a storage unit 8000 constructed by the HD 804 shown in Fig. 5.

[0088] (User authentication management table) Fig. 11(A) is a conceptual diagram showing a user authentication management table. A user authentication management DB 8001 configured with a user authentication management table such as that shown in Fig. 11(A) is constructed in the storage unit 8000. In this user authentication management table, a user ID for identifying a user is associated with an organization ID for identifying the organization to which the user belongs, and a password, and these are managed.

[0089] (User management table) Fig. 11(B) is a conceptual diagram showing a user management table. A user management DB 8002 configured with a user management table such as that shown in Fig. 11(B) is constructed in the storage unit 8000. In this user management table, a user ID and the name of the user indicated by the user ID (user name) are associated and managed for each organization ID.

[0090] (Union management table) Fig. 11(C) is a conceptual diagram showing a union management table. A union management DB 8003 configured with a union management table such as that shown in Fig. 11(C) is constructed in the storage unit 8000. In this union management table, a union ID for identifying a union and a name of the union (union name) are associated and managed for each organization ID.

[0091] (Union reservation management table) FIG. 12(A) is a conceptual diagram showing a union reservation management table. A union reservation management DB 8004 configured with a union reservation management table such as that shown in FIG. 12(A) is constructed in the storage unit 8000. This union reservation management table manages reservation information in which each piece of information is associated with other pieces of information. The reservation information includes, for each organization ID, a union ID, a union name, the user ID of the person making the reservation, a scheduled start date and time of use, a scheduled end date and time of use, and an event name. Of these, the scheduled start date and time of use indicates the scheduled start date and time of use of the union. The scheduled end date and time of use indicates the scheduled end date and time of use of the union. Each date and time indicates the year, month, day, hour, minute, second, and time zone, but due to space limitations, FIG. 12(A) only shows the year, month, day, hour, and minute.

[0092] (Event management table) FIG. 12B is a conceptual diagram illustrating an event management table. The storage unit 8000 stores an event management DB 8005 configured with an event management table such as that shown in FIG. 12B. This event management table manages schedule information in which each piece of information is associated with the other pieces of information. The schedule information manages, for each scheduled event ID, the organization ID, user ID, user name, scheduled event start date and time, scheduled event end date and time, and event name in association with each other. Of these, the scheduled event start date and time indicates the scheduled start date and time if the event is to be executed. The scheduled event end date and time indicates the scheduled end date and time if the event is to be executed. Each date and time is represented in the format of year, month, day, hour, minute, second, and time zone; however, due to space limitations, FIG. 12B only shows the year, month, day, hour, and minute. The event management table also manages file data for material files used in events indicated in the schedule information, associated with the scheduled event ID.

[0093] (Server authentication management table) 13(A) is a conceptual diagram showing a server authentication management table. A server authentication management DB 8006 configured with a server authentication management table such as that shown in FIG. 13(A) is constructed in the storage unit 8000. In this server authentication management table, access IDs and access passwords are associated and managed. The access IDs and access passwords are conceptually the same as the access IDs and access passwords managed in the access management DB 6002 of the sharing support server 6.

[0094] (Project member management table) Fig. 13(B) is a conceptual diagram showing a project member management table. A project member management DB 8007 configured with a project member management table such as that shown in Fig. 13(B) is constructed in the storage unit 8000. In this project member management table, the project ID, project name, and user ID of each project member are associated and managed for each organization ID.

[0095] (Execution event history management table) Fig. 14(A) is a conceptual diagram showing an execution event history management table. An execution event history management DB 8008 configured with an execution event history management table such as that shown in Fig. 14(A) is constructed in the storage unit 8000. In this execution event history management table, a content processing ID, a type of content processing, content details, and start and end dates and times of content processing are associated and managed for each project ID and execution event ID. This execution event history management DB 8008 manages some of the same data as the content management DB 6005. The same data includes the execution event ID, content processing ID, type of content processing, and start and end dates and times of content processing. The content details are stored in the same destination, with the only difference being the notation for the destination where the content data is stored (http: / / or c: / / ).

[0096] (Execution event management table) Fig. 14(B) is a conceptual diagram showing an execution event management table. An execution event management DB 8009 configured with an execution event management table such as that shown in Fig. 14(B) is constructed in the storage unit 8000. In this execution event management table, an execution event ID is associated with an event name, and the start and end dates and times of the event. This execution event management DB 8009 manages information related to events that have actually been executed, out of the schedule information managed in the event management DB 8005.

[0097] (Related information management table) FIG. 15 is a conceptual diagram illustrating a related information management table. The storage unit 8000 stores a related information management DB 8010 configured with the related information management table shown in FIG. 15. In this related information management table, related information is managed in a state in which each piece of information (data) is associated with each project ID and each executed event ID. The related information includes a content occurrence time period, sound data, audio text data, and screen data, all of which are associated with each other. The content occurrence time period indicates the elapsed time from the start date and time of an event at the time the content occurred in the executed event. The content occurrence time period is generated by the generation unit 83 based on the event start date and time stored in the event management DB 8005 and the start date and time and end date and time of the content processing stored in the executed event history management DB 8008. The content occurrence time period is an example of time information. The sound data includes a content processing ID and a content type. The audio text data and screen data include a content processing ID, a content type, and a sequence number. The sequence number indicates the chronological order in which the content processing occurred in the audio text data and the screen data, respectively.

[0098] (Schedule management server functional configuration) Next, we will explain in detail each functional configuration of the schedule management server 8. In explaining each functional configuration of the schedule management server 8, we will also explain the relationship between each component shown in Fig. 5 and the main components that realize each functional configuration of the schedule management server 8.

[0099] The transmission / reception unit 81 of the schedule management server 8 shown in Figure 8 is realized by instructions from the CPU 801 shown in Figure 5 and the network I / F 809 shown in Figure 5, and transmits and receives various data (or information) with other terminals, devices, or systems via the communication network 10.

[0100] 5, and performs authentication by determining whether the information (user ID, organization ID, and password) sent from the union is information that has been pre-registered in the user authentication management DB 8001. Also, the authentication unit 82 performs authentication by determining whether the information (access ID and access password) sent from the sharing support server 6 is information that has been pre-registered in the server authentication management DB 8006.

[0101] The generating unit 83 is realized by an instruction from the CPU 801 shown in FIG. 5, and has a function of generating related information to be registered in the related information management DB 8010 .

[0102] The storage / readout processing unit 89 is realized by instructions from the CPU 801 shown in FIG. 5 and the HDD controller 805 shown in FIG. 5, and performs processing such as storing various data in the memory unit 8000 and reading out various data stored in the memory unit 8000.

[0103] <Functional configuration of speech-to-text conversion server> The speech-to-text conversion server 9 has a transmission / reception unit 91, a conversion unit 93, and a storage / readout processing unit 99. Each of these units is a function or a means of functioning that is realized when any of the components shown in Fig. 5 operates in response to an instruction from the CPU 901 in accordance with a program loaded from the HD 904 onto the RAM 903. The speech-to-text conversion server 9 also has a memory unit 9000 constructed by the HD 904 shown in Fig. 5.

[0104] (Functional configuration of speech-to-text conversion server) Next, we will explain in detail each functional configuration of the speech-to-text conversion server 9. In explaining each functional configuration of the speech-to-text conversion server 9, we will also explain the relationship between each component shown in Figure 5 and the main components that realize each functional configuration of the speech-to-text conversion server 9.

[0105] The transmission / reception unit 91 of the speech-to-text conversion server 9 shown in Figure 8 is realized by instructions from the CPU 901 shown in Figure 5 and the network I / F 909 shown in Figure 5, and transmits and receives various data (or information) with other terminals, devices, or systems via the communication network 10.

[0106] The conversion unit 93 is realized by an instruction from the CPU 901 shown in FIG. 5, and converts the sound data received by the transmission / reception unit 91 via the communication network 10 into text data.

[0107] The storage / readout processing unit 99 is realized by instructions from the CPU 901 shown in Figure 5 and the HDD controller 905 shown in Figure 5, and performs processing such as storing various data in the memory unit 9000 and reading out various data stored in the memory unit 9000.

[0108] The above IDs are examples of identification information. Organization IDs include company names, business establishment names, department names, and local area names. User IDs include employee numbers, driver's license numbers, and My Numbers under Japan's social security and tax numbering system.

[0109] <<Processing or Operation of the Embodiment>> The processing or operation of each embodiment will be described below.

[0110] <Schedule registration process> Here, the process in which reservation person A (Rikotaro) registers his / her schedule in schedule management server 8 from PC 5 will be described with reference to Figs. 16 to 19. Fig. 16 is a sequence diagram showing the schedule registration process. Fig. 17 is a diagram showing a sign-in screen. Fig. 18 is an example of the initial screen of a PC. Fig. 19 is a diagram showing a schedule input screen.

[0111] First, user A operates the keyboard 511 or the like of the PC 5, causing the display control unit 54 of the PC 5 to display a sign-in screen 530 for signing in on the display 508, as shown in FIG. 17 (step S11). This sign-in screen 530 includes an input field 531 for inputting the user's user ID and organization ID, an input field 532 for inputting a password, a sign-in button 538 to be pressed to sign in, and a cancel button 539 to be pressed to cancel the sign-in. Here, the user ID and organization ID are user A's email address. The username portion of the email address represents the user ID, and the domain name portion represents the organization ID. Note that input field 531 may be an input field for separately inputting the user ID and organization ID instead of the email address.

[0112] Next, when user A enters his / her user ID and organization ID in input field 531, enters his / her password in input field 532, and presses sign-in button 538, reception unit 52 accepts the sign-in request (step S12). Then, transmission / reception unit 51 of PC 5 transmits sign-in request information indicating the sign-in request to schedule management server 8 (step S13). This sign-in request information includes the information accepted in step S12 (user ID, organization ID, and password). As a result, transmission / reception unit 81 of schedule management server 8 receives the sign-in request information.

[0113] Next, the authentication unit 82 of the schedule management server 8 authenticates user A using the user ID, organization ID, and password (step S14). Specifically, the storage / readout processing unit 89 searches the user authentication management DB 8001 (see FIG. 11(A)) for a set of user ID, organization ID, and password that corresponds to the set of user ID, organization ID, and password received in step S13. If a corresponding set is found, the authentication unit 82 determines that user A, the requesting user, is a valid user. If a corresponding set is not found, the authentication unit 82 determines that user A is an invalid (illegitimate) user. If user A is not legitimate, the transmission / reception unit 81 notifies the PC 5 that the user is not legitimate; however, here we will continue to explain the case where user A is legitimate.

[0114] Next, the transmitting / receiving unit 81 transmits the authentication result to the PC 5 (step S15), whereby the transmitting / receiving unit 51 of the PC 5 receives the authentication result.

[0115] Next, if the generation unit 56 of the PC 5 receives an authentication result indicating authenticity in step S15, the generation unit 56 generates an initial screen 540 as shown in FIG. 18 (step S16). Then, the display control unit 54 of the PC 5 displays the initial screen 540 as shown in FIG. 18 on the display 508 (step S17). This initial screen 540 includes a "Register Schedule" button 541 that is pressed to register a schedule, and a "View Execution Event History" button 543 that is pressed to view the execution event history. When the user presses the "Register Schedule" button 541, the acceptance unit 52 accepts the schedule registration (step S18). Then, the transmission / reception unit 51 transmits a schedule registration request to the schedule management server 8 (step S19). As a result, the transmission / reception unit 81 of the schedule management server 8 receives the schedule registration request.

[0116] Next, the storage / readout processing unit 89 of the schedule management server 8 searches the user management DB 8002 (see FIG. 11(B)) using the organization ID received in step S13 as a search key, and reads out all corresponding user IDs and all user names (step S20). Then, the transmission / reception unit 81 transmits schedule input screen information to the PC 5 (step S21). This schedule input screen information includes all user IDs and all user names read out in step S20. All user names also include the name (first name) of the reserver, who is user A, who entered information to sign in in step S12. As a result, the transmission / reception unit 51 of the PC 5 receives the schedule input screen information.

[0117] Next, in the PC 5, the generation unit 56 generates a schedule input screen 550 using the schedule input screen information received in step S21 (step S22). Then, the display control unit 24 of the PC 5 displays the schedule input screen 550 as shown in Fig. 19 on the display 508 (step S23).

[0118] This schedule input screen 550 includes an input field 551 for entering the event name, an input field 552 for entering the union ID or union name, an input field 553 for entering the scheduled start date and time of the event (using the union), an input field 554 for entering the scheduled end date and time of the event (using the union), an input field 555 for entering notes such as an agenda, a display area 556 for displaying the name of the person who made the reservation, a selection menu 557 for selecting the names of participants other than the person who made the reservation, an "OK" button 558 to be pressed to register the reservation, and a "CANCEL" button 559 to be pressed to cancel the content being entered or that has been entered. The name of the person who made the reservation is the name of the user who entered information to sign in to the PC 5 in step S12. A mouse pointer p1 is also displayed.

[0119] An email address may also be entered in the input field 552. When a union name is selected in the selection menu 557, the union is also added as another participant.

[0120] Next, user A enters predetermined information in input fields 551 to 555, uses mouse pointer p1 to select the name (user name) of the user he or she wants to invite to the event from selection menu 557, and presses "OK" button 558. The accepting unit 52 accepts the input of the schedule information (step S24). The transmitting / receiving unit 51 then transmits the schedule information to the schedule management server 8 (step S25). This schedule information includes the event name, union ID (or union name), scheduled start date and time, scheduled end date and time, the user ID of each participant, and a memo. If a union ID is entered in input field 552 on the schedule input screen 550, this union ID is transmitted. If a union name is entered in input field 552, this union name is transmitted. Furthermore, on the schedule input screen 550, a user name is selected in selection menu 557. Since the user ID was also received in step S21, the user ID corresponding to the user name is transmitted. As a result, the transmitting / receiving unit 81 of the schedule management server 8 receives the schedule information.

[0121] Next, the schedule management server 8 searches the union management DB 8003 (see Figure 11(C)) using the union ID (or union name) received in step S25 as a search key to read out the corresponding union name (or union ID) (step S26).

[0122] Next, the storage / readout processing unit 89 stores the reservation information in the union reservation management DB 8004 (see FIG. 12(A)) (step S27). In this case, the storage / readout processing unit 89 adds one record's worth of reservation information to the union reservation management table of the union reservation management DB 8004, which is managed by a pre-registered scheduler. The reservation information is configured based on the schedule information received in step S25 and the union name (or union ID) read out in step S26. The scheduled start date and time of use in the union reservation management DB 8004 corresponds to the scheduled start date and time in the schedule information. The scheduled end date and time of use in the union reservation management DB 8004 corresponds to the scheduled end date and time in the schedule information.

[0123] Furthermore, the storage and readout processing unit 89 stores the schedule information in the event management DB 8005 (see FIG. 12(B)) (step S28). In this case, the storage and readout processing unit 89 adds one record's worth of schedule information to the event management table of the event management DB 8005, which is managed by a pre-registered scheduler. The schedule information is configured based on the schedule information received in step S25. Note that the scheduled start date and time of the event in the event management DB 8005 corresponds to the scheduled start date and time in the schedule information. Also, the scheduled end date and time of the event in the event management DB 8005 corresponds to the scheduled end date and time in the schedule information.

[0124] In this way, user A can register his / her own schedule in the schedule management server 8. Although the case where schedule registration is performed using the PC 5 has been described in Figs. 16 to 19, schedule registration may also be performed using the electronic whiteboard 2, the video conference terminal 3, or the car navigation device 4 using similar processing.

[0125] <Event start processing> Here, with reference to Figs. 20 to 26, we will explain the process of using the electronic whiteboard 2 to hold a conference with other participants in conference room X reserved by reservation user A (Rikotaro). Figs. 20 and 23 are sequence diagrams showing the start process of an event. Fig. 21 is a diagram showing a sign-in screen displayed on the electronic whiteboard. Fig. 22 is a diagram showing a union reservation list screen. Fig. 24 is a diagram showing a project list screen. Fig. 25 is a diagram showing an event detailed information screen. Fig. 26 is an image diagram showing a state in which a user uses the electronic whiteboard.

[0126] First, user A presses the power switch 222 of the electronic whiteboard 2, causing the reception unit 22 of the electronic whiteboard 2 to receive a power-on command from the user (step S31). When the reception unit 22 receives the power-on command, the Launcher 102 shown in FIG. 6 is started up. Then, the display control unit 24 of the electronic whiteboard 2 displays a sign-in screen 110 for signing in on the display 220, as shown in FIG. 21 (step S32). This sign-in screen 110 includes a selection icon 111 that user A presses when signing in using his / her IC card, a selection icon 113 that user A presses when signing in by entering his / her email address (user ID and organization ID) and password, and a power icon 115 that user A presses when turning off the power without signing in.

[0127] Next, user A presses the selection icon 111 to communicate with the short-range communication circuit 219, such as an IC card reader, or user A presses the selection icon 113 to enter his or her email address and password, and the accepting unit 22 accepts the sign-in request (step S33). Then, the transmitting / receiving unit 21 transmits sign-in request information indicating the sign-in request to the sharing support server 6 (step S34). This sign-in request information includes the information accepted in step S33 (user ID, organization ID, and password), time zone information for the country or region where the electronic whiteboard 2 is installed, and the user ID, organization ID, and password of the communication terminal (here, the electronic whiteboard 2). As a result, the transmitting / receiving unit 61 of the sharing support server 6 receives the sign-in request information.

[0128] Next, the authentication unit 62 of the sharing support server 6 authenticates user A using the user ID, organization ID, and password of user A received in step S34 (step S35). Specifically, the storage / readout processing unit 69 uses the user ID, organization ID, and password of user A received in step S34 as search keys to search the user authentication management DB 6001 (see FIG. 9A) for a set of user ID, organization ID, and password that corresponds to the received set of user ID, organization ID, and password of user A. If a corresponding set is found, the authentication unit 62 determines that user A is a legitimate user. If a corresponding set is not found, the authentication unit 62 determines that user A is an invalid (illegitimate) user. If user A is not legitimate, the transmission / reception unit 61 notifies the electronic whiteboard 2 that the requesting user is not legitimate. However, here, the description will continue for the case where user A is legitimate.

[0129] Next, the storage / read processing unit 69 of the sharing support server 6 searches the access management DB 6002 (see Figure 9(B)) using the organization ID of user A received in step S34 as a search key, and reads out the corresponding access ID and access password (step S36).

[0130] Next, the transmitting / receiving unit 61 transmits reservation request information indicating a request for reservation information of the union and schedule request information indicating a request for schedule information of the user to the schedule management server 8 (step S37). The reservation request information and schedule request information include the time zone information, the user ID and organization ID of the communication terminal received in step S34, and the access ID and password read in step S36. As a result, the transmitting / receiving unit 81 of the schedule management server 8 receives the reservation request information and schedule request information.

[0131] Next, the authentication unit 82 of the schedule management server 8 authenticates the sharing support server 6 using the access ID and access password (step S38). Specifically, the storage / readout processing unit 89 searches the server authentication management DB 8006 (see FIG. 13(A)) for a pair of access ID and access password that corresponds to the pair of access ID and access password received in step S37. If a corresponding pair is found, the authentication unit 82 determines that the requesting sharing support server 6 is a legitimate accessor. If a corresponding pair is not found, the authentication unit 82 determines that the requesting sharing support server 6 is an unauthorized (illegitimate) accessor. If the access is not legitimate, the transmission / reception unit 81 notifies the sharing support server 6 that the access is not legitimate, but here we will continue to explain the case where the access is legitimate.

[0132] The storage / readout processing unit 89 of the schedule management server 8 uses the user ID of the communication terminal received in step S37 as a search key to search the shared reservation management DB 8004 (see FIG. 12(A)) managed by the scheduler, and reads out the corresponding reservation information (step S39). In this case, the storage / readout processing unit 89 reads out reservation information whose scheduled start date and time is today.

[0133] Furthermore, the storage / readout processing unit 89 searches the event management DB 8005 (see FIG. 12(B)) managed by the scheduler using the user ID of the communication terminal received in step S37 as a search key, thereby reading out the corresponding schedule information (step S40). In this case, the storage / readout processing unit 89 reads out schedule information in which the scheduled start date and time of the event is today. If the schedule management server 8 is located in a different country or region from the communication terminal such as the electronic whiteboard 2, the time zone is adjusted based on the time zone information to match the country or region in which the communication terminal is installed.

[0134] Next, the storage / readout processing unit 89 searches the project member management DB 8007 (see Figure 13(B)) using the user ID of the communication terminal received in step S37 as a search key, and reads out the project IDs and project names of all projects that include the user ID of the communication terminal (step S41).

[0135] Next, the transmitting / receiving unit 81 transmits the reservation information read in step S39, the schedule information read in step S40, and all project IDs and all project names read in step S41 to the sharing support server 6 (step S42). As a result, the transmitting / receiving unit 61 of the sharing support server 6 receives the reservation information, schedule information, all project IDs, and all project names.

[0136] Next, the creation unit 63 of the sharing support server 6 creates a reservation list based on the reservation information and schedule information received in step S42 (step S43). Then, the transmission / reception unit 61 transmits reservation list information indicating the contents of the reservation list, as well as all project IDs and all project names, to the electronic whiteboard 2 (step S44). As a result, the transmission / reception unit 21 of the electronic whiteboard 2 receives the reservation list information, as well as all project IDs and all project names.

[0137] Next, in the electronic whiteboard 2, the display control unit 24 displays a reservation list screen 230 as shown in Fig. 22 on the display 220 (step S45). This reservation list screen 230 includes a display area 231 that displays the union name (here, the location name) and a display area 232 that displays today's date and time. Furthermore, the reservation list screen 230 displays event information 235, 236, 237, etc., indicating events that will use this union (here, conference room X) today. Each piece of event information includes, for each event, the scheduled start time and scheduled end time of use of this union, the event name, and the name of the person who reserved this union (reservation name). The event information also includes start buttons 235s, 236s, 237s, etc., which a user presses to specify the event to start.

[0138] 23, when user A presses start button 235s with, for example, electronic pen 2500, reception unit 22 receives the selection of the event indicated in event information 235 (step S51). Then, display control unit 24 displays project list screen 240 as shown in FIG. 24 on display 220 based on the project ID and project name received in step S42 (step S52). Project list screen 240 includes project icons 241 to 246 indicating the respective projects. Project list screen 240 also includes an "OK" button 248 that is pressed to confirm the selected project icon and a "CANCEL" button 249 that is pressed to cancel the selection of the project icon.

[0139] Next, in FIG. 24, when user A presses the project icon 241 with, for example, the electronic pen 2500, the accepting unit 22 accepts the selection of the project indicated by the project icon 241 (step S53).

[0140] Next, the transmitting / receiving unit 21 of the electronic whiteboard 2 transmits to the sharing support server 6 a scheduled event ID indicating the scheduled event selected in step S51 and a project ID indicating the project selected in step S53 (step S54). The process of step S54 is a transmission request process for the execution event identification information. As a result, the transmitting / receiving unit 61 of the sharing support server 6 receives the selected scheduled event ID and the selected project ID.

[0141] Next, in the sharing support server 6, the generation unit 64 generates a unique event ID (step S55). Then, the storage / readout processing unit 69 associates and manages the event ID generated in step S55, the reserved event ID received in step S54, the user ID and organization ID of the person who made the reservation, and the event information (step S56). Note that the user ID and organization ID of the person who made the reservation, and the event information are IDs and information based on the reservation information and schedule information received in step S42. Note that at this point, there is no entry in the participation / non-participation column in the reservation management table (see FIG. 9(C)).

[0142] Next, the storage / read processing unit 69 stores and manages the project ID received in step S54 and the execution event ID generated in step S55 in association with each other (step S57).

[0143] Next, in the sharing support server 6, the transmitting / receiving unit 61 transmits file data transmission request information indicating a request to transmit the file data registered in the schedule management server 8 to the schedule management server 8 (step S58). The file data transmission request information includes the scheduled event ID received in step S54, the user ID and organization ID of the communication terminal received in step S34, and the access ID and access password read in step S36. As a result, the transmitting / receiving unit 81 of the schedule management server 8 receives the file data transmission request information.

[0144] Next, the storage / readout processing unit 89 of the schedule management server 8 searches the event management DB 8005 (see FIG. 12(B)) using the scheduled event ID received in step S58 as a search key, thereby reading out file data associated with the scheduled event ID (step S59). Then, the transmitting / receiving unit 81 transmits the file data read in step S59 to the sharing support server 6 (step S60). As a result, the transmitting / receiving unit 61 of the sharing support server 6 receives the file data.

[0145] Next, the storage / read processing unit 69 of the sharing support server 6 stores and manages the file data received in step S60 in the schedule management DB 6003 (see Figure 9(C)) in association with the scheduled event ID received in step S54 and the executed event ID generated in step S55 (step S61).

[0146] Then, the transmitting / receiving unit 61 transmits the execution event ID generated in step S55 and the file data received in step S60 to the interactive whiteboard 2 (step S62). As a result, the transmitting / receiving unit 21 of the interactive whiteboard 2 receives the execution event ID and the file data.

[0147] Next, in the electronic whiteboard 2, the storage / readout processing unit 29 stores the execution event ID and file data in the storage unit 2000 (step S63). Here, the file data transmitted from the sharing support server 6 is stored in a specific storage area of ​​the storage unit 2000. As a result, the electronic whiteboard 2 accesses the specific storage area during the execution of the event, and the display control unit 24 displays the file data stored in the specific storage area on the display 220. Here, the specific storage area is a temporary storage destination for data provided for each ongoing event, and is a storage area identified by an arbitrary path (character string) indicating the location within the storage unit 2000. Note that the specific storage area does not have to be configured to be provided inside the electronic whiteboard 2, but may be configured to be provided in an external storage device connected to the electronic whiteboard 2, or in a local server present in an on-premises environment and capable of communicating with the electronic whiteboard 2.

[0148] Next, the display control unit 24 displays a detailed information screen 250 of the selected event on the display 220, as shown in Fig. 25 (step S64). This detailed information screen 250 of the event includes a display area 251 for the event name, a display area 252 for the planned time of the event (planned start time and planned end time), and a display area 253 for the names of people who have reserved the event. Furthermore, the detailed information screen 250 of the event also displays a display area 256 for displaying the contents of notes, a display area 257 for displaying the names of people who will attend the event, and a display area 258 for displaying identification information (e.g., file name) for identifying file data stored in a specific storage area of ​​the storage unit 2000. The display area 257 displays the name of the person who reserved the event and the names of other selected participants shown in Fig. 19, and displays a check box for each person who will attend the event. The display area 258 also displays the file names of file data stored in a specific storage area of ​​the storage unit 2000, i.e., file data downloaded from the sharing support server 6, as well as the file names of file data currently being downloaded from the sharing support server 6. Furthermore, the bottom right of the event detail information screen 250 includes a "Close" button 259 that is pressed to close the detail information screen 250.

[0149] Next, when the user checks the checkboxes of the users who are actually participating among the names of the prospective participants and presses the "Close" button 259, the reception unit 22 receives the selection of the participants (step S65). Then, the transmission / reception unit 21 transmits information indicating the user ID of each prospective participant and whether or not they will participate to the sharing support server 6 (step S66). As a result, the transmission / reception unit 61 of the sharing support server 6 receives the user ID of each prospective participant and information indicating whether or not they will participate.

[0150] Next, the sharing support server 6 stores and manages whether or not the person will participate in the participation column that has not been entered in the schedule management DB 6003 (step S67).

[0151] As described above, user A starts an event (here, a policy decision meeting) using the union (here, conference room X) and a communication terminal (here, the electronic whiteboard 2). Then, as shown in FIG. 26, user A can hold a meeting in conference room X using the electronic whiteboard 2. The display control unit 24 displays an event execution screen R as shown in FIG. 26 on the display 220. The display control unit 24 displays the remaining time for which this union can be used in the upper right area of ​​the event execution screen R. In this case, the display control unit 24 displays the time (remaining time) from the current time to the scheduled end time indicated in the event information selected in step S51.

[0152] The display control unit 24 also displays, on the event execution screen R, an icon r1 that is pressed to register an action item, an icon r2 that is pressed to view the execution event history, an icon r3 that is pressed to view a material file stored in a specific storage area of ​​the storage unit 2000, and file data r4 of the material file. The icon r3 is an example of a selection image that accepts a selection for displaying file data stored in a specific storage area. For example, when a user of the electronic whiteboard 2 presses the icon r3, the accepting unit 22 accepts the selection of the icon r3, and the display control unit 24 displays file data r4 of the material file stored in a specific storage area of ​​the storage unit 2000. The display control unit 24 of the electronic whiteboard 2 may display not only the file data received in step S62, but also file data previously stored in the storage unit 2000 or file data newly generated during the currently started event. In this case, the storage / readout processing unit 29 of the electronic whiteboard 2 stores file data generated or updated during the currently started event in a specific storage area of ​​the storage unit 2000.

[0153] <Execution event history registration process> Next, the execution event history registration process will be described with reference to Fig. 27 to Fig. 33. Fig. 27 and Fig. 29 are sequence diagrams showing the execution event history registration process. Fig. 28A is a flowchart showing the speech-to-text conversion process.

[0154] First, the determination unit 25 of the interactive whiteboard 2 determines the type of content processing in the started event (step S71). Specifically, if the content is sound data generated by sound recording by the image and sound processing unit 23, the determination unit 25 determines that the type of content processing is "sound recording." If the content is image data acquired by a snapshot (capture) by the image and sound processing unit 23, the determination unit 25 determines that the type of content processing is "snapshot." If the content is material file data transmitted by the transmission / reception unit 21, the determination unit 25 determines that the type of content processing is "material transmission."

[0155] Next, the transmitting / receiving unit 21 transmits registration request information indicating a request to register the generated content to the sharing support server 6 (step S72). In this case, the transmitting / receiving unit 21 automatically transmits registration request information every time content is generated. This registration request information includes an execution event ID, a user ID of the content sender, content data, and content processing type information. As a result, the transmitting / receiving unit 61 of the sharing support server 6 receives the registration request information.

[0156] Next, the determination unit 65 of the sharing support server 6 determines the type of the received content processing based on the type information included in the registration request information received by the transmission / reception unit 61 (step S73). If the determination unit 65 determines that the type of content processing is "recording", the transmission / reception unit 61 of the sharing support server 6 transmits sound data, which is content data, to the speech-to-text conversion server 9 (step S74). As a result, the transmission / reception unit 91 of the speech-to-text conversion server 9 receives the sound data. Note that if the type of content processing is other than "recording", the sharing support server 6 skips steps S74 to S76 and proceeds to step S77.

[0157] Next, the conversion unit 93 of the speech-to-text conversion server 9 converts the sound data received by the transmission / reception unit 91 into text data (step S75). Here, the speech-to-text conversion process in the speech-to-text conversion server 9 will be described with reference to FIG. 28A. First, the conversion unit 93 acquires information indicating the date and time when the sound data was received by the transmission / reception unit 91 (step S75-1). Note that the information acquired in step S75-1 may be information indicating the date and time when the sharing support server 6 received the sound data or the date and time when the sharing support server 6 transmitted the sound data. In this case, in step S74, the transmission / reception unit 91 of the speech-to-text conversion server 9 receives the sound data transmitted from the sharing support server 6 and the information indicating the date and time.

[0158] Next, the conversion unit 93 executes text conversion processing of the sound data received by the transmission / reception unit 91 (step S75-2). If the text conversion processing of the sound data is complete (YES in step S75-3), the conversion unit 93 proceeds to step S75-4. Meanwhile, the conversion unit 93 repeats the processing of step S75-2 until the text conversion processing of the sound data is complete. In step S75-3, the conversion unit 93 determines that the text conversion processing of the sound data is complete when a predetermined amount of text conversion has been performed based on the sound data received by the transmission / reception unit 91. For example, the conversion unit 93 determines that the text conversion processing of the sound data is complete when text conversion of one sentence has been performed. Then, the conversion unit 93 generates text data converted from the sound data (step S75-4). As a result, the speech-to-text conversion server 9 converts the sound data transmitted from the sharing support server 6 into text data. The speech-to-text conversion server 9 receives the audio data transmitted from the sharing support server 6 at any time, and therefore repeatedly executes the process shown in FIG. 28A.

[0159] Here, the speech-to-text conversion process described in steps S74 to S76 of Fig. 27 and Fig. 28A will be described in more detail with reference to Fig. 28B. Fig. 28B is a sequence diagram showing the speech-to-text conversion process.

[0160] 27, the transmitting / receiving unit 21 of the interactive whiteboard 2 transmits sound data generated by an ongoing event to the sharing support server 6 at any time (steps Sa1 and Sa2). As a result, the transmitting / receiving unit 61 of the sharing support server 6 receives the sound data transmitted from the interactive whiteboard 2 at any time. The transmitting / receiving unit 61 of the sharing support server 6 then transmits the sound data transmitted from the interactive whiteboard 2 to the speech-to-text conversion server 9 at any time (steps Sb1 and Sb2). As a result, the transmitting / receiving unit 91 of the speech-to-text conversion server 9 receives the sound data transmitted from the sharing support server 6 at any time.

[0161] Here, the sound data transmitted from the electronic whiteboard 2 to the sharing support server 6 in step Sa1 corresponds to the sound data transmitted from the sharing support server 6 to the speech-to-text conversion server 9 in step Sb1, and the sound data transmitted from the electronic whiteboard 2 to the sharing support server 6 in step Sa2 corresponds to the sound data transmitted from the sharing support server 6 to the schedule management server 8 in step Sb2. In this way, the sound data collected and processed by the image / sound processing unit 23 of the electronic whiteboard 2 is transmitted to the speech-to-text conversion server 9 as needed.

[0162] The conversion unit 93 of the speech-to-text conversion server 9 converts the sound data received by the transmission / reception unit 91 into text data. Then, the transmission / reception unit 91 of the speech-to-text conversion server 9 transmits the text data converted by the conversion unit 93 to the sharing support server 6 (step Sc1). In this case, the text data transmitted in step Sc1 is text data obtained by converting the sound data received in steps Sb1 and Sb2 into text.

[0163] Here, the text data transmitted from the speech-to-text conversion server 9 to the sharing support server 6 in step Sc1 is unconfirmed text data. When a predetermined amount of text conversion has been performed, the conversion unit 93 of the speech-to-text conversion server 9 determines that the text conversion of the sound data is complete and the text data is confirmed. For example, when text conversion of one sentence has been performed, the conversion unit 93 determines that the text conversion of the sound data is complete and the text data is confirmed. In step Sc1, since the text data is unconfirmed, the transmission / reception unit 91 transmits an unconfirmed flag indicating that the text data has not been confirmed (unconfirmed) along with the text data converted by the conversion unit 93 ("In this fiscal year, more").

[0164] Then, the transmitting / receiving unit 61 of the sharing support server 6 transmits the text data (undetermined text data) transmitted from the speech-to-text conversion server 9 to the electronic whiteboard 2 (step Sc2). As a result, the transmitting / receiving unit 21 of the electronic whiteboard 2 receives the text data (undetermined text data) transmitted from the sharing support server 6. Then, the display control unit 24 of the electronic whiteboard 2 causes the display 508 to display the text data received by the transmitting / receiving unit 21. The electronic whiteboard 2 can display the text data as subtitles for sound data generated by the currently running event on the event execution screen R as shown in FIG. 26. Note that by receiving and displaying even undetermined text data, the electronic whiteboard 2 can confirm to the user of the electronic whiteboard 2 that the speech-to-text conversion process by the speech-to-text conversion server 9 is being performed at any time.

[0165] The shared system 1 repeats the above-described processing (steps Sa3 to Sa7, Sb3 to Sb7, Sc3 to Sc4) to repeatedly perform text conversion processing of sound data collected and processed by the image / sound processing unit 23 of the interactive whiteboard 2. The number of times (frequency) that the speech-to-text conversion server 9 transmits unconfirmed text data can be set appropriately.

[0166] When the conversion unit 93 has completed the text conversion of the sound data and determined that the text data has been finalized, the transmission / reception unit 91 of the speech-to-text conversion server 9 transmits the finalized text data to the sharing support server 6 (step Sc5). In this case, the transmission / reception unit 91 transmits the text data converted by the conversion unit 93 ("In this fiscal year, more than twice the results of last year.") along with a finalization flag indicating that the text data has been finalized. As a result, the transmission / reception unit 61 of the sharing support server 6 receives the text data (finalized text data) transmitted from the speech-to-text conversion server 9.

[0167] Here, the text data received in step Sc5 is stored and managed in the content management DB 6005 by the processing of step S79 in Fig. 27. In this case, the date and time related to the processing of the text data received in step Sc5 is the date and time when the text data is received in step Sc5, after the date and time when the sound data is transmitted in step Sb2. That is, the start date and time of the content processing stored in the content management DB 6005 is the date and time when the sound data is transmitted in step Sb2, and the end date and time of the content processing is the date and time when the text data is received in step Sc5. In this way, the sharing support server 6 can manage, as the start and end dates and times of the content processing, the text data transmitted from the speech-to-text conversion server 9 (confirmed text data), as well as the date and time when the transmission of sound data corresponding to the text data to the speech-to-text conversion server 9 was initiated, after the date and time when the confirmed text data was received.

[0168] Then, the transmitting / receiving unit 61 of the sharing support server 6 transmits the text data (confirmed text data) transmitted from the speech-to-text conversion server 9 to the interactive whiteboard 2 (step Sc6). As a result, the transmitting / receiving unit 21 of the interactive whiteboard 2 receives the text data transmitted from the sharing support server 6. Then, the transmitting / receiving unit 21 of the interactive whiteboard 2 displays the text data transmitted from the sharing support server 6 on the display 220.

[0169] In this way, the sharing system 1 can convert sound data generated by an ongoing event into text data using the speech-to-text conversion server 9, and can store and manage information regarding the time the text data was generated (the start date and time and end date and time of content processing) in the sharing support server 6 and the schedule management server 8.

[0170] Returning to Fig. 27, the description of the execution event history registration process will be continued. The transmitting / receiving unit 91 transmits the text data converted by the converting unit 93 to the sharing support server 6 (step S76). Here, the transmitting / receiving unit 91 transmits information indicating the date and time acquired in step S75-1 together with the text data to the sharing support server 6. As a result, the transmitting / receiving unit 61 of the sharing support server 6 receives the text data.

[0171] Next, the generation unit 64 generates a unique content processing ID for identifying the content processing that occurred due to the event (step S77). The generation unit 64 also generates a URL for content data that indicates the details of the content (step S78). The storage / reading processing unit 69 then associates the type of content processing, the start date and time and end date and time of the content processing, the content processing ID generated in step S77, and the URL indicating the save destination of the content generated in step S78 with each execution event ID received in step S72 in the content management DB 6005 (see FIG. 10(B)), and manages these (step S79).

[0172] Here, when the type of content processing is "speech-to-text conversion," the start date / time and end date / time of the content processing are the date / time when the sound data is converted to text data. Here, the date / time when the sound data is converted to text data is the date / time when the sound data is transmitted by the transmission / reception unit 61 of the sharing support server 6 and the date / time when the text data is received by the transmission / reception unit 61 of the sharing support server 6, as shown in FIG. 28B. Note that the date / time when the sound data is converted to text data may also be the date / time when the sound data is received by the transmission / reception unit 91 of the speech-to-text conversion server 9 and the date / time when the text data is transmitted by the transmission / reception unit 91 of the speech-to-text conversion server 9. Furthermore, when the type of content processing is "speech-to-text conversion," the start date / time and end date / time of the content processing may be the same as the start date / time and end date / time of the content processing related to the sound data to be converted to text.

[0173] Furthermore, when the type of content processing is "sound recording," "snapshot," or "material transmission," the start date and time and end date and time of the content processing are the dates and times when the content data (sound data, image data, file data) is received by the transmitting / receiving unit 61 of the sharing support server 6. Note that when the type of content processing is "sound recording," "snapshot," or "material transmission," the start date and time and end date and time of the content processing may be the dates and times when the content data is transmitted by the transmitting / receiving unit 21 of the interactive whiteboard 2. Furthermore, when the type of content processing is "sound recording," the start date and time and end date and time may be the dates and times when sound recording is performed by the image / sound processing unit 23. Furthermore, when the type of content processing is "snapshot," the start date and time and end date and time of the content processing may be the dates and times when a snapshot (capture) is performed by the image / sound processing unit 23.

[0174] 29, the storage / readout processing unit 69 of the sharing support server 6 searches the execution event management DB 6004 (see FIG. 10(A)) using the execution event ID received in step S72 as a search key to read out the corresponding project ID (step S91). Next, the storage / readout processing unit 69 searches the user authentication management DB 6001 (see FIG. 9(A)) using the user ID of the content sender as a search key to read out the corresponding organization ID (step S92).

[0175] Next, the storage / read processing unit 69 searches the access management DB 6002 (see FIG. 9(B)) using the organization ID read in step S92 as a search key, and reads out the corresponding access ID and access password (step S93).

[0176] Next, the transmitting / receiving unit 61 transmits execution event history registration request information indicating a request to register an execution event history to the schedule management server 8 (step S94). This execution event history registration request information includes the project ID read out in step S91, the execution event ID received in step S72, the sender's user ID, content data, the content processing ID generated in step S77, the URL of the content data generated in step S78, the access ID and access password read out in step S93, and the start date / time and end date / time of the content processing. As a result, the transmitting / receiving unit 81 of the schedule management server 8 receives the execution event history registration request information.

[0177] Next, in the schedule management server 8, the authentication unit 82 authenticates the sharing support server 6 using the access ID and access password (step S95). This authentication is the same process as step S38, so a description thereof will be omitted. Here, the description will continue for the case where the server is legitimate.

[0178] Next, the storage / readout processing unit 89 stores and manages the various data (information) received in step S94 in the executed event history management DB 8008 (see FIG. 14(A)) (step S96). The storage / readout processing unit 89 associates the various data (information) with the project ID and executed event ID received in step S94 and stores them in the executed event history management DB 8008. In this way, the schedule management server 8 manages data with the same content as that on the sharing support server 6 side.

[0179] The generation unit 83 of the schedule management server 8 generates related information that associates the content data received in step S94 with each content occurrence time period (step S97). Here, the content occurrence time period included in the related information is generated using the start date and time of the event stored in the event management DB 8005 and the start date and time and end date and time of the content processing stored in the executed event history management DB 8008. In other words, the content occurrence time period indicates the elapsed time from the start date and time of the event at the time when the content occurred in the executed event. Then, the storage / readout processing unit 89 of the schedule management server 8 stores and manages the related information generated by the generation unit 83 in the related information management DB 8010 (see FIG. 15) in association with the project ID and executed event ID received in step S94 (step S98). In this way, the schedule management server 8 manages content data of different content processing types by associating them with each content occurrence time.

[0180] Here, the data format of the text data stored in the sharing support server 6 and the schedule management server 8 will be described with reference to Fig. 28C. Fig. 28C is a diagram showing an example of a data file containing text data. The data file d1 shown in Fig. 28C is stored and managed at the URL of the content storage destination shown in the content management DB 6005 and the execution event history management DB 8008.

[0181] As shown in Fig. 28C, data file d1 contains 10 pieces of text data (transcript). Each piece of text data (0 to 9) contains a sequence number (seq_num) indicating the order in which the content processing occurred, text data (text), the start date and time of the content processing (start time), the end date and time of the content processing (end_time), and a content processing ID (id). This information is the same as the information corresponding to each item in content management DB 6005 and execution event history management DB 8006. Note that the number of text data contained in data file d1 is not limited to this.

[0182] In this way, the sharing support server 6 and the schedule management server 8 can associate text data with time information regarding the processing of that text data (start date and time and end date and time of content processing) and manage them in a data file d1 as shown in FIG. 28C. can.

[0183] As a result, the electronic whiteboard 2 can transmit the execution event ID of an event in a specific project and the content generated by the execution of the corresponding event to the schedule management server 8. Furthermore, the schedule management server 8 can store the received content in the execution event history management DB 8008 for each execution event ID associated with the project ID. This allows the shared system 1 to store the content generated by the event for each project by having the user select which project the event belongs to at the start of the event.

[0184] (Action item registration process) Here, the processing when the generated content is an action item will be described with reference to Fig. 30 to Fig. 33. Fig. 30 is a flowchart showing the action item registration processing. Fig. 31 is an example of a screen for recognizing an action item. Fig. 32 is an example of a screen showing a list of candidates for executing the action item. Fig. 33 is an example of a screen showing a calendar for selecting the deadline date for the action item.

[0185] First, in FIG. 30, when the user presses icon r1, the receiving unit 22 receives a request to register an action item (step S71-1). Next, as shown in FIG. 31, the user uses the electronic pen 2500 to draw an action item (here, "Submit minutes") on the drawing screen 260a of the electronic whiteboard 2, and then surrounds an image (drawn image) 261 showing the content of the action item with a line (designated area) 262. The receiving unit 22 receives the designation of the predetermined area including the image 261, and the recognition unit 26 recognizes the image 261 included in the predetermined area (step S71-2). Note that FIG. 31 illustrates an example in which the image (drawn image) 261 is surrounded by the line (designated area) 262. However, the recognition unit 26 may be configured to recognize an area at an arbitrary distance from the designated area 262 as the image 261. The designation of the predetermined area is performed, for example, by drawing a predetermined shape, such as a circle or a polygon, with the electronic pen 2500.

[0186] Next, the display control unit 24 displays the candidate list 265 of performers of the action item on the drawing screen 260b as shown in Fig. 32 (step S71-3). Then, when the user uses the electronic pen 2500 to select a specific performer who will perform the action item, the receiving unit 22 receives the selection of the specific performer (step S71-4).

[0187] Next, the display control unit 24 displays a calendar 267 for accepting the specification of the deadline for executing the action item on the drawing screen 260c as shown in Fig. 33 (step S71-5). Then, when the user selects a specific deadline using the electronic pen 2500, the accepting unit 22 accepts the selection of the specific deadline (step S71-6). Note that the calendar 267 is an example of a deadline specification screen. The deadline specification screen may be a date list or the like that does not include the days of the week, etc.

[0188] Through this process, the electronic whiteboard 2 transmits content registration request information indicating a request to register an action item to the sharing support server 6 as a content registration request. This content registration request information includes an execution event ID indicating the event in which the action item occurred, the user ID of the person who executed the action item selected in step S71-4, image data of the action item recognized in step S71-2 (here, image data of "Submit minutes"), and the deadline date for execution of the action item accepted in step S71-6. That is, the transmitting / receiving unit 21 transmits image data within the predetermined area as image data indicating the content of the action item generated in the executed event. As a result, the transmitting / receiving unit 61 of the sharing support server 6 receives the content registration request information. Note that the processing performed after the sharing support server 6 receives the content registration request information is similar to the processing shown in FIGS. 27 and 29, and therefore will not be described here.

[0189] <Event termination processing> Next, the process of ending an ongoing event will be described with reference to Fig. 34 to Fig. 38. Fig. 34 and Fig. 35 are sequence diagrams showing the process of ending an event. Fig. 36 is a diagram showing an event end screen on an electronic whiteboard. Fig. 37 is a diagram showing a file data upload screen on an electronic whiteboard. Fig. 38 is a diagram showing a file data upload completion screen on an electronic whiteboard.

[0190] In FIG. 34, when user A closes the screen displayed on display 220, reception unit 22 receives a notice to end the event being executed (step S301).

[0191] Then, the transmitting / receiving unit 21 transmits to the sharing support server 6 execution event start / end information indicating the start date / time and end date / time of the execution event, and file data registration request information indicating a file data registration request (step S302). This execution event start / end information includes the execution event ID, event name, and start date / time and end date / time of the event. Furthermore, this file data registration request information includes the execution event ID, the sender user ID, and the file data. As a result, the transmitting / receiving unit 61 of the sharing support server 6 receives the execution event start / end information and file data registration request information.

[0192] Next, the generation unit 64 of the sharing support server 6 generates a unique content processing ID for identifying the content processing that occurred due to the event (step S303). The generation unit 64 also generates a URL for content data that indicates the content details (step S304). The storage / readout processing unit 69 then associates the type of content processing, the start date and time and the end date and time of the content processing, the content processing ID generated in step S303, and the URL indicating the save destination of the content generated in step S304 with each execution event ID received in step S302 in the content management DB 6005 (see FIG. 10(B)), and manages these (step S305).

[0193] Next, the storage / readout processing unit 69 of the sharing support server 6 searches the execution event management DB 6004 (see FIG. 10A) using the execution event ID received in step S72 as a search key to read out the corresponding project ID (step S306). Next, the storage / readout processing unit 69 searches the user authentication management DB 6001 (see FIG. 9A) using the user ID of the content sender as a search key to read out the corresponding organization ID (step S307).

[0194] Next, the storage / read processing unit 69 searches the access management DB 6002 (see FIG. 9(B)) using the organization ID read in step S92 as a search key, and reads out the corresponding access ID and access password (step S308).

[0195] 35, the transmitter / receiver 61 transmits the execution event start / end information received in step S302 and file data registration request information indicating a file data registration request to the schedule management server 8 (step S309). This file data registration request information includes the project ID read in step S306, the execution event ID received in step S302, the sender's user ID and file data, the content processing ID generated in step S303, the URL of the file data generated in step S304, the access ID and access password read in step S308, and the start date / time and end date / time of the content processing. As a result, the transmitter / receiver 81 of the schedule management server 8 receives the execution event start / end information and the file data registration request information.

[0196] Next, in the schedule management server 8, the authentication unit 82 authenticates the sharing support server 6 using the access ID and access password (step S310). This authentication is the same process as step S38, so a description thereof will be omitted. Here, the description will continue for the case where the server is legitimate.

[0197] Next, the storage / readout processing unit 89 of the schedule management server 8 stores the execution event start / end information received in step S309 in the execution event management DB 8009 (see FIG. 14(B)) (step S311). In this case, the storage / readout processing unit 89 adds one record's worth of execution event start / end information to the execution event management table of the execution event management DB 8009.

[0198] Next, the storage / read processing unit 89 stores and manages the various data (information) received in step S309 in the execution event history management DB 8008 (step S312). The storage / read processing unit 89 stores the various data (information) including the file data in association with the project ID and execution event ID received in step S309 in the execution event history management DB 8008 (see FIG. 14(A)). In this way, the schedule management server 8 manages the same file data as on the sharing support server 6 side.

[0199] Next, the transmitting / receiving unit 81 transmits file data registration information indicating that the file data has been registered to the sharing support server 6 (step S313). As a result, the transmitting / receiving unit 61 of the sharing support server 6 receives the file data registration information.

[0200] Then, the transmitting / receiving unit 61 of the sharing support server 6 transmits the file data registration information transmitted from the schedule management server 8 to the interactive whiteboard 2 (step S314). As a result, the transmitting / receiving unit 21 of the interactive whiteboard 2 receives the file data registration information.

[0201] Next, when the file data registration information is received by the transmitting / receiving unit 21, the storage / readout processing unit 29 of the electronic whiteboard 2 deletes the file data stored in a specific storage area of ​​the storage unit 2000 (step S315). In this way, the electronic whiteboard 2 can reduce the risk of leakage of meeting information and the like by deleting the file data that has already been sent to the sharing support server 6.

[0202] Here, we will explain the transition of screens displayed on the interactive whiteboard 2 when an ongoing event is ended. First, in step S301, when the accepting unit 22 accepts the end of the ongoing event, the display control unit 24 displays an event end screen 270 as shown in FIG. 36 on the display 220. This event end screen 270 includes a toolbar area 271, a file display area 272, a file upload selection area 273, a "Close" button 278 that is pressed to proceed with the end process of the ongoing event, and a "CANCEL" button 279 that is pressed to cancel the end process of the ongoing event. The toolbar area 271 displays images representing icons r1, r2, and r3 as shown in FIG. 26. The file display area 272 displays file data images 272a, 272b, and 272c for identifying file data of material files stored in a specific storage area of ​​the storage unit 2000. The file upload selection area 273 includes a check box for accepting a selection as to whether or not the file data corresponding to the file data image displayed in the file display area 272 is to be uploaded to the sharing support server 6.

[0203] When the reception unit 22 receives selection of the "Close" button 278 while the file upload selection area 273 is selected, the display control unit 24 displays a file upload screen 280a as shown in FIG. 37 on the display 220. This file upload screen 280a is displayed on the display 220 while file data stored in a specific storage area of ​​the storage unit 2000 is being uploaded to the sharing support server 6. The file upload screen 280a includes an event name 281 of the event to be ended, an event end date and time 282, a file data upload progress display area 283, and a "CANCEL" button 288 that is pressed to interrupt (cancel) the file data upload. The upload progress display area 283 displays the number of file data to be uploaded (e.g., "3" shown in FIG. 37) and the number of file data that have been uploaded (e.g., "0" shown in FIG. 37).

[0204] Then, when the upload of the file data is completed, the display control unit 24 displays an upload completion screen 280b as shown in Fig. 38. The upload completion screen 280b includes a "Close" button 288 that is pressed to end the event. When the upload completion screen 280b is displayed on the display 220, the storage / readout processing unit 29 of the interactive whiteboard 2 deletes the uploaded file data from a specific storage area of ​​the storage unit 2000, as shown in step S315.

[0205] On the other hand, if uploading of file data fails while the file upload screen 280a is displayed on the display 220, the display control unit 24 displays information (for example, a file name) for identifying the file data for which uploading failed. This allows, for example, in the event of a failure or the like in the communication network 10, event participants to take home the file data for which uploading failed by printing the file data created or edited at the event or by storing the file data in the USB memory 2600 connected to the interactive whiteboard 2.

[0206] Furthermore, if file data remains in a specific storage area of ​​the storage unit 2000 even after the event has ended, the storage / readout processing unit 29 of the electronic whiteboard 2 deletes the file data stored in the specific storage area, for example, at the start of the next event using the electronic whiteboard 2. This allows the electronic whiteboard 2 to reduce the risk of leaking meeting information and the like.

[0207] <Viewing execution event history> Next, the processing for viewing the execution event history will be described with reference to Fig. 39 to Fig. 45. Fig. 39 and Fig. 40 are sequence diagrams showing the processing for viewing the execution event history. Fig. 41 is a diagram showing a project list screen on a PC. Fig. 42 is a diagram showing an execution event list screen on a PC. Figs. 43 and 44 are diagrams showing an execution event history screen. Fig. 45 is a diagram showing an action item screen on a PC. Note that the processing of steps S111 to S117 in Fig. 39 is the same as the processing of steps S11 to S17 in Fig. 16, and therefore description thereof will be omitted.

[0208] When the user presses the "View Execution Event History" button 543 on the initial screen 540 shown in FIG. 18, the reception unit 52 receives a request to view the execution event history (step S118).

[0209] Then, the transmitting / receiving unit 51 transmits executed event history viewing request information indicating a request to view an executed event history to the schedule management server 8 (step S119). As a result, the transmitting / receiving unit 81 of the schedule management server 8 receives the executed event history viewing request information.

[0210] Next, the storage / readout processing unit 89 of the schedule management server 8 searches the project member management DB 8007 using the user ID and organization ID received in step S113 as search keys to read out the corresponding project ID and project name (step S120).Then, the transmission / reception unit 81 transmits the project ID and project name to the PC 5 (step S121).

[0211] Next, the generation unit 56 of the PC 5 generates a project list screen 560 as shown in Fig. 41 using the project ID and project name received in step S121 (step S122). Then, in the PC 5, the display control unit 54 displays the project list screen 560 generated by the generation unit 56 on the display 508 (step S123). As shown in Fig. 24, this project list screen 560 displays the same content as the project list screen 240 displayed on the interactive whiteboard 2. That is, the project icons 561 to 566 and buttons 568 and 569 in Fig. 41 correspond to the project icons 241 to 246 and buttons 248 and 249 in Fig. 24, respectively.

[0212] Next, in FIG. 40, when user A presses project icon 561 with, for example, mouse 512, acceptance unit 52 accepts the selection of the project indicated by project icon 561 (step S124).

[0213] Next, the transmitting / receiving unit 51 of the PC 5 transmits the project ID of the project selected in step S124 to the schedule management server 8 (step S125). As a result, the schedule management server 8 receives the project ID.

[0214] Next, in the schedule management server 8, the storage / readout processing unit 89 searches the execution event history management DB 8008 using the project ID received in step S125 as a search key to read out the corresponding execution event ID (step S126). In this case, the storage / readout processing unit 89 reads out all execution event IDs associated with the project ID received in step S125. Then, the storage / readout processing unit 89 searches the execution event management DB 8009 using the execution event ID read out in step S126 as a search key to read out the corresponding execution event start and end information (step S127). This execution event start and end information includes the execution event ID, event name, start date and time, and end date and time.

[0215] Then, the transmitting / receiving unit 81 transmits the execution event ID, event name, start date and time, and end date and time read in step S127 to the PC 5 (step S128). As a result, the transmitting / receiving unit 51 of the PC 5 receives the execution event ID, event name, start date and time, and end date and time.

[0216] Next, the generation unit 56 of the PC 5 generates an execution event list screen 570 as shown in FIG. 42 using the data (information) received in step S128 (step S129). Then, in the PC 5, the display control unit 54 displays the execution event list screen 570 generated by the generation unit 56 on the display 508 (step S130). As shown in FIG. 42, the execution event list screen 570 includes information 571 to 573 indicating execution events. For example, the information 571 to 573 indicating execution events includes the execution event name, the start date and time, and the end date and time of the event. The information 571 to 573 indicating execution events is an example of a history information selection area that accepts selection for displaying history information indicating the execution content of the event. In addition, the lower right portion of the execution event list screen 570 includes a "close" button 575 that is pressed to close the execution event list screen 570. In addition, the lower left portion of the execution event list screen 570 includes an "action item" button 577 that is pressed to view action items. "Action Items" button 577 is an example of an action item selection area that accepts selections for displaying action items.

[0217] Next, the reception unit 52 of the PC 5 receives a selection of an execution event included in the execution event list screen 570 (step S131). Specifically, when the user selects information indicating an execution event included in the execution event list screen 570, the reception unit 52 receives the selection of the information indicating the execution event. Then, the transmission / reception unit 51 of the PC 5 transmits the execution event ID of the execution event selected by the reception unit 52 to the schedule management server 8 (step S132). As a result, the transmission / reception unit 81 of the schedule management server 8 receives the execution event ID.

[0218] Next, the storage / readout processing unit 89 of the schedule management server 8 searches the execution event history management DB 8008 using the execution event ID received in step S132 as a search key, and reads out the corresponding execution event history information (step S133). This execution event history information includes the content processing ID, the type of content processing, and the start and end dates and times of the content processing.

[0219] The storage / readout processing unit 89 of the schedule management server 8 searches the related information management DB 8010 using the execution event ID received in step S132 as a search key to read out corresponding related information (step S134). This related information includes the content occurrence time period, content processing ID, content processing type, and sequence number. The content occurrence time period included in the related information is an example of time information.

[0220] Then, the storage / readout processing unit 89 reads out the content data indicating the details of the execution event history from the storage location of the content data indicating the details of the execution event history read out in step S133 (step S135). Then, the transmission / reception unit 81 transmits the read out related information and content data to the PC 5 (step S136). As a result, the transmission / reception unit 51 of the PC 5 receives the related information and content data.

[0221] Next, the sound control unit 58 of the schedule management server 8 sets the playback position of the sound data, which is the content data received in step S136 (step S137). In this case, the sound control unit 58 sets the playback position of the sound data, which corresponds to the sound data associated with the content occurrence time period "00:00" in the received related information, as the playback position of the sound data.

[0222] Next, the generation unit 56 of the PC 5 generates an execution event history screen 580 as shown in FIG. 43 using the related information and content data received in step S136 (step S138). More specifically, the generation unit 56 generates the execution event history screen 580 so that images related to the text data, which is the received content data, are displayed in the text data display area 582 in the order of the sequence numbers included in the received related information. The generation unit 56 also generates the execution event history screen 580 so that images related to the screen data, which is the received content data, are displayed in the snapshot display area 583 in the order of the sequence numbers included in the received related information. Furthermore, the generation unit 56 generates the execution event history screen 580 so that a playback portion identification image 581p included in the playback portion display area 581 is displayed at a position related to the sound data set in step S137 within the entire playback time of the sound data.

[0223] Then, the display control unit 54 of the PC 5 causes the display 508 to display the execution event history screen 580 generated by the generation unit 56 (step S139). Furthermore, the sound control unit 58 of the PC 5 plays back the sound data of the playback point set in step S137. As shown in FIG. 43 , the execution event history screen 580 displays content data generated by executed events for each type of content processing. The execution event history screen 580 includes a playback point display area 581 that displays the playback point of the recorded data, a text data display area 582 that displays the audio text, a snapshot display area 583 that displays snapshots, an "action item" button 584 that is pressed to view action items, a "material file" button 585 that is pressed to view materials used in the event, and a "pagination button" 588 that is pressed to display a pagination display area 589 (described later).

[0224] Here, the playback location display area 581 includes a playback location specifying image 581p indicating the playback location within the total playback time, a slider 581b indicating the playback history (playback process) of the sound recording data, and a playback time display image 581t indicating the playback time within the total playback time. The playback location specifying image 581p and the slider 581b form a seek bar. The seek bar displays the playback location of the sound recording data and is an operation area for specifying the playback location of the sound recording data. The user can visually grasp which part of the sound recording data is currently being played from the beginning to the end by looking at the positions of the playback location specifying image 581p and the slider 581b. The user can also move the playback location specifying image 581p using input means such as the mouse 507, thereby playing back the sound recording data from any playback location. The playback location specifying image 581p is an example of a first image. The seek bar including the playback location specifying image 581p and the slider 581b is an example of a second image.

[0225] Furthermore, in text data display area 582, images 582a, 582b, 582c, 582d, and 582e showing text data are displayed in chronological order from top to bottom. Similarly, in snapshot display area 583, snapshot (capture) images 583a, 583b, and 583c are displayed in chronological order from top to bottom. In this manner, text data display area 582 and snapshot display area 583 are displayed in parallel in chronological order. This allows a user to easily understand, for example, the time period when a pie chart was explained in a conference by viewing image 583b, and thus easily find images 582c and 582d showing text data displayed parallel to image 583b. In this case, images 583a, 583b, and 583c are snapshot images of the entire display 220. In this manner, in execution event history screen 580, images of the corresponding content are displayed in chronological order in specific display areas for each type of content (text data, snapshot).

[0226] FIG. 44 shows an image that appears on the display 508 when the user presses the "Pagination" button 588 shown in FIG. 43. Here, "pagination" is a navigation system designed to move between pages when information is expanded or divided into multiple pages. For example, in the case of a web page, "pagination" refers to dividing a page with a lot of content into multiple pages and arranging links to each page. Using "pagination" makes it easier for users to access and understand which page they are viewing. "Pagination" is also called "page division," "page forwarding," "paging," or "pager."

[0227] When the user presses the "Pagination" button 588, the reception unit 52 accepts the pressing of the "Pagination" button 588. As a result, the display control unit 54 displays a pagination display area 589 at the bottom of the executed event history screen 580, as shown in FIG. 44. In this pagination display area 589, for example, the time periods (here, every 5 minutes) in which content occurred in an event such as a conference are displayed, and the time periods of occurrence that are the same as the time of the playback point in the playback point identification image 581p are enclosed in a frame (elapsed time display area 589j). Note that the reception unit 52 remains in a waiting state for the "Pagination" button 588 to be pressed until it accepts the pressing of the button.

[0228] This allows the shared system 1 to easily share the execution event history generated by events belonging to the same project among members belonging to the same project. Also, the shared system 1 can omit processing such as viewing settings for each executed event so that users can view the execution event history managed for each project by registering or changing members belonging to the project using the project registration screen 520 displayed on the PC 5.

[0229] On the other hand, if the accepting unit 52 of the PC 5 accepts selection of the "Action Item" button 577 on the execution event list screen 570 in step S131, the generating unit 56 generates an action item screen 590 as shown in FIG. 45. Then, the display control unit 54 displays the action item screen 590 generated by the generating unit 56 on the display 508. As shown in FIG. 45, the action item screen 590 includes action item information 591 to 594. For example, the action item information 591 includes an image showing the content of the action item identified in FIG. 31, the user name selected in FIG. 32, and the deadline date specified in FIG. 33. In addition, the lower right portion of the action item screen 590 includes a "Close" button 599 that is pressed to close the action item screen 590. The action item screen 590 shown in FIG. 45 displays all action items associated with the project ID whose selection was accepted in step S124. In addition, when the reception unit 52 receives the selection of the “Action Item” button 584 on the execution event history screen 580 shown in FIG. 43, the display control unit 54 similarly displays the action item screen 590 on the display 508.

[0230] <Selection process of execution event history> Next, the selection process of the execution event history will be described with reference to Fig. 46 to Fig. 51. Fig. 46 and Fig. 49 to Fig. 51 are sequence diagrams showing the selection process of the execution event history. Fig. 47 and Fig. 48 are diagrams showing the execution event history screen on a PC.

[0231] First, the reception unit 52 of the PC 5 receives a selection of a specific image included in the text data display area 582 of the execution event history screen 580 (step S151). Specifically, for example, as shown in Fig. 47, when the user selects an image (e.g., image 582t) indicating specific text data included in the text data display area 582 of the execution event history screen 580 using the mouse pointer 586, the reception unit 52 receives the selection of the specific image included in the text data display area 582.

[0232] The transmitting / receiving unit 51 of the PC 5 transmits information related to the selection accepted by the accepting unit 52 to the schedule management server 8 (step S152). Here, the selection information includes a content processing ID related to the specific image (e.g., image 582t) whose selection has been accepted. As a result, the transmitting / receiving unit 81 of the schedule management server 8 receives the selection information transmitted from the PC 5.

[0233] Next, the storage / readout processing unit 89 of the schedule management server 8 searches the related information management DB 8010 using the content processing ID included in the selection information received in step S152 as a search key, thereby retrieving the corresponding related information (step S153). More specifically, the storage / readout processing unit 89 retrieves the related information associated with the content processing ID included in the selection information from the related information management DB 8010. This related information includes the content occurrence time period, content processing ID, content processing type, and sequence number. The content occurrence time period included in the related information is an example of time information. This allows the schedule management server 8 to identify content that occurred in the same time period as the image data input by the user, from the data (information) related to various content included in the related information.

[0234] The storage / readout processing unit 89 of the schedule management server 8 searches the execution event history management DB 8008 using the content processing ID included in the related information read out in step S153 as a search key, thereby reading out the corresponding execution event history information (step S154). This execution event history information includes the content processing ID, the type of content processing, and the content details.

[0235] Then, the storage / readout processing unit 89 reads out content data indicating the details of the execution event history from the storage location of the content data indicating the details of the content included in the execution event history information read out in step S154 (step S155). Then, the transmission / reception unit 81 transmits the read related information and content data to the PC 5 (step S156). As a result, the transmission / reception unit 51 of the PC 5 receives the related information and content data.

[0236] Next, the sound control unit 58 of the PC 5 sets the playback position corresponding to the sound data, which is the content data received in step S156, as the playback position of the sound data (start time of the sound data) (step S157).

[0237] Next, the generation unit 56 of the PC 5 generates an execution event history screen 580 based on the content data received in step S156 (step S158). Specifically, the generation unit 56 generates the execution event history screen 580 so that an image related to the text data, which is the received content data, is displayed in a text data display area 582. The generation unit 56 also generates the execution event history screen 580 so that an image related to the screen data, which is the received content data, is displayed in a snapshot display area 583. Furthermore, the generation unit 56 generates the execution event history screen 580 so that a playback point identification image 581p included in a playback point display area 581 is displayed at a playback point related to the total playback time of the sound data, which is the received content data.

[0238] Furthermore, the generation unit 56 generates the execution event history screen 580 so that the playback point identification image 581p included in the playback point display area 581 is displayed at the position related to the sound data set in step S136 during the entire playback time of the sound data.

[0239] The display control unit 54 causes the display 508 to display the execution event history screen 580 generated in step S158 (step S159). The execution event history screen 580 displayed on the display 508 in step S159 is shown in FIG. 48. In the execution event history screen 580 shown in FIG. 48, the position of the playback location identification image 581p included in the playback location display area 581 has moved to the time position corresponding to the image 582t selected by the user. In this manner, when a specific image is selected from among images related to text data obtained by converting sound data into text, the PC 5 moves the playback location identification image 581p to the time position indicated by the occurrence time zone of the content related to the selected specific image. This allows the PC 5 to play back the sound data at the playback location corresponding to the image simply by selecting an image related to text data whose audio the user wants to check. Furthermore, by moving the playback location identification image 581p in accordance with the movement of the image related to the text data, the PC 5 can easily grasp which playback location within the entire playback time the sound data is being played back.

[0240] 49 to 51 are processes performed when content data (execution event history) of a type different from the text data (execution event history) shown in FIG. 46 is selected. FIG. 49 shows processes performed when an image related to screen data displayed in the snapshot display area 583 is selected. First, the reception unit 52 of the PC 5 receives a selection of a specific image included in the snapshot display area 583 of the execution event history screen 580 (step S161). Specifically, for example, when the user selects a specific image (e.g., snapshot image 583r) included in the snapshot display area 583 of the execution event history screen 580 using the mouse pointer 586, the reception unit 52 receives the selection of the specific image included in the snapshot display area 583. Note that the processes of steps S162 to S169 are similar to the processes of steps S152 to S159 shown in FIG. 46, respectively, and therefore will not be described.

[0241] 50 shows a process when an image related to screen data displayed in the pagination display area 589 is selected. First, the reception unit 52 of the PC 5 receives a selection of an image showing a specific occurrence time period included in the pagination display area 589 of the execution event history screen 580 (step S171). Specifically, for example, when the user selects a specific elapsed time display image (e.g., elapsed time display area 589j) included in the pagination display area 589 of the execution event history screen 580 using the mouse pointer 586, the reception unit 52 receives a selection of a specific image included in the snapshot display area 583.

[0242] The transmitting / receiving unit 51 of the PC 5 transmits selection information relating to the selection accepted by the accepting unit 52 to the schedule management server 8 (step S172). Here, the selection information includes the content occurrence time period relating to the elapsed time display image whose selection has been accepted. As a result, the transmitting / receiving unit 81 of the schedule management server 8 receives the selection information transmitted from the PC 5.

[0243] Next, the storage / readout processing unit 89 of the schedule management server 8 searches the related information management DB 8010 using the content occurrence time period included in the selection information received in step S172 as a search key, thereby retrieving corresponding related information (step S173). More specifically, the storage / readout processing unit 89 retrieves related information associated with the content occurrence time period included in the selection information from the related information management DB 8010. This related information includes the content occurrence time period, content processing ID, content processing type, and sequence number. This allows the schedule management server 8 to identify the content that occurred during the time period indicated in the elapsed time display image for which the user's selection was accepted, from the data (information) related to various contents included in the related information. Note that the processing in steps S174 to S179 is similar to the processing in steps S154 to S159 shown in FIG. 46, and therefore description thereof will be omitted.

[0244] 51 shows a process when a playback location specifying image 581p displayed in the playback location display area 581 is selected. First, the reception unit 52 of the PC 5 receives a request to move the playback location specifying image 581p included in the playback location display area 581 on the execution event history screen 580 (step S181). Specifically, for example, when the user moves the playback location specifying image 581p included in the playback location display area 581 on the execution event history screen 580 using the mouse pointer 586, the reception unit 52 receives a request to move the playback location specifying image 581p included in the playback location display area 581.

[0245] The transmitting / receiving unit 51 of the PC 5 transmits the selection information related to the move accepted by the accepting unit 52 to the schedule management server 8 (step S182). Here, the selection information includes the content processing ID related to the sound data of the accepted playback location of the move destination. As a result, the transmitting / receiving unit 81 of the schedule management server 8 receives the selection information transmitted from the PC 5. Note that the processes of steps S182 to S189 are similar to the processes of steps S152 to S159 shown in FIG. 46, respectively, and therefore description thereof will be omitted.

[0246] 39 to 51 have described the case where the execution event history is viewed by the PC 5, but the execution event history can also be viewed by the same process on the electronic whiteboard 2 when the user presses icon r2 in Fig. 26. Furthermore, the processes and screen examples shown in Fig. 39 to 51 may be performed not only on the PC 5 but also on the electronic whiteboard 2, the videoconferencing terminal 3, or the car navigation device 4.

[0247] 39 to 51, the image data for each screen to be displayed on the PC 5 is generated by the generation unit 56 of the PC 5, but this image data may be generated by the schedule management server 8. In this case, in the schedule management server 8, the image data is generated by the generation unit 83, and the generated image data is transmitted to the PC 5 by the transmission / reception unit 81.

[0248] <<Main Effects of the Embodiment>> According to this embodiment, as shown in FIGS. 46 to 48, PC 5 is a display terminal capable of communicating with schedule management server 8 (an example of a management system) that manages sound data and text data converted from the sound data. Furthermore, PC 5's transceiver 51 receives sound data, text data, and time information related to the occurrence time of the text data from schedule management server 8. Display control unit 54 (an example of display control means) displays the received text data on display 508 (an example of a display unit) in the order of the occurrence time of the text data, and also displays playback portion identification image 581p (an example of a first image) indicating the playback portion of the received sound data within the total playback time of the sound data. Acceptance unit 52 (an example of acceptance means) accepts selection of specific text data from the displayed text data. Display control unit 54 of PC 5 then displays playback portion identification image 581p at the time position indicated by the time information corresponding to the specific text data whose selection was accepted. This allows a user of PC 5 to easily find the playback portion of the desired audio data by viewing the displayed text data and playing back the audio data corresponding to that text data.

[0249] According to this embodiment, as shown in FIGS. 46 to 51, the PC 5 is a display terminal capable of communicating with a schedule management server 8 (an example of a management system) that manages sound data and content image data related to content generated in an event. The transmission / reception unit 51 of the PC 5 receives time information related to the sound data, content image data, and content occurrence time from the schedule management server 8. The display control unit 54 (an example of a display control means) displays content images related to the content image data on the display 508 in order of content occurrence time based on the received time information, and also displays a playback location identification image 581p (an example of a first image) indicating the playback location within the total playback time of the sound data on the display 508. The reception unit 52 (an example of a reception means) accepts the selection of a specific content image from among the displayed content images. The display control unit 54 of the PC 5 then displays the playback location identification image 581p at the time position indicated by the time information corresponding to the specific content image whose selection was accepted. This allows the user of the PC 5 to easily find the playback location of the desired audio data when viewing the displayed image and wanting to play the audio data corresponding to that image.

[0250] 46 to 51, the sound control unit 58 (an example of a playback means) of the PC 5 (an example of a display terminal) plays back sound data relating to the time indicated in the time information corresponding to a specific content image, and the display control unit 54 (an example of a display control means) displays the playback portion of the sound data being played back as a playback portion identification image 581p (an example of a first image). This allows the PC 5 to play back the sound data of the playback portion corresponding to a content image simply by the user selecting the image.

[0251] 46 to 51, the display control unit 54 (an example of a display control means) of the PC 5 (an example of a display terminal) displays the text data, which is the content image data, and the images related to the screen data in parallel in chronological order. This allows the user to easily grasp the time period in which the pie chart was explained in the meeting by viewing, for example, image 583b, and thus easily find images 582c and 582d showing the text data displayed in parallel with image 583b.

[0252] 46 to 51, the display control unit 54 (an example of a display control means) of the PC 5 (an example of a display terminal) displays on the display 508 (an example of a display unit) a seek bar (an example of a second image) that includes a playback location specifying image 581p (an example of a first image) and that can change the playback location within the entire playback time of the sound data. In this way, the PC 5 moves the playback location specifying image 581p in accordance with the movement of the image related to the text data, thereby allowing the user to easily understand which playback location within the entire playback time the sound data relates to.

[0253] <<Supplementary Information>> Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices designed to perform each of the above-described functions, such as an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), a system on a chip (SOC), a graphics processing unit (GPU), or a conventional circuit module. [Explanation of symbols]

[0254] 1 Shared System 2. Electronic whiteboard (an example of a communication terminal) 3. Video conferencing terminal (an example of a communication terminal) 4. Car navigation device (an example of a communication terminal) 5 PC (example of display terminal) 6 Shared Support Server 8 Schedule management server (an example of a management system) 9. Speech-to-text server 10. Communication Networks 21 Transmitter / Receiver 22 Reception 24 Display control unit 29 Storage and readout processing section 51 Transmitting / receiving unit (an example of receiving means) 52 Reception unit (an example of reception means) 54 Display control unit (an example of a display control means) 56 Generation unit (an example of generation means) 58 Sound control unit (an example of a playback means) 508 Display (an example of a display unit) 581p Playback location specific image (example of the first image) 81 Transmitting / receiving unit (an example of a transmitting means) [Prior art documents] [Patent documents]

[0255] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-105241 [Patent Document 2] Japanese Patent Application Laid-Open No. 2017-129720 [Patent Document 3] Japanese Patent Application Laid-Open No. 2017-173768

Claims

1. A receiving means for receiving sound data, a plurality of image data, and time information associated with each of the plurality of image data from a server device, A display control means that displays the received plurality of image data in chronological order in a first area based on the time information, and displays first image data indicating the playback location in the total playback time of the sound data in a second area, A receiving means for receiving the selection of one image data from the plurality of image data displayed in the first region, Equipped with, When the receiving means receives the selection of the first image data, the display control means moves the first image data to a position corresponding to the time information associated with the selected first image data within the total playback time of the sound data. Display device.

2. The display device according to claim 1, wherein the sound data and the image data are generated by an event.

3. The display device according to claim 2, wherein the image data is image data of the display screen at a certain point in time of the event.

4. The receiving means receives a plurality of text data converted from the sound data, The display control means displays a third area for displaying the received plurality of text data. The display device according to any one of claims 1 to 3.

5. The display device according to claim 4, wherein when the receiving means receives the selection of the first image data, the display control means causes the text data corresponding to the time of the sound data corresponding to the time information associated with the selected first image data to be displayed in the third area.

6. The receiving means receives the selection of one text data from the plurality of text data displayed in the third area, The display control means moves the first image data to the time of the audio data corresponding to the selected text data within the total playback time of the audio data. The display device according to claim 4.

7. The display device according to any one of claims 1 to 6, wherein the display control means further displays a second image data showing the playback history of the sound data in the second area.

8. The display device according to claim 7, wherein the first image data and the second image data constitute a seek bar that can change the playback position over the entire playback time of the sound data.

9. A display method performed on a display device, A receiving process that receives sound data, multiple image data, and time information associated with each of the multiple image data from a server device. A display control step which involves displaying the received plurality of image data in chronological order in a first area based on the time information, and displaying a first image data indicating the playback location in the total playback time of the sound data in a second area, A receiving step of receiving the selection of one image data from the plurality of image data displayed in the first region, Equipped with, The display control step, upon receiving the selection of the first image data in the reception step, moves the first image data to a position corresponding to the time information associated with the selected first image data within the total playback time of the sound data. Display method.

10. A computer, A receiving means that receives sound data, multiple image data, and time information associated with each of the multiple image data from a server device. A display control means that displays the received plurality of image data in chronological order in a first area based on the time information, and displays first image data indicating the playback location in the total playback time of the sound data in a second area, A receiving means for receiving the selection of one image data from the plurality of image data displayed in the first region, It is designed to function as such. When the receiving means receives the selection of the first image data, the display control means moves the first image data to a position corresponding to the time information associated with the selected first image data within the total playback time of the sound data. program.

Images