Terminal, shared system, display control method, and program

Abstract

To provide a display screen that allows a user to search for a playback part in sound data that the user wants to play back.SOLUTION: In a PC 5, a transmitting / receiving unit 51 receives, from a schedule management server 8, sound data, text data converted from the sound data, and time information related to a time of generation of the text data. A display control unit 54 displays, on a display 508, the received text data in order of the time of generation of the text data, and displays, on the display 508, a playback part specification image 581p that shows a playback part in a total playback time of the received sound data. A receiving unit 52 receives selection for specific text data from the displayed text data. The display control unit 54 displays the playback part specification image 581p at a position of time indicated in time information corresponding to the specific text data selection for which has been received.SELECTED DRAWING: Figure 48

Description

Technical Field

[0001] The present disclosure relates to a display terminal, a sharing system, a display control method, and a program.

Background Art

[0002] In recent years, in meetings and the like in companies, educational institutions, administrative organizations, etc., an electronic blackboard that displays a background image on a display and allows a user to draw a stroke image such as characters, numbers, and figures on this background image has been used (for example, Patent Document 1).

[0003] Also, when an event such as a meeting is held, a technique is known in which a server converts recording data in the event into text data, and a terminal displays an image related to the text data to automatically create minutes (for example, Patent Document 2). In addition, a technique is also known in which a terminal displays minutes including an image related to text data and plays back the recording data in the event (for example, Patent Document 3).

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional method, when a user wants to view a displayed image and play back the recording data corresponding to that image, the user has to select which recording data to play back within the total playback time. Therefore, there is a problem that the user cannot find the playback location of the recording data that the user wants to play back.

Means for Solving the Problems

[0005] In order to solve the above problems, the invention according to claim 1 is a display terminal capable of communicating with a management system that manages execution event history information including execution event identification information for identifying an execution event participated by a user, a storage destination of sound data generated in the execution event and text data converted from the sound data, and time information regarding the sound data and the text data. The display terminal transmits, to the management system, the execution event identification information of an execution event selected by the user on the display terminal from among one or more execution events participated by the user specified based on user identification information for identifying the user of the display terminal. Based on the execution event history information associated with the execution event identification information of the selected execution event in the management system, a receiving means for receiving the text data and the time information regarding the text data specified thereby from the management system, and the text data in the history information of a predetermined execution event selected by the user of the received display terminal is caused to be displayed on a display unit in chronological order based on the time information, and a first image indicating a reproduction position in the entire reproduction time of the sound data is displayed at a first position set as the reproduction position of the sound data in a display area of the display unit. A display control means for displaying, and a reception means for receiving a selection for predetermined text data among the displayed text data. The display control means displays the first image at a second position of the time indicated by the time information corresponding to the predetermined text data for which the selection has been received from the first position. The receiving means receives the audio data from the management system. It is a display terminal.

[0006] According to the present invention, there is an effect that the reproduction position of the sound data that the user wants to reproduce can be found.

Brief Description of Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28A

Figure 28B

Figure 28C

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Figure 43

Figure 44

Figure 45

Figure 46

Figure 47

Figure 48

Figure 49

Figure 50

Figure 51

Embodiments for Carrying Out the Invention

[0008] Hereinafter, the shared system 1 according to this embodiment will be described in detail with reference to the drawings. In this embodiment, "file" refers to "electronic file".

[0009] <<Outline of System Configuration>> First, the outline of the configuration of the shared system 1 will be described. FIG. 1 is a schematic diagram of the shared system according to this embodiment.

[0010] As shown in FIG. 1, the sharing system 1 of this embodiment is constructed by an electronic blackboard 2, a video conferencing terminal 3, a car navigation device 4, a PC (Personal Computer) 5, a sharing support server 6, a schedule management server 8, and a voice text conversion server 9.

[0011] Also, the electronic blackboard 2, the video conferencing terminal 3, the car navigation device 4, the PC 5, the sharing support server 6, the schedule management server 8, and the voice text conversion server 9 can communicate via a communication network 10. The communication network 10 is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), etc. The communication network 10 may include not only wired communication but also a network by wireless communication such as 3G (3rd Generation), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution).

[0012] Also, the electronic blackboard 2 is used in conference room X. The video conferencing terminal 3 is used in conference room Y. Also, the shared entity is subject to reservation by each user. The car navigation device 4 is used in vehicle α. Vehicle α in this case is a vehicle subject to car sharing. Vehicles include cars, motorcycles, bicycles, wheelchairs, etc.

[0013] A "shared entity" refers to an object, service, space (room), place, or information jointly used by a plurality of people or organizations. Conference room X, conference room Y, and vehicle α are examples of shared entities shared by a plurality of users. An example of information assigned to a shared entity is an account. For example, in a specific service provided on the Web, when the account used is limited to one.

[0014] The electronic blackboard 2, the video conferencing terminal 3, and the car navigation device 4 are examples of communication terminals. A "communication terminal" is, for example, a terminal that can be used for user sign-in (see S32 described later). Note that the communication terminals used in the vehicle α include not only the car navigation device 4 but also smartphones or smartwatches etc. on which a car navigation application is installed.

[0015] The PC 5 is an example of a display terminal and is an example of a registration device for registering reservations for using each shared facility and registering events that each user has scheduled to execute with respect to the schedule management server 8. An event is, for example, a meeting, a gathering, a get-together, a meeting, a consultation, a negotiation, driving, riding, or moving, etc.

[0016] The shared support server 6 is a computer and provides support when sharing a shared facility remotely with each communication terminal.

[0017] The schedule management server 8 is a computer and manages the reservations of each shared facility and the schedules of each user.

[0018] The voice text conversion server 9 is a computer and converts voice (audio) data received from an external computer (for example, the shared support server 6) into text data.

[0019] Here, the shared support server 6, the schedule management server 8, and the voice-text conversion server 9 are referred to as a management system. Note that the management system may be a computer that aggregates all or part of the functions of, for example, the shared support server 6, the schedule management server 8, and the voice-text conversion server 9. Also, each of the shared support server 6, the schedule management server 8, and the voice-text conversion server 9 may be configured to distribute and implement each function on a plurality of computers. Further, although the shared support server 6, the schedule management server 8, and the voice-text conversion server 9 are described as server computers existing in a cloud environment, the shared support server 6 and the voice-text conversion server 9 may be servers operating on a communication terminal such as the electronic blackboard 2, or may be servers existing in an on-premises environment. Also, the schedule management server 8 may be a server existing in an on-premises environment.

[0020] <<Hardware Configuration>> Subsequently, with reference to FIGS. 2 to 5, the hardware configuration of the device or terminal that constructs the shared system 1 will be described.

[0021] <Hardware Configuration of Electronic Blackboard> FIG. 2 is a hardware configuration diagram of the electronic blackboard. As shown in FIG. 2, the electronic blackboard 2 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an SSD (Solid State Drive) 204, a network I / F 205, and an external device connection I / F (Interface) 206.

[0022] Among these, the CPU 201 controls the operation of the entire electronic blackboard 2. The ROM 202 stores programs used for driving the CPU 201 such as the CPU 201 and the IPL (Initial Program Loader). The RAM 203 is used as a work area for the CPU 201. The SSD 204 stores various data such as programs for the electronic blackboard. The network I / F 205 controls communication with the communication network 10. The external device connection I / F 206 controls communication with the USB (Universal Serial Bus) memory 2600, the PC 2700, and external devices (the microphone 2200, the speaker 2300, the camera 2400).

[0023] Also, the electronic blackboard 2 includes a capture device 211, a GPU 212, a display controller 213, a touch sensor 214, a sensor controller 215, an electronic pen controller 216, a short-range communication circuit 219, an antenna 219a for the short-range communication circuit 219, and a power switch 222.

[0024] Among these, the capture device 211 acquires the image data displayed on the display 220, which is an example of a display unit (display means), via the display controller 213, and stores it in the RAM 203 or the like. The GPU (Graphics Processing Unit) 212 is a semiconductor chip specialized in handling graphics. The display controller 213 controls and manages the screen display in order to output the output image from the capture device 211 or the GPU 212 to the display 220 or the like. The touch sensor 214 detects that an electronic pen 2500 or the user's hand H or the like has touched the display 220. The sensor controller 215 controls the processing of the touch sensor 214. The touch sensor 214 performs coordinate input and coordinate detection by an infrared blocking method. The method of this coordinate input and coordinate detection is that two light emitting and receiving devices installed at both upper ends of the display 220 emit a plurality of infrared rays parallel to the display 220, and the light is reflected by a reflecting member provided around the display 220, and the light receiving element receives the light returning on the same optical path as the optical path of the light emitted. The touch sensor 214 outputs the IDs of the infrared rays emitted by the two light emitting and receiving devices blocked by the object to the sensor controller 215, and the sensor controller 215 specifies the coordinate position that is the contact position of the object. The electronic pen controller 216 determines the presence or absence of pen tip touch or pen butt touch on the display 220 by communicating with the electronic pen 2500. The short-range communication circuit 219 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The power switch 222 is a switch for switching the ON / OFF of the power of the electronic blackboard 2.

[0025] Furthermore, the electronic blackboard 2 includes a bus line 210. The bus line 210 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 201 shown in FIG. 2.

[0026] Note that the contact sensor 214 is not limited to the infrared interruption method, and various detection means such as a capacitance type touch panel that identifies the contact position by detecting a change in capacitance, a resistive film type touch panel that identifies the contact position by a voltage change between two opposing resistive films, and an electromagnetic induction type touch panel that detects electromagnetic induction generated when a contact object contacts the display unit and identifies the contact position may be used. Further, the electronic pen controller 216 may be configured to determine the presence or absence of touch not only at the tip and the butt of the electronic pen 2500 but also at the portion held by the user of the electronic pen 2500 and other portions of the electronic pen.

[0027] <Hardware Configuration of Video Conference Terminal> FIG. 3 is a diagram showing the hardware configuration of the video conferencing terminal. As shown in FIG. 3, the video conferencing terminal 3 includes a CPU 301, a ROM 302, a RAM 303, a flash memory 304, an SSD 305, a media I / F 307, operation buttons 308, a power switch 309, a bus line 310, a network I / F 311, a CMOS (Complementary Metal Oxide Semiconductor) sensor 312, an imaging device I / F 313, a microphone 314, a speaker 315, an audio input / output I / F 316, a display I / F 317, an external device connection I / F 318, a short-range communication circuit 319, and an antenna 319a of the short-range communication circuit 319. Among these, the CPU 301 controls the operation of the entire video conferencing terminal 3. The ROM 302 stores programs used for driving the CPU 301 such as IPL. The RAM 303 is used as a work area for the CPU 301. The flash memory 304 stores various data such as communication programs, image data, and audio data. The SSD 305 controls the reading or writing of various data to and from the flash memory 304 according to the control of the CPU 301. Note that an HDD may be used instead of the SSD. The media I / F 307 controls the reading or writing (storage) of data to and from a recording medium 306 such as a flash memory. The operation buttons 308 are buttons that are operated when selecting a destination of the video conferencing terminal 3 or the like. The power switch 309 is a switch for switching the ON / OFF of the power of the video conferencing terminal 3.

[0028] The network I / F 311 is an interface for data communication using a communication network 10 such as the Internet. The CMOS sensor 312 is a type of built-in imaging means that captures a subject according to the control of the CPU 301 to obtain image data. The imaging device I / F 313 is a circuit that controls the driving of the CMOS sensor 312. The microphone 314 is a type of built-in sound collection means for inputting sound. The audio input / output I / F 316 is a circuit that processes the input / output of audio signals between the microphone 314 and the speaker 315 according to the control of the CPU 301. The display I / F 317 is a circuit that transmits image data to an external display 320 according to the control of the CPU 301. The external device connection I / F 318 is an interface for connecting various external devices. The short-range communication circuit 319 is a communication circuit such as NFC or Bluetooth.

[0029] The bus line 310 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 301 shown in FIG. 3.

[0030] The display 320 is a type of display unit (display means) composed of liquid crystal or organic EL that displays an image of a subject, operation icons, etc. Also, the display 320 is connected to the display I / F 317 by a cable 320c. This cable 320c may be a cable for analog RGB (VGA) signals, a cable for component video, or a cable for DisplayPort, HDMI (High-Definition Multimedia Interface (registered trademark)), or DVI (Digital Video Interactive) signals.

[0031] Note that instead of the CMOS sensor 312, an imaging device such as a CCD (Charge Coupled Device) sensor may be used. External device connection I / F 318 can be connected to external devices such as an external camera, an external microphone, and an external speaker respectively by a USB cable or the like. When an external camera is connected, the external camera is driven preferentially over the built-in CMOS sensor 312 under the control of the CPU 301. Similarly, when an external microphone is connected or when an external speaker is connected, the external microphone or the external speaker is driven preferentially over the built-in microphone 314 or the built-in speaker 315 respectively under the control of the CPU 301.

[0032] Also, the recording medium 306 is configured to be detachable from the video conferencing terminal 3. Also, as long as it is a non-volatile memory that reads or writes data according to the control of the CPU 301, not limited to the flash memory 304, an EEPROM or the like may be used.

[0033] <Hardware Configuration of the Car Navigation Device> FIG. 4 is a hardware configuration diagram of the car navigation device. As shown in FIG. 4, the car navigation device 4 includes a CPU 401, a ROM 402, a RAM 403, an EEPROM 404, a power switch 405, an acceleration / azimuth sensor 406, a media I / F 408, and a GPS receiver 409.

[0034] Among these, the CPU 401 controls the operation of the entire car navigation device 4. The ROM 402 stores programs used for driving the CPU 401 such as the IPL. The RAM 403 is used as a work area for the CPU 401. The EEPROM 404 reads or writes various data such as the program for the car navigation device 4 according to the control of the CPU 401. The power switch 405 is a switch for switching the ON / OFF of the power supply of the car navigation device 4. The acceleration / azimuth sensor 406 is various sensors such as an electronic magnetic compass for detecting geomagnetism, a gyrocompass, and an acceleration sensor. The media I / F 408 controls the reading or writing (storage) of data to / from the recording medium 407 such as a flash memory. The GPS receiver 409 receives GPS signals from GPS satellites.

[0035] Also, the car navigation device 4 includes a long-distance communication circuit 411, an antenna 411a of the long-distance communication circuit 411, a CMOS sensor 412, an imaging device I / F 413, a microphone 414, a speaker 415, an audio input / output I / F 416, a display 417, a display I / F 418, an external device connection I / F 419, a short-distance communication circuit 420, and an antenna 420a of the short-distance communication circuit 420.

[0036] Among these, the long-distance communication circuit 411 is a circuit for receiving traffic information, road construction information, traffic accident information, etc. provided by external infrastructure outside the vehicle, and conversely, transmitting the vehicle position, emergency rescue signals, etc. to the outside. The external infrastructure is, for example, a road information guidance system such as the VICS (registered trademark) (Vehicle Information and Communication System). The CMOS sensor 412 is a type of built-in imaging means for capturing a subject according to the control of the CPU 401 to obtain image data. The imaging device I / F 413 is a circuit for controlling the driving of the CMOS sensor 412. The microphone 414 is a type of built-in sound collection means for inputting sound. The audio input / output I / F 416 is a circuit for processing the input / output of sound signals between the microphone 414 and the speaker 415 according to the control of the CPU 401. The display 417 is a type of display unit (display means) such as a liquid crystal or organic EL for displaying the image of the subject, various icons, etc., and has the function of a touch panel. The touch panel is a type of input means for the user to operate the car navigation device 4. The display I / F 418 is a circuit for displaying an image on the display 417. The external device connection I / F 419 is an interface for connecting various external devices. The short-distance communication circuit 420 is a communication circuit such as NFC or Bluetooth. Also, the car navigation device 4 includes a bus line 410. The bus line 410 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 401 shown in FIG. 4.

[0037] <Hardware Configuration of PC and Server> FIG. 5 is a hardware configuration diagram of a PC and a server. The PC 5 is constructed by a computer and, as shown in FIG. 5, includes a CPU 501, a ROM 502, a RAM 503, an HD 504, an HDD (Hard Disk Drive) controller 505, a media I / F 507, a display 508, a network I / F 509, a keyboard 511, a mouse 512, a CD-RW drive 514, a speaker 515, and a bus line 510.

[0038] Among these, the CPU 501 controls the operation of the entire PC 5. The ROM 502 stores programs used to drive the CPU 501 such as the IPL. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD (Hard Disk Drive) controller 505 controls the reading or writing of various data to and from the HD 504 according to the control of the CPU 501. The media I / F 507 controls the reading or writing (storage) of data to and from the recording medium 506 such as a flash memory. The display 508 displays various information such as a cursor, menu, window, characters, or images. The display 508 is an example of a display unit (display means). The network I / F 509 is an interface for data communication using the communication network 10. The keyboard 511 is a type of input means having a plurality of keys for inputting characters, numerical values, various instructions, etc. The mouse 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The CD-RW drive 514 controls the reading or writing of various data to and from the CD-RW (Compact Disc-ReWritable) 513 as an example of a removable recording medium. The speaker 515 outputs an audio signal according to the control of the CPU 501.

[0039] Also, the PC 5 includes a bus line 510. The bus line 510 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 501 shown in FIG. 5.

[0040] In addition, the shared support server 6 is constructed by a computer and, as shown in FIG. 5, includes a CPU 601, a ROM 602, a RAM 603, an HD 604, an HDD controller 605, a recording medium 606, a media I / F 607, a display 608, a network I / F 609, a keyboard 611, a mouse 612, a CD-RW drive 614, and a bus line 610. Since these have the same configurations as the CPU 501, the ROM 502, the RAM 503, the HD 504, the HDD controller 505, the recording medium 506, the media I / F 507, the display 508, the network I / F 509, the keyboard 511, the mouse 512, the CD-RW drive 514, and the bus line 510 in the PC 5 respectively, the description thereof is omitted.

[0041] Furthermore, the schedule management server 8 is constructed by a computer and, as shown in FIG. 5, includes a CPU 801, a ROM 802, a RAM 803, an HD 804, an HDD controller 805, a recording medium 806, a media I / F 807, a display 808, a network I / F 809, a keyboard 811, a mouse 812, a CD-RW drive 814, and a bus line 810. Since these have the same configurations as the CPU 501, the ROM 502, the RAM 503, the HD 504, the HDD controller 505, the recording medium 506, the media I / F 507, the display 508, the network I / F 509, the keyboard 511, the mouse 512, the CD-RW drive 514, and the bus line 510 in the PC 5 respectively, the description thereof is omitted.

[0042] In addition, the voice text conversion server 9 is constructed by a computer. As shown in FIG. 5, it includes a CPU 901, a ROM 902, a RAM 903, an HD 904, an HDD controller 905, a recording medium 906, a media I / F 907, a display 908, a network I / F 909, a keyboard 911, a mouse 912, a CD-RW drive 914, and a bus line 910. Since these have the same configuration as the CPU 501, ROM 502, RAM 503, HD 504, HDD controller 505, recording medium 506, media I / F 507, display 508, network I / F 509, keyboard 511, mouse 512, CD-RW drive 514, and bus line 510 in the PC 5 respectively, the description thereof is omitted.

[0043] Note that each of the above programs may be in an installable format or an executable format file and may be recorded on a computer-readable recording medium and distributed. Examples of the recording medium include a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), a Blu-ray Disc, an SD card, and the like. Further, the recording medium can be provided as a program product, either domestically or abroad.

[0044] Furthermore, the shared support server 6 may be constructed by a single computer, or may be constructed by a plurality of computers in which each part (function, means, or storage part) is divided and arbitrarily assigned. The same applies to the schedule management server 8 and the voice text conversion server 9.

[0045] <<Software Configuration of Electronic Blackboard>> Next, with reference to FIG. 6, the computer software installed in the electronic blackboard 2 will be described. Here, computer software (hereinafter referred to as software) is a program related to the operation of a computer and other information used for processing by the computer that conforms to the program. A program is a command for a computer, which is a combination that can obtain a single result. Also, what conforms to the program is not a direct command for the computer and thus cannot be called a program, but has a property similar to that of a program in that it defines the processing of the computer. For example, a data structure (a logical structure of data represented by the mutual relationship between data elements) corresponds to what conforms to the program.

[0046] Also, an application is a general term for software used to perform specific processing among the classifications of software. On the other hand, an operating system (hereinafter referred to as OS) is software that controls a computer and enables applications, etc. to use computer resources. The OS performs basic management and control of the computer, such as control of input / output, management of hardware such as memory and hard disks, and management of processes. Applications operate using the functions provided by the OS.

[0047] FIG. 6 is a diagram showing the software configuration of the electronic blackboard. As shown in FIG. 6, the OS 101, Launcher 102, schedule viewer 103a, file viewer 103b, and browser application 103c operate in the work area 15 of the RAM 203. The OS 101 is basic software that provides the basic functions of the electronic blackboard 2 and manages the whole of the electronic blackboard 2.

[0048] Launcher 102 is a launcher application that operates on OS 101. Launcher 102 manages, for example, the start and end of an event such as a meeting executed on the electronic blackboard 2, or manages external applications such as the schedule viewer 103a, file viewer 103b, and browser application 103c used during the execution of the event.

[0049] The schedule viewer 103a, file viewer 103b, and browser application 103c are external applications that operate on Launcher 102 (hereinafter, referred to as external application 103 when there is no particular need to distinguish). The external application 103 is executed independently of Launcher 102 and executes services or functions provided on OS 101. Note that FIG. 6 shows an example in which three external applications, the schedule viewer 103a, file viewer 103b, and browser application 103c, are installed on the electronic blackboard 2, but the number of external applications is not limited to this.

[0050] <<Software Configuration of PC>> Subsequently, with reference to FIG. 7, the computer software installed on PC 5 will be described. FIG. 7 is a diagram showing the software configuration of the PC. As shown in FIG. 7, the OS 5501, minutes application 5502a, and browser 5502b operate on the work area 5500 of the RAM 503. The OS 5501 is basic software that provides the basic functions of PC 5 and manages the whole of PC 5.

[0051] The minutes application 5502a and browser 5502b are applications for generating and displaying an execution event history screen, which is the minutes of an event executed by the electronic blackboard 2, using various data transmitted from the schedule management server 8. Note that FIG. 7 shows an example in which two external applications, the minutes application 5502a and browser 5502b, are installed on PC 5, but the number of external applications is not limited to this.

[0052] <<Functional Configuration of the Shared System>> Next, the functional configuration of this embodiment will be described with reference to FIGS. 8 to 15. FIG. 8 is a functional block diagram of the shared system. In FIG. 8, among each terminal, device, and server shown in FIG. 1, those related to the processes or operations described later are shown.

[0053] <Functional Configuration of the Electronic Blackboard> As shown in FIG. 8, the electronic blackboard 2 includes a transmission / reception unit 21, a reception unit 22, an image / audio processing unit 23, a display control unit 24, a determination unit 25, a recognition unit 26, an acquisition / providing unit 28, and a storage / reading processing unit 29. Each of these units is a function or means realized by the operation of any of the components shown in FIG. 2 according to instructions from the CPU 201 expanded onto the RAM 203 from the SSD 204. Also, the electronic blackboard 2 has a storage unit 2000 constructed by the RAM 203, SSD 204, or USB memory 2600 shown in FIG. 2.

[0054] (Functional Configurations of the Electronic Blackboard) Next, each component of the electronic blackboard 2 will be described. The transmission / reception unit 21 is realized by instructions from the CPU 201 shown in FIG. 2, as well as the network I / F 205 and the external device connection I / F 206, and performs transmission and reception of various data (or information) with other terminals, devices, or systems via the communication network 10.

[0055] The reception unit 22 is mainly realized by instructions from the CPU 201 shown in FIG. 2, as well as the contact sensor 214 and the electronic pen controller 216, and receives various inputs by the user.

[0056] The image / sound processing unit 23 stores the image data that is realized by the instructions from the CPU 201 shown in FIG. 2 and the capture device 211 and is displayed on the display 220. Also, the image / sound processing unit 23 performs image processing for displaying an image on the display 220, which is realized by the instructions from the CPU 201 shown in FIG. 2 and the GPU 212. Further, the image / sound processing unit 23 performs image processing on the image data obtained by the camera 2400 capturing a subject. Also, after the voice of the user is converted into a voice signal by the microphone 2200, the image / sound processing unit 23 performs voice processing on the sound data related to this voice signal. Further, the image / sound processing unit 23 outputs the voice signal related to the sound data to the speaker 2300 to output voice from the speaker 2300. Also, the image / sound processing unit 23 performs a process of converting the drawn image data obtained by being drawn on the display 220 with an electronic pen 2500 or by hand by the user into coordinate data. For example, when the electronic blackboard (2a) at an arbitrary base transmits coordinate data to the electronic blackboard (2b) at another base, the electronic blackboard (2b) displays the drawing of the same content on the display 220 on the side of the other electronic blackboard (2b) based on the coordinate data.

[0057] The display control unit 24 is realized by the instructions from the CPU 201 shown in FIG. 2 and the display controller 213 shown in FIG. 2, and causes the display 220 to display a drawn image or accesses the shared support server 6 using a web browser to display various screen data. Specifically, the display control unit 24 starts and executes the Launcher 102 and the external application 103 that operate on the OS 101 shown in FIG. 6, and causes the display 220 to display various screens drawn by the API (Application Programming Interface) provided by the OS 101.

[0058] The determination unit 25 is realized by the instructions from the CPU 201 shown in FIG. 2 and performs various determinations.

[0059] The recognition unit 26 is realized by an instruction from the CPU 201 shown in FIG. 2, and recognizes a designated area 262 surrounded on the display 220 as shown in FIG. 31 described later.

[0060] The acquisition / providing unit 28 is executed by an instruction from the CPU 201 shown in FIG. 2, and the short-distance communication circuit 219 and the antenna 219a, and performs acquisition and provision of data by short-distance communication with a proprietary terminal such as an IC card or a smartphone.

[0061] The storage / reading processing unit 29 is executed by an instruction from the CPU 201 shown in FIG. 2 and the SSD 204, and performs processing for storing various data in the storage unit 2000 or reading out various data stored in the storage unit 2000. Further, in the storage unit 2000, image data and audio data received when communicating with other electronic blackboards or video conference terminals are overwritten and stored each time they are received. Among these, an image is displayed on the display 220 by the image data before overwriting, and audio is output from the speaker 2300 by the audio data before overwriting.

[0062] Note that the video conference terminal 3 and the car navigation device 4 have the same functions as the electronic blackboard 2, and thus the description thereof is omitted here.

[0063] <Functional Configuration of PC> The PC 5 includes a transmission / reception unit 51, a reception unit 52, a display control unit 54, a generation unit 56, an audio control unit 58, and a storage / reading processing unit 59. Each of these units is a function or means for operating realized by an instruction from the CPU 501 according to a program developed from the HD 504 onto the RAM 503 by any of the components shown in FIG. 5. Further, the PC 5 has a storage unit 5000 constructed by the HD 504 shown in FIG. 5.

[0064] (Functional Configurations of PC) Next, each component of the PC 5 will be described. The transmission / reception unit 51 is realized by instructions from the CPU 501 shown in FIG. 5 and the network I / F 509, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 10. The transmission / reception unit 51 is an example of reception means.

[0065] The reception unit 52 is mainly realized by instructions from the CPU 501 shown in FIG. 5, as well as the keyboard 511 and the mouse 512, and receives various inputs from the user. The reception unit 52 is an example of reception means.

[0066] The display control unit 54 is realized by instructions from the CPU 501 shown in FIG. 5, and causes an image to be displayed on the display 508, or accesses the shared support server 6 using a web browser to display various screen data. Specifically, the display control unit 54 accesses the shared support server 6 or the schedule management server 8, for example, by starting and executing the minutes application 5502a or the browser 5502b that operates on the OS 5501 shown in FIG. 7. Then, the display control unit 54 downloads a WebAPP (Web Application) that includes at least HTML (HyperText Markup Language) and includes CSS (Cascading Style Sheets) or JAVA SCRIPT (registered trademark), etc., and causes various image data generated by the WebAPP to be displayed on the display 508. The display control unit 54 causes image data generated by HTML5 that includes data in a format such as XML (Extensible Markup Language), JSON (JavaScript Object Notation), or SOAP (Simple Object Access Protocol) to be displayed on the display 508. The display control unit 54 is an example of display control means.

[0067] The generation unit 56 is realized by an instruction from the CPU 501 shown in FIG. 5, and is a function for generating various image data to be displayed on the display 508. The generation unit 56 generates various image data using the content data received by the transmission / reception unit 51. For example, the generation unit 56 renders text data which is content data, and generates image data related to the text data which is content image data in order to display the rendered data. Rendering is a process of interpreting data described in a language for describing web pages (such as HTML, CSS, or XML), and calculating the arrangement of characters, image data, etc. actually displayed on the screen. The generation unit 56 is an example of generation means.

[0068] The sound control unit 58 is realized by an instruction from the CPU 501 shown in FIG. 5, and is a function for outputting a sound signal from the speaker 515. The sound control unit 58 sets the sound data to be output from the speaker 515, and reproduces the sound data by outputting the sound signal related to the set sound data from the speaker 515. The sound control unit 58 is an example of reproduction means.

[0069] The storage / reading processing unit 59 is executed by an instruction from the CPU 501 shown in FIG. 5 and by the HDD controller 505, etc., and performs processing for storing various data in the storage unit 5000 and reading various data from the storage unit 5000.

[0070] <Functional configuration of the shared support server> The shared support server 6 has a transmission / reception unit 61, an authentication unit 62, a creation unit 63, a generation unit 64, a determination unit 65, and a storage / reading processing unit 69. Each of these units is a function or means realized by operating according to an instruction from the CPU 601 in accordance with a shared support program developed from the HD 604 onto the RAM 603 by any of the components shown in FIG. 5. Further, the shared support server 6 has a storage unit 6000 constructed by the HD 604, etc. shown in FIG. 5.

[0071] (User authentication management table) FIG. 9(A) is a conceptual diagram showing a user authentication management table. In the storage unit 6000, a user authentication management DB6001 configured by a user authentication management table as shown in FIG. 9(A) is constructed. In this user authentication management table, a user ID for identifying a user, a user name, an organization ID for identifying the organization to which the user belongs, and a password are managed in association with each other. Note that the organization ID includes a domain name representing a group or an organization for managing a plurality of computers on a communication network.

[0072] (Access Management Table) FIG. 9(B) is a conceptual diagram showing an access management table. In the storage unit 6000, an access management DB6002 configured by an access management table as shown in FIG. 9(B) is constructed. In this access management table, an organization ID, an access ID required for authentication when accessing the schedule management server 8, and an access password are managed in association with each other. This access ID and access password are necessary for the shared support server 6 to use a service (function) provided by the schedule management server 8 via a Web API or the like using a protocol of HTTP (Hypertext Transfer Protocol) or HTTPS (Hypertext Transfer Protocol Secure). Since the schedule management server 8 manages a plurality of schedulers and the schedulers used may be different if the organizations are different, management by an access management table is necessary.

[0073] (Schedule Management Table) FIG. 9(C) is a conceptual diagram showing a schedule management table. In the storage unit 6000, a schedule management DB 6003 configured by a schedule management table as shown in FIG. 9(C) is constructed. In this schedule management table, for each schedule event ID and execution event ID, the organization ID, the user ID of the reserving person, the presence or absence of the participation of this reserving person, the name of the reserving person, the scheduled start time, the scheduled end time, the event name, the user ID of other participants, the presence or absence of the participation of these other participants, the names of other participants, and file data are associated and managed.

[0074] Among these, the schedule event ID is identification information for identifying a reserved event. The schedule event ID is an example of schedule event identification information for identifying an event scheduled for execution. The execution event ID is identification information for identifying an event that has been actually executed or is being executed among the reserved events. The execution event ID is an example of execution event identification information for identifying an event that has been executed or is being executed. The name of the reserving person is the name of the person who reserved the shared resource. When the shared resource is a meeting room, it is, for example, the name of the organizer. When the shared resource is a vehicle, it is, for example, the name of the driver. The scheduled start time indicates the scheduled start time of using the shared resource. The scheduled end time indicates the scheduled end time of using the shared resource. The event name indicates the name of the event that the reserving person has scheduled for execution. The user ID of other participants is identification information for identifying participants other than the reserving person. The names of other participants are the names of participants other than the reserving person and include the shared resource. That is, in this case, the users include the reserving person, other participants, and the shared resource. The file data is the file data of the material file used in the event corresponding to the schedule event ID, registered by user A using the schedule input screen 550 (see FIG. 19) described later. Also, the file data is data in a predetermined file format created using various applications. The file format of the file data is, for example, a PowerPoint file, an Excel file, etc.

[0075] (Execution Event Management Table) Figure 10(A) is a conceptual diagram showing an execution event management table. In the storage unit 6000, an execution event management DB6004 is constructed, which is composed of an execution event management table as shown in Figure 10(A). In this execution event management table, for each project ID, an execution event ID is associated and managed. The project ID is identification information (an example of project identification information) for identifying a project. A project refers to a group, team, or collective composed of a plurality of members to achieve a specific goal, plan, or project, etc. Members belonging to the same project can share the execution event history such as the minutes of events associated with the project ID. As shown in Figure 24 to be described later, the project ID is allocated for each project such as the next year's policy and customer development. Note that the project ID may also be referred to as a group ID or a team ID.

[0076] (Content Management Table) Figure 10(B) is a conceptual diagram showing a content management table. In the storage unit 6000, a content management DB6005 is constructed, which is composed of a content management table as shown in Figure 10(B). In this content management table, for each execution event ID, a content processing ID, the type of content processing, the content of the content, as well as the start date and time and end date and time of the content processing are associated and managed. Here, the content is the content of an execution event that occurred in an event such as a meeting in a certain project, or materials used in the event. The types of content processing include recording, snapshot, voice text conversion, generation of action items, and material sending, etc. Also, the content processing ID is identification information for identifying the processing of the content generated in each event.

[0077] Here, the content includes history information indicating the execution details of an event and action items generated by the executed event. The history information is data such as recording data, snapshots, voice text, or materials. A snapshot is a process of acquiring the display screen at a certain point in time during an ongoing event as image data. Note that a snapshot is also referred to as, for example, capture or image recognition.

[0078] When the type of content processing is "recording", the content of the content includes a URL indicating the storage destination of the recorded audio data. Also, when the type of content processing is "snapshot", the content of the content includes a URL indicating the storage destination of the image data of the screen acquired by the snapshot (capture). Capture means saving the image (still image, moving image) displayed on the display 220 as image data. Furthermore, when the type of content processing is "voice text conversion", the content of the content includes a URL indicating the storage destination of the text data of the received voice text.

[0079] Here, the action item occurs in an event such as a meeting in a certain project and indicates the content of the action that the person related to the event should take. When the type of content processing is "action item generation", the content of the content includes the user ID of the executor of the action item, the deadline for completing the action item, and a URL indicating the storage destination of the image data indicating the action item.

[0080] (Functional Components of the Shared Support Server) Next, each functional component of the shared support server 6 will be described in detail. In the following, when describing each functional component of the shared support server 6, the relationship with the main components for realizing each functional component of the shared support server 6 among the components shown in FIG. 5 will also be described.

[0081] The transmission / reception unit 61 of the shared support server 6 shown in FIG. 8 is realized by the instruction from the CPU 601 shown in FIG. 5 and the network I / F 609 shown in FIG. 5, and transmits and receives various data (or information) with other terminals, devices or systems via the communication network 10.

[0082] The authentication unit 62 is realized by the instruction from the CPU 601 shown in FIG. 5, and performs authentication by determining whether the information (user ID, organization ID and password) sent from the communication terminal is the information registered in advance in the user authentication management DB 6001.

[0083] The creation unit 63 is realized by the instruction from the CPU 601 shown in FIG. 5, and creates a reservation list screen 230 as shown in FIG. 22 described later based on the reservation information and schedule information sent from the schedule management server 8.

[0084] The generation unit 64 is realized by the instruction from the CPU 601 shown in FIG. 5, and generates an execution event ID, a content processing ID and a storage destination URL.

[0085] The determination unit 65 is realized by the instruction from the CPU 601 shown in FIG. 5, and makes various determinations. This determination will be described later.

[0086] The storage / reading processing unit 69 is realized by the instruction from the CPU 601 shown in FIG. 5 and the HDD controller 605 shown in FIG. 5, and performs processing for storing various data in the storage unit 6000 or reading out various data stored in the storage unit 6000.

[0087] <Functional configuration of the schedule management server> The schedule management server 8 has a transmission / reception unit 81, an authentication unit 82, a generation unit 83, and a storage / reading processing unit 89. Each of these units is a function or means realized by any of the components shown in FIG. 5 operating according to an instruction from the CPU 801 according to the schedule management program expanded from the HD804 onto the RAM803. Further, the schedule management server 8 has a storage unit 8000 constructed by the HD804 shown in FIG. 5.

[0088] (User authentication management table) FIG. 11(A) is a conceptual diagram showing a user authentication management table. In the storage unit 8000, a user authentication management DB8001 constituted by a user authentication management table as shown in FIG. 11(A) is constructed. In this user authentication management table, an organization ID and a password for identifying the organization to which the user belongs are associated with the user ID for identifying the user and managed.

[0089] (User management table) FIG. 11(B) is a conceptual diagram showing a user management table. In the storage unit 8000, a user management DB8002 constituted by a user management table as shown in FIG. 11(B) is constructed. In this user management table, for each organization ID, the user ID and the name of the user (user name) indicated by the user ID are associated and managed.

[0090] (Community management table) FIG. 11(C) is a conceptual diagram showing a community management table. In the storage unit 8000, a community management DB8003 constituted by a community management table as shown in FIG. 11(C) is constructed. In this community management table, for each organization ID, a community ID for identifying the community and the name of the community (community name) are associated and managed.

[0091] (Community reservation management table) FIG. 12(A) is a conceptual diagram showing a union reservation management table. In the storage unit 8000, a union reservation management DB8004 is constructed which is composed of a union reservation management table as shown in FIG. 12(A). In this union reservation management table, reservation information in a state where each piece of information is associated is managed. The reservation information includes, for each organization ID, a union ID, a union name, a user ID of the person making the reservation, a scheduled start date and time of use, a scheduled end date and time of use, and an event name. Among these, the scheduled start date and time of use indicates the scheduled start date and time of using the union. The scheduled end date and time of use indicates the scheduled end date and time of using the union. Each date and time indicates year·month·day·hour·minute·second·time zone, but in FIG. 12(A), due to space limitations on the page, only year·month·day·hour·minute are shown.

[0092] (Event management table) FIG. 12(B) is a conceptual diagram showing an event management table. In the storage unit 8000, an event management DB8005 is constructed which is composed of an event management table as shown in FIG. 12(B). In this event management table, schedule information in a state where each piece of information is associated is managed. The schedule information includes, for each scheduled event ID, an organization ID, a user ID, a user name, a scheduled start date and time of the event, a scheduled end date and time of the event, and an event name, which are associated and managed. Among these, the scheduled start date and time of the event indicates the scheduled start date and time when the event is to be executed. The scheduled end date and time of the event indicates the scheduled end date and time when the event is to be executed. Each date and time indicates year·month·day·hour·minute·second·time zone, but in FIG. 12(B), due to space limitations on the page, only year·month·day·hour·minute are shown. Also, in the event management table, file data of the material files used in the event indicated in the schedule information is managed in association with the scheduled event ID.

[0093] (Server authentication management table) Figure 13(A) is a conceptual diagram showing a server authentication management table. In the storage unit 8000, a server authentication management DB8006 composed of a server authentication management table as shown in Figure 13(A) is constructed. In this server authentication management table, an access ID and an access password are managed in association with each other. The access ID and the access password are the same concept as the access ID and the access password managed in the access management DB6002 of the shared support server 6.

[0094] (Project member management table) Figure 13(B) is a conceptual diagram showing a project member management table. In the storage unit 8000, a project member management DB8007 composed of a project member management table as shown in Figure 13(B) is constructed. In this project member management table, for each organization ID, a project ID, a project name, and the user ID of each project member are managed in association with each other.

[0095] (Execution event history management table) Figure 14(A) is a conceptual diagram showing an execution event history management table. In the storage unit 8000, an execution event history management DB8008 composed of an execution event history management table as shown in Figure 14(A) is constructed. In this execution event history management table, for each project ID and execution event ID, a content processing ID, a type of content processing, content, and a start date / time and an end date / time of the content processing are managed in association with each other. This execution event history management DB8008 manages some of the same data as the content management DB6005. The same data is the execution event ID, the content processing ID, the type of content processing, the start date / time and the end date / time of the content processing. The content only differs in the notation method (http: / / or c: / / ) of the storage destination of the content data, and the storage destination is the same.

[0096] (Execution event management table) FIG. 14(B) is a conceptual diagram showing an execution event management table. In the storage unit 8000, an execution event management DB 8009 configured by an execution event management table as shown in FIG. 14(B) is constructed. In this execution event management table, an event name, a start date / time, and an end date / time of an event are associated with and managed by an execution event ID. This execution event management DB 8009 manages information related to events that have actually been executed among the schedule information managed in the event management DB 8005.

[0097] (Related Information Management Table) FIG. 15 is a conceptual diagram showing a related information management table. In the storage unit 8000, a related information management DB 8010 configured by a related information management table as shown in FIG. 15 is constructed. In this related information management table, related information in a state where each piece of information (data) is associated is managed for each project ID and execution event ID. Content generation time zone, audio data, voice text data, and screen data are associated with and managed in the related information. The content generation time zone indicates the elapsed time from the start date / time of the event at the time when the content is generated in the executed event. Here, the content generation time zone is generated by the generation unit 83 based on the start date / time of the event stored in the event management DB 8005 and the start date / time and end date / time of the content process stored in the execution event history management DB 8008. The content generation time zone is an example of time information. Also, the audio data includes a content process ID and a content type. The voice text data and the screen data include a content process ID, a content type, and a sequence No. The sequence No indicates the order of occurrence times when the content process occurs for each of the voice text data and the screen data.

[0098] (Each Functional Configuration of the Schedule Management Server) Next, each functional configuration of the schedule management server 8 will be described in detail. In the following, when describing each functional configuration of the schedule management server 8, the relationship with the main components for realizing each functional configuration of the schedule management server 8 among the components shown in FIG. 5 will also be described.

[0099] The transmission / reception unit 81 of the schedule management server 8 shown in FIG. 8 is realized by commands from the CPU 801 shown in FIG. 5 and the network I / F 809 shown in FIG. 5, and transmits and receives various data (or information) to and from other terminals, devices, or systems via the communication network 10.

[0100] The authentication unit 82 is realized by commands from the CPU 801 shown in FIG. 5, and performs authentication by determining whether the information (user ID, organization ID, and password) sent from the union is the information registered in advance in the user authentication management DB 8001. Also, the authentication unit 82 performs authentication by determining whether the information (access ID and access password) sent from the shared support server 6 is the information registered in advance in the server authentication management DB 8006.

[0101] The generation unit 83 is realized by commands from the CPU 801 shown in FIG. 5, and is a function for generating related information to be registered in the related information management DB 8010.

[0102] The storage / reading processing unit 89 is realized by commands from the CPU 801 shown in FIG. 5 and the HDD controller 805 shown in FIG. 5, and performs processing for storing various data in the storage unit 8000 and reading out various data stored in the storage unit 8000.

[0103] <Functional Configuration of Voice-Text Conversion Server> The voice text conversion server 9 has a transmission / reception unit 91, a conversion unit 93, and a storage / reading processing unit 99. Each of these units is a function or means realized by any of the components shown in FIG. 5 operating according to instructions from the CPU 901 deployed on the RAM 903 from the HD 904. Further, the voice text conversion server 9 has a storage unit 9000 constructed by the HD 904 shown in FIG. 5.

[0104] (Functional configurations of the voice text conversion server) Next, the functional configurations of the voice text conversion server 9 will be described in detail. In the following, when explaining the functional configurations of the voice text conversion server 9, the relationship with the main components for realizing the functional configurations of the voice text conversion server 9 among the components shown in FIG. 5 will also be explained.

[0105] The transmission / reception unit 91 of the voice text conversion server 9 shown in FIG. 8 is realized by instructions from the CPU 901 shown in FIG. 5 and the network I / F 909 shown in FIG. 5, and transmits and receives various data (or information) with other terminals, devices, or systems via the communication network 10.

[0106] The conversion unit 93 is realized by instructions from the CPU 901 shown in FIG. 5, and converts the audio data received via the communication network 10 by the transmission / reception unit 91 into text data.

[0107] The storage / reading processing unit 99 is realized by instructions from the CPU 901 shown in FIG. 5 and the HDD controller 905 shown in FIG. 5, and performs processing to store various data in the storage unit 9000 or read out various data stored in the storage unit 9000.

[0108] Note that the above each ID is an example of identification information. Also, the organization ID includes company name, office name, department name, regional name, etc. The user ID includes employee number, driver's license number, my number in the Japanese social security and tax number system, etc.

[0109] <<Processing or operation of the embodiment>> Hereinafter, the processing or operation of each embodiment will be described.

[0110] <Schedule registration processing> Here, with reference to FIGS. 16 to 19, the process in which the reserving person A (Taro Ricoh) registers his / her schedule from the PC 5 to the schedule management server 8 will be described. FIG. 16 is a sequence diagram showing the schedule registration process. FIG. 17 is a diagram showing the sign-in screen. FIG. 18 is an example of the initial screen of the PC. FIG. 19 is a diagram showing the schedule input screen.

[0111] First, by operating the keyboard 511 etc. of the PC 5, the display control unit 54 of the PC 5 causes a sign-in screen 530 for signing in to be displayed on the display 508 as shown in FIG. 17 (step S11). This sign-in screen 530 includes an input field 531 for entering the user ID and organization ID of the user, an input field 532 for entering the password, a sign-in button 538 to be pressed when signing in, and a cancel button 539 to be pressed when canceling the sign-in. Here, the user ID and organization ID are the e-mail address of the user A. The user name part of the e-mail address is the user ID, and the domain name part indicates the organization ID. Note that the input field 531 may be an input field for separately entering the user ID and organization ID instead of the e-mail address.

[0112] Next, when user A enters their user ID and organization ID in input field 531, enters their password in input field 532, and presses the sign-in button 538, the reception unit 52 receives the sign-in request (step S12). Then, the transmission / reception unit 51 of the PC 5 transmits sign-in request information indicating the sign-in request to the schedule management server 8 (step S13). This sign-in request information includes the information (user ID, organization ID, and password) received in step S12. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the sign-in request information.

[0113] Next, the authentication unit 82 of the schedule management server 8 authenticates user A using the user ID, organization ID, and password (step S14). Specifically, the storage / reading processing unit 89 searches the user authentication management DB 8001 (see Fig. 11(A)) for the set of user ID, organization ID, and password corresponding to the set of user ID, organization ID, and password received in step S13. If there is a corresponding set, the authentication unit 82 determines that the requesting user A is a legitimate user. If there is no corresponding set, the authentication unit 82 determines that user A is an improper (not legitimate) user. If not legitimate, the transmission / reception unit 81 notifies the PC 5 that it is not legitimate, but here, the description continues for the case where it is legitimate.

[0114] Next, the transmission / reception unit 81 transmits the authentication result to the PC 5 (step S15). As a result, the transmission / reception unit 51 of the PC 5 receives the authentication result.

[0115] Next, when the generation unit 56 of the PC 5 receives the authentication result indicating validity in step 15, it generates an initial screen 540 as shown in FIG. 18 (step S16). Then, the display control unit 54 of the PC 5 causes the initial screen 540 as shown in FIG. 18 to be displayed on the display 508 (step S17). This initial screen 540 includes a "Schedule Registration" button 541 to be pressed when registering a schedule and an "Execution Event History Viewing" button 543 to be pressed when viewing the execution event history. Here, when the user presses the "Schedule Registration" button 541, the reception unit 52 accepts schedule registration (step S18). Then, the transmission / reception unit 51 transmits a schedule registration request to the schedule management server 8 (step S19). As a result, the transmission / reception unit 81 of the schedule management server 8 receives the schedule registration request.

[0116] Next, the storage / reading processing unit 89 of the schedule management server 8 searches the user management DB 8002 (see FIG. 11(B)) using the organization ID received in step S13 as a search key, and reads out all corresponding user IDs and all user names (step S20). Then, the transmission / reception unit 81 transmits schedule input screen information to the PC 5 (step S21). This schedule input screen information includes all the user IDs and all the user names read out in step S20. All the user names also include the name of the user A who is the reserving person and entered the input for sign-in in step S12. As a result, the transmission / reception unit 51 of the PC 5 receives the schedule input screen information.

[0117] Next, on the PC 5, the generation unit 56 uses the schedule input screen information received in step S21 to generate a schedule input screen 550 (step S22). Then, the display control unit 24 of the PC 5 causes the schedule input screen 550 as shown in FIG. 19 to be displayed on the display 508 (step S23).

[0118] This schedule input screen 550 includes an input field 551 for entering an event name, an input field 552 for entering a shared body ID or shared body name, an input field 553 for entering the scheduled start date and time of event execution (use of the shared body), an input field 554 for entering the scheduled end date and time of event execution (use of the shared body), an input field 555 for entering a memo such as an agenda, a display area 556 for displaying the name of the reserving person, a selection menu 557 for selecting the names of participants other than the reserving person, an "OK" button 558 to be pressed when registering a reservation, and a "CANCEL" button 559 to be pressed when canceling the input or the entered content. The name of the reserving person is the name of the user who entered the sign-in input on the PC 5 in step S12. Also, the mouse pointer p1 is also displayed.

[0119] Note that an email address may be entered in the input field 552. Also, when a shared body name is selected in the selection menu 557, the shared body is also added as another participant.

[0120] Next, when user A enters predetermined items in the input fields 551 to 555, selects the name (user name) of the user who wants to participate in the event from the selection menu 557 using the mouse pointer p1, and presses the "OK" button 558, the reception unit 52 receives the input of the schedule information (step S24). Then, the transmission / reception unit 51 transmits the schedule information to the schedule management server 8 (step S25). This schedule information includes an event name, a shared body ID (or shared body name), a scheduled start date and time, a scheduled end date and time, the user ID of each participant, and a memo. Note that in the schedule input screen 550, when a shared body ID is entered in the input field 552, this shared body ID is transmitted, and when a shared body name is entered in the input field 552, this shared body name is transmitted. Also, in the schedule input screen 550, although a user name is selected in the selection menu 557, since the user ID is also received in step S21, the user ID corresponding to the user name is transmitted. Thereby, the transmission / reception unit 81 of the schedule management server 8 receives the schedule information.

[0121] Next, the schedule management server 8 searches the union management DB 8003 (see FIG. 11(C)) using the union ID (or union name) received in step S25 as a search key, and reads out the corresponding union name (or union ID) (step S26).

[0122] Next, the storage / reading processing unit 89 stores reservation information in the union reservation management DB 8004 (see FIG. 12(A)) (step S27). In this case, the storage / reading processing unit 89 adds one record's worth of reservation information to the union reservation management table of the union reservation management DB 8004 managed by a previously registered scheduler. The reservation information is configured based on the schedule information received in step S25 and the union name (or union ID) read out in step S26. Note that the scheduled start date / time in the union reservation management DB 8004 corresponds to the scheduled start date / time in the schedule information. Also, the scheduled end date / time in the union reservation management DB 8004 corresponds to the scheduled end date / time in the schedule information.

[0123] Furthermore, the storage / reading processing unit 89 stores scheduled information in the event management DB 8005 (see FIG. 12(B)) (step S28). In this case, the storage / reading processing unit 89 adds one record's worth of scheduled information to the event management table of the event management DB 8005 managed by a previously registered scheduler. The scheduled information is configured based on the schedule information received in step S25. Note that the scheduled event start date / time in the event management DB 8005 corresponds to the scheduled start date / time in the schedule information. Also, the scheduled event end date / time in the event management DB 8005 corresponds to the scheduled end date / time in the schedule information.

[0124] As described above, user A can register his / her schedule with the schedule management server 8. Although FIGS. 16 to 19 described the case where schedule registration is performed by the PC 5, schedule registration may be performed by the electronic blackboard 2, the video conference terminal 3, or the car navigation device 4 by the same process.

[0125] <Event start process> Here, with reference to FIGS. 20 to 26, the process of user A (Taro Ricoh) having a meeting with other participants using the electronic blackboard 2 in the reserved meeting room X will be described. FIGS. 20 and 23 are sequence diagrams showing the start process of the event. FIG. 21 is a diagram showing the sign-in screen displayed on the electronic blackboard. FIG. 22 is a diagram showing the reservation list screen of the shared body. FIG. 24 is a diagram showing the project list screen. FIG. 25 is a diagram showing the detailed information screen of the event. FIG. 26 is an image diagram showing the state where the user uses the electronic blackboard.

[0126] First, when user A presses the power switch 222 of the electronic blackboard 2, the reception unit 22 of the electronic blackboard 2 receives the power-on from the user (step S31). When the reception unit 22 receives the power-on, the Launcher 102 shown in FIG. 6 starts up. Then, as shown in FIG. 21, the display control unit 24 of the electronic blackboard 2 displays a sign-in screen 110 for signing in on the display 220 (step S32). This sign-in screen 110 includes a selection icon 111 to be pressed when user A signs in using his / her IC card, a selection icon 113 to be pressed when user A signs in by entering his / her email address (user ID and organization ID) and password, and a power icon 115 to be pressed when turning off the power without signing in.

[0127] Next, user A presses the selection icon 111 to communicate with the short-distance communication circuit 219 such as an IC card reader, or user A presses the selection icon 113 and inputs their own email address and password, whereupon the reception unit 22 receives a sign-in request (step S33). Then, the transmission / reception unit 21 transmits sign-in request information indicating a sign-in request to the shared support server 6 (step S34). This sign-in request information includes the information received in step S33 (user ID, organization ID, and password), the time zone information of the country or region where the electronic blackboard 2 is installed, and the user ID, organization ID, and password of the communication terminal (here, the electronic blackboard 2). As a result, the transmission / reception unit 61 of the shared support server 6 receives the sign-in request information.

[0128] Next, the authentication unit 62 of the shared support server 6 authenticates user A using the user ID, organization ID, and password of user A received in step S34 (step S35). Specifically, the storage / read processing unit 69 uses the user ID, organization ID, and password of user A received in step S34 as search keys to search the user authentication management DB6001 (see Fig. 9(A)) for the set of user ID, organization ID, and password corresponding to the received set of user ID, organization ID, and password of user A. If there is a corresponding set, the authentication unit 62 determines that the requesting user A is a legitimate user. If there is no corresponding set, the authentication unit 62 determines that the requesting user A is an improper (illegitimate) user. In the case of illegitimacy, the transmission / reception unit 61 notifies the electronic blackboard 2 that it is not legitimate, but here, the explanation continues for the case of legitimacy.

[0129] Next, the storage / read processing unit 69 of the shared support server 6 searches the access management DB6002 (see Fig. 9(B)) using the organization ID of user A received in step S34 as a search key to read out the corresponding access ID and access password (step S36).

[0130] Next, the transmission / reception unit 61 transmits reservation request information indicating a request for reservation information of the shared body and schedule request information indicating a request for the user's schedule information to the schedule management server 8 (step S37). The reservation request information and the schedule request information include the time zone information, the user ID and organization ID of the communication terminal received in step S34, and the access ID and password read in step S36. Accordingly, the transmission / reception unit 81 of the schedule management server 8 receives the reservation request information and the schedule request information.

[0131] Next, the authentication unit 82 of the schedule management server 8 authenticates the shared support server 6 using the access ID and the access password (step S38). Specifically, the storage / read processing unit 89 searches for a set of an access ID and an access password corresponding to the set of the access ID and the access password received in step S37 in the server authentication management DB8006 (see FIG. 13(A)). If there is a corresponding set, the authentication unit 82 determines that the source shared support server 6 is a legitimate accessor. If there is no corresponding set, the authentication unit 82 determines that the source shared support server 6 is an improper (illegitimate) accessor. In the case of being illegitimate, the transmission / reception unit 81 notifies the shared support server 6 that it is not legitimate. Here, the case of being legitimate will be continued with the explanation.

[0132] The storage / read processing unit 89 of the schedule management server 8 reads out the corresponding reservation information by searching the shared body reservation management DB8004 (see FIG. 12(A)) managed by the scheduler using the user ID of the communication terminal received in step S37 as a search key (step S39). In this case, the storage / read processing unit 89 reads out the reservation information whose scheduled start date and time is today.

[0133] Furthermore, the memory / reading processing unit 89 searches the event management DB 8005 (see FIG. 12(B)) managed by the scheduler using the user ID of the communication terminal received in step S37 as a search key, and reads out the corresponding schedule information (step S40). In this case, the memory / reading processing unit 89 reads out the schedule information whose event start scheduled date and time is today. When the schedule management server 8 is in a different country or region from the communication terminal such as the electronic blackboard 2, the time zone is adjusted according to the country or region where the communication terminal is installed based on the time zone information.

[0134] Next, the memory / reading processing unit 89 searches the project member management DB 8007 (see FIG. 13(B)) using the user ID of the communication terminal received in step S37 as a search key, and reads out the project IDs and project names of all projects including the user ID of the communication terminal (step S41).

[0135] Next, the transmission / reception unit 81 transmits the reservation information read in step S39, the schedule information read in step S40, and all the project IDs and all the project names read in step S41 to the shared support server 6 (step S42). As a result, the transmission / reception unit 61 of the shared support server 6 receives the reservation information, the schedule information, all the project IDs, and all the project names.

[0136] Next, the creation unit 63 of the shared support server 6 creates a reservation list based on the reservation information and the schedule information received in step S42 (step S43). Then, the transmission / reception unit 61 transmits the reservation list information indicating the content of the reservation list and all the project IDs and all the project names to the electronic blackboard 2 (step S44). As a result, the transmission / reception unit 21 of the electronic blackboard 2 receives the reservation list information and all the project IDs and all the project names.

[0137] Next, on the electronic blackboard 2, the display control unit 24 causes the reservation list screen 230 as shown in FIG. 22 to be displayed on the display 220 (step S45). This reservation list screen 230 includes a display area 231 for displaying the name of the shared entity (here, the name of the location) and a display area 232 for displaying the date and time of today. Further, on the reservation list screen 230, event information 235, 236, 237, etc. indicating events using the shared entity of today (here, Conference Room X) is displayed. Each piece of event information includes, for each event, the scheduled start time and scheduled end time of using the shared entity, the event name, and the name of the person who reserved the shared entity (reservation name). The event information includes start buttons 235s, 236s, 237s, etc. that are pressed when the user wants to identify the event to start.

[0138] Next, in FIG. 23, when user A presses the start button 235s with, for example, an electronic pen 2500, etc., the reception unit 22 receives the selection of the event indicated by the event information 235 (step S51). Then, based on the project ID and project name received in step S42, the display control unit 24 causes a project list screen 240 as shown in FIG. 24 to be displayed on the display 220 (step S52). The project list screen 240 includes project icons 241 to 246 indicating each project. Further, the project list screen 240 includes an "OK" button 248 that is pressed when the selected project icon is confirmed and a "CANCEL" button 249 that is pressed when the selection of the project icon is canceled.

[0139] Next, in FIG. 24, when user A presses the project icon 241 with, for example, an electronic pen 2500, etc., the reception unit 22 receives the selection of the project indicated by the project icon 241 (step S53).

[0140] Next, the transmission / reception unit 21 of the electronic blackboard 2 transmits to the shared support server 6 a scheduled event ID indicating the scheduled event selected in step S51 and a project ID indicating the project selected in step S53 (step S54). The process of step S54 is a transmission request process for execution event identification information. As a result, the transmission / reception unit 61 of the shared support server 6 receives the selected scheduled event ID and the selected project ID.

[0141] Next, in the shared support server 6, the generation unit 64 generates a unique execution event ID (step S55). Then, the storage / reading processing unit 69 manages by associating the execution event ID generated in step S55, the reservation event ID received in step S54, the user ID and organization ID of the reserving person, and the event information (step S56). Note that the user ID and organization ID of the reserving person, and the event information are IDs and information based on the reservation information and scheduled information received in step S42. Note that at this point, there is no input to the participation presence / absence column in the reservation management table (see FIG. 9(C)).

[0142] Next, the storage / reading processing unit 69 stores and manages by associating the project ID received in step S54 and the execution event ID generated in step S55 (step S57).

[0143] Next, in the shared support server 6, the transmission / reception unit 61 transmits to the schedule management server 8 file data transmission request information indicating a transmission request for file data registered in the schedule management server 8 (step S58). The file data transmission request information includes the scheduled event ID received in step S54, the user ID and organization ID of the communication terminal received in step S34, and the access ID and access password read in step S36. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the file data transmission request information.

[0144] Next, the storage / reading processing unit 89 of the schedule management server 8 searches the event management DB 8005 (see FIG. 12(B)) using the scheduled event ID received in step S58 as a search key, and reads out the file data associated with the scheduled event ID (step S59). Then, the transmission / reception unit 81 transmits the file data read in step S59 to the shared support server 6 (step S60). As a result, the transmission / reception unit 61 of the shared support server 6 receives the file data.

[0145] Next, the storage / reading processing unit 69 of the shared support server 6 stores and manages the file data received in step S60 in association with the scheduled event ID received in step S54 and the executed event ID generated in step S55 in the schedule management DB 6003 (see FIG. 9(C)) (step S61).

[0146] Then, the transmission / reception unit 61 transmits the executed event ID generated in step S55 and the file data received in step S60 to the electronic blackboard 2 (step S62). As a result, the transmission / reception unit 21 of the electronic blackboard 2 receives the executed event ID and the file data.

[0147] Next, in the electronic blackboard 2, the memory / readout processing unit 29 stores the execution event ID and the file data in the memory unit 2000 (step S63). Here, the file data transmitted from the shared support server 6 is stored in a specific storage area of the memory unit 2000. Thereby, during the execution of the event, the electronic blackboard 2 accesses the specific storage area, and the display control unit 24 causes the file data stored in the specific storage area to be displayed on the display 220. Here, the specific storage area is a temporary storage destination of data provided for each event being executed, and is a storage area specified by an arbitrary path (character string) indicating the location within the memory unit 2000. Note that the specific storage area is not limited to a configuration provided inside the electronic blackboard 2, and may be a configuration provided in an external storage device connected to the electronic blackboard 2, or a local server or the like that exists in an on-premises environment and is communicable with the electronic blackboard 2.

[0148] Next, as shown in FIG. 25, the display control unit 24 causes the detailed information screen 250 of the selected event to be displayed on the display 220 (step S64). The detailed information screen 250 of this event includes a display area 251 for the event name, a display area 252 for the scheduled execution time of the event (scheduled start time and scheduled end time), and a display area 253 for the name of the reserving person. Further, the detailed information screen 250 of the event includes a display area 256 for displaying the content of the memo, a display area 257 for displaying the names of the scheduled participants, and a display area 258 for displaying identification information (e.g., file name) for identifying the file data stored in a specific storage area of the storage unit 2000. In the display area 257, the name of the reserving person shown in FIG. 19 and the names of the other selected participants are displayed, and a check box for checking the person actually participating in the meeting is displayed for each name of the scheduled participants. Also, in the display area 258, the file name of the file data stored in a specific storage area of the storage unit 2000, that is, the file data downloaded from the shared support server 6, is displayed, and the file name of the file data being downloaded from the shared support server 6 is also displayed. Further, on the lower right side of the detailed information screen 250 of the event, a "Close" button 259 to be pressed when closing this detailed information screen 250 is included.

[0149] Next, when the user checks the check box of the user actually participating among the names of the scheduled participants and presses the "Close" button 259, the reception unit 22 receives the selection of the participants (step S65). Then, the transmission / reception unit 21 transmits information indicating the user ID and the presence or absence of participation of each scheduled participant to the shared support server 6 (step S66). As a result, the transmission / reception unit 61 of the shared support server 6 receives information indicating the user ID and the presence or absence of participation of each scheduled participant.

[0150] Next, in the shared support server 6, in the schedule management DB 6003, the presence or absence of participation is stored and managed in the column for the presence or absence of participation that was not input (step S67).

[0151] As described above, user A starts an event (here, a policy decision meeting) using a shared entity (here, conference room X) and a communication terminal (here, electronic blackboard 2). Then, as shown in FIG. 26, user A can conduct a meeting in conference room X using electronic blackboard 2. The display control unit 24 causes the display 220 to display an event execution screen R as shown in FIG. 26. The display control unit 24 displays the remaining time available for using this shared entity in the upper right region of the event execution screen R. In this case, the display control unit 24 displays the time (remaining time) from the current time until the scheduled end time indicated by the event information selected in step S51.

[0152] Also, the display control unit 24 displays an icon r1 to be pressed when registering an action item, an icon r2 to be pressed when browsing the execution event history, an icon r3 to be pressed when browsing a document file stored in a specific storage area of the storage unit 2000, and the file data r4 of the document file on the event execution screen R. The icon r3 is an example of a selection image that accepts a selection for displaying the file data stored in a specific storage area. For example, the user of the electronic blackboard 2 presses the icon r3, the reception unit 22 accepts the selection of the icon r3, and the display control unit 24 causes the file data r4 of the document file stored in a specific storage area of the storage unit 2000 to be displayed. Note that the display control unit 24 of the electronic blackboard 2 may display not only the file data received in step S62 but also the file data pre-stored in the storage unit 2000 or the file data newly generated in the ongoing event. In this case, the storage / reading processing unit 29 of the electronic blackboard 2 stores the file data generated or updated in the ongoing event in a specific storage area of the storage unit 2000.

[0153] <Registration Process of Execution Event History> Next, the registration process of the execution event history will be described with reference to FIGS. 27 to 33. FIGS. 27 and 29 are sequence diagrams showing the registration process of the execution event history. FIG. 28A is a flowchart showing the voice text conversion process.

[0154] First, the determination unit 25 of the electronic blackboard 2 determines the type of content processing in the started event (step S71). Specifically, when the content is audio data generated by recording by the image / audio processing unit 23, the determination unit 25 determines that the type of content processing is "recording". When the content is image data acquired by a snapshot (capture) by the image / audio processing unit 23, the determination unit 25 determines that the type of content processing is "snapshot". When the content is material file data transmitted by the transmission / reception unit 21, the determination unit 25 determines that the type of content processing is "material transmission".

[0155] Next, the transmission / reception unit 21 transmits registration request information indicating a registration request for the generated content to the shared support server 6 (step S72). In this case, the transmission / reception unit 21 automatically transmits the registration request information every time the content is generated. This registration request information includes the execution event ID, the user ID of the content transmission source, the content data, and the content processing type information. As a result, the transmission / reception unit 61 of the shared support server 6 receives the registration request information.

[0156] Next, the determination unit 65 of the shared support server 6 determines the type of received content processing based on the type information included in the registration request information received by the transmission / reception unit 61 (step S73). Then, when the determination unit 65 determines that the type of content processing is "recording", the transmission / reception unit 61 of the shared support server 6 transmits the audio data, which is the content data, to the voice text conversion server 9 (step S74). Thereby, the transmission / reception unit 91 of the voice text conversion server 9 receives the audio data. When the type of content processing is other than "recording", the shared support server 6 skips the processes of steps S74 to S76 and proceeds to the process of step S77.

[0157] Next, the conversion unit 93 of the voice text conversion server 9 converts the audio data received by the transmission / reception unit 91 into text data (step S75). Here, the voice text conversion process in the voice text conversion server 9 will be described with reference to FIG. 28A. First, the conversion unit 93 acquires information indicating the date and time when the audio data was received by the transmission / reception unit 91 (step S75-1). Note that the information acquired in step S75-1 may be information indicating the date and time when the shared support server 6 received the audio data or the date and time when the shared support server 6 transmitted the audio data. In this case, the transmission / reception unit 91 of the voice text conversion server 9 receives the audio data transmitted from the shared support server 6 and the information indicating the above date and time in step S74.

[0158] Next, the conversion unit 93 executes text conversion processing on the audio data received by the transmission / reception unit 91 (step S75-2). Then, when the text conversion processing of the audio data is completed (YES in step S75-3), the conversion unit 93 transfers the process to step S75-4. On the other hand, the conversion unit 93 repeats the process of step S75-2 until the text conversion processing of the audio data is completed. Here, in step S75-3, when a predetermined amount of text conversion is performed based on the audio data received by the transmission / reception unit 91, the conversion unit 93 determines that the text conversion processing of the audio data is completed. For example, when the text conversion for one sentence is performed, the conversion unit 93 determines that the text conversion processing of the audio data is completed. Then, the conversion unit 93 generates text data converted from the audio data (step S75-4). Thereby, the voice text conversion server 9 converts the audio data transmitted from the shared support server 6 into text data. Since the voice text conversion server 9 receives the audio data transmitted from the shared support server 6 at any time, the process shown in FIG. 28A is repeatedly executed.

[0159] Here, with reference to FIG. 28B, the steps S74 to S76 in FIG. 27 and the voice text conversion process described in FIG. 28A will be described in more detail. FIG. 28B is a sequence diagram showing the voice text conversion process.

[0160] As shown in step S74 of FIG. 27, the transmission / reception unit 21 of the electronic blackboard 2 transmits the audio data generated by the event being executed to the shared support server 6 at any time (steps Sa1, Sa2). Thereby, the transmission / reception unit 61 of the shared support server 6 receives the audio data transmitted from the electronic blackboard 2 at any time. Then, the transmission / reception unit 61 of the shared support server 6 transmits the audio data transmitted from the electronic blackboard 2 to the voice text conversion server 9 at any time (steps Sb1, Sb2). Thereby, the transmission / reception unit 91 of the voice text conversion server 9 receives the audio data transmitted from the shared support server 6 at any time.

[0161] Here, the sound data transmitted from the electronic blackboard 2 to the shared support server 6 in step Sa1 corresponds to the sound data transmitted from the shared support server 6 to the speech text conversion server 9 in step Sb1, and the sound data transmitted from the electronic blackboard 2 to the shared support server 6 in step Sa2 corresponds to the sound data transmitted from the shared support server 6 to the schedule management server 8 in step Sb2. In this way, the sound data collected and processed by the image / sound processing unit 23 of the electronic blackboard 2 is transmitted to the speech text conversion server 9 as needed.

[0162] The conversion unit 93 of the speech text conversion server 9 converts the sound data received by the transmission / reception unit 91 into text data. Then, the transmission / reception unit 91 of the speech text conversion server 9 transmits the text data converted by the conversion unit 93 to the shared support server 6 (step Sc1). In this case, the text data transmitted in step Sc1 is the text data obtained by text-converting the sound data received in steps Sb1 and Sb2.

[0163] Here, the text data transmitted from the speech text conversion server 9 to the shared support server 6 in step Sc1 is undetermined text data. When a predetermined amount of text conversion is performed by the conversion unit 93 of the speech text conversion server 9, it determines that the text conversion of the sound data is completed and the text data is determined. For example, when the conversion unit 93 performs text conversion for one sentence, it determines that the text conversion of the sound data is completed and the text data is determined. In step Sc1, since it is undetermined text data, the transmission / reception unit 91 transmits an undetermined flag indicating that the text data is not determined (undetermined) together with the text data ("In this fiscal year, more") converted by the conversion unit 93.

[0164] Then, the transceiver unit 61 of the shared support server 6 transmits the text data (uncertain text data) transmitted from the voice text conversion server 9 to the electronic blackboard 2 (step Sc2). As a result, the transceiver unit 21 of the electronic blackboard 2 receives the text data (uncertain text data) transmitted from the shared support server 6. Then, the display control unit 24 of the electronic blackboard 2 causes the text data received by the transceiver unit 21 to be displayed on the display 508. The electronic blackboard 2 can display the text data as the subtitle display of the sound data generated by the event being executed on the event execution screen R as shown in FIG. 26. Note that even if the text data is uncertain, the electronic blackboard 2 can receive and display it, allowing the user of the electronic blackboard 2 to confirm that the voice text conversion process by the voice text conversion server 9 is being performed at any time.

[0165] The shared system 1 repeats the processes as described above (steps Sa3~Sa7, Sb3~Sb7, Sc3~Sc4), thereby repeatedly executing the text conversion process of the sound data collected and processed by the image / sound processing unit 23 of the electronic blackboard 2. Note that the number of times (frequency) of transmitting the uncertain text data by the voice text conversion server 9 can be set as appropriate.

[0166] When the transceiver unit 91 of the voice text conversion server 9 determines that the text conversion of the sound data is completed by the conversion unit 93 and the text data is determined, the transceiver unit 91 transmits the determined text data to the shared support server 6 (step Sc5). In this case, the transceiver unit 91 transmits a confirmation flag indicating that the text data is determined, together with the text data (「In this fiscal year, more than twice the results of last year.」) converted by the conversion unit 93. As a result, the transceiver unit 61 of the shared support server 6 receives the text data (determined text data) transmitted from the voice text conversion server 9.

[0167] Here, the text data received in step Sc5 is stored and managed in the content management DB6005 by the process of step S79 in FIG. 27. In this case, the date and time related to the process of the text data received in step Sc5 is the date and time when the text data is received in step Sc5 from the date and time when the audio data is transmitted in step Sb2. That is, the start date and time of the content process stored in the content management DB6005 is the date and time when the audio data is transmitted in step Sb2, and the end date and time of the content process is the date and time when the text data is received in step Sc5. In this way, the shared support server 6 can manage the date and time when the text data (confirmed text data) is received from the date and time when the transmission of the audio data corresponding to the text data to the voice text conversion server 9 is started as the start and end dates and times of the content process together with the text data transmitted from the voice text conversion server 9.

[0168] Then, the transceiver 61 of the shared support server 6 transmits the text data (confirmed text data) transmitted from the voice text conversion server 9 to the electronic blackboard 2 (step Sc6). As a result, the transceiver 21 of the electronic blackboard 2 receives the text data transmitted from the shared support server 6. Then, the transceiver 21 of the electronic blackboard 2 causes the text data transmitted from the shared support server 6 to be displayed on the display 220.

[0169] In this way, the shared system 1 can convert the audio data generated by the executed event into text data using the voice text conversion server 9, and can store and manage the information related to the generation time of the text data (the start date and time and the end date and time of the content process) in the shared support server 6 and the schedule management server 8.

[0170] Returning to FIG. 27, the description of the registration process of the execution event history will be continued. The transmission / reception unit 91 transmits the text data converted by the conversion unit 93 to the shared support server 6 (step S76). Here, the transmission / reception unit 91 transmits, together with the text data, information indicating the date and time acquired in step S75-1 to the shared support server 6. As a result, the transmission / reception unit 61 of the shared support server 6 receives the text data.

[0171] Next, the generation unit 64 generates a unique content processing ID for identifying the content processing generated by the event (step S77). Also, the generation unit 64 generates a URL of content data indicating the content of the content (step S78). Then, the storage / reading processing unit 69 manages, in association with the content management DB 6005 (see FIG. 10(B)), for each execution event ID received in step S72, the type of content processing, the start date and time and end date and time of the content processing, the content processing ID generated in step S77, and the URL indicating the storage destination of the content generated in step S78 (step S79).

[0172] Here, when the type of content processing is "voice-to-text conversion", the start date and time and end date and time of the content processing are the date and time when the audio data is converted into text data. Here, the date and time when the audio data is converted into text data is, as shown in FIG. 28B, the date and time when the audio data is transmitted by the transmission / reception unit 61 of the shared support server 6 and the date and time when the text data is received by the transmission / reception unit 61 of the shared support server 6. Note that the date and time when the audio data is converted into text data may be the date and time when the audio data is received by the transmission / reception unit 91 of the voice-to-text conversion server 9 and the date and time when the text data is transmitted by the transmission / reception unit 91 of the voice-to-text conversion server 9. Also, when the type of content processing is "voice-to-text conversion", the start date and time and end date and time of the content processing may be the same date and time as the start date and time and end date and time of the content processing related to the audio data to be text-converted.

[0173] Also, when the type of content processing is "recording", "snapshot", or "document transmission", the start date and time and end date and time of content processing are the date and time when content data (audio data, image data, file data) is received by the transmission / reception unit 61 of the shared support server 6. Note that when the type of content processing is "recording", "snapshot", or "document transmission", the start date and time and end date and time of content processing may be the date and time when content data is transmitted by the transmission / reception unit 21 of the electronic blackboard 2. Also, when the type of content processing is "recording", the start date and time and end date and time may be the start date and time and end date and time when recording is performed by the image / audio processing unit 23. Further, when the type of content processing is "snapshot", the start date and time and end date and time of content processing may be the date and time when a snapshot (capture) is performed by the image / audio processing unit 23.

[0174] Next, in FIG. 29, the storage / read processing unit 69 of the shared support server 6 searches the execution event management DB 6004 (see FIG. 10(A)) using the execution event ID received in step S72 as a search key, and reads out the corresponding project ID (step S91). Next, the storage / read processing unit 69 searches the user authentication management DB 6001 (see FIG. 9(A)) using the user ID of the content transmission source as a search key, and reads out the corresponding organization ID (step S92).

[0175] Next, the storage / read processing unit 69 searches the access management DB 6002 (see FIG. 9(B)) using the organization ID read in step S92 as a search key, and reads out the corresponding access ID and access password (step S93).

[0176] Next, the transmission / reception unit 61 transmits execution event history registration request information indicating a registration request for the execution event history to the schedule management server 8 (step S94). This execution event history registration request information includes the project ID read in step S91, the execution event ID received in step S72, the user ID of the transmission source, the content data, the content processing ID generated in step S77, the URL of the content data generated in step S78, the access ID and access password read in step S93, and the start date and time and end date and time of the content processing. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the execution event history registration request information.

[0177] Next, in the schedule management server 8, the authentication unit 82 authenticates the shared support server 6 using the access ID and access password (step S95). Since this authentication is the same process as in step S38, the description is omitted. Here, the description continues for the case where it is valid.

[0178] Next, the storage / read processing unit 89 stores and manages various data (information) received in step S94 in the execution event history management DB8008 (see FIG. 14(A)) (step S96). The storage / read processing unit 89 stores the various data (information) in the execution event history management DB8008 in association with the project ID and execution event ID received in step S94. As a result, the schedule management server 8 manages data with the same content as on the shared support server 6 side.

[0179] Also, the generation unit 83 of the schedule management server 8 generates association information associating the content data received in step S94 for each time zone when the content is generated (step S97). Here, the time zone when the content is generated included in the association information is generated using the start date and time of the event stored in the event management DB8005, and the start date and time and end date and time of the content process stored in the execution event history management DB8008. That is, the time zone when the content is generated indicates the elapsed time from the start date and time of the event at the point in time when the content is generated in the executed event. Then, the storage / read processing unit 89 of the schedule management server 8 stores and manages the association information generated by the generation unit 83 in association with the project ID and the execution event ID received in step S94 in the association information management DB8010 (see FIG. 15) (step S98). Thereby, the schedule management server 8 manages content data with different types of content processing in association with each generation time of the content.

[0180] Here, with reference to FIG. 28C, the data format of the text data stored in the shared support server 6 and the schedule management server 8 will be described. FIG. 28C is a diagram showing an example of a data file including text data. The data file d1 shown in FIG. 28C is stored and managed at the URL of the storage destination of the content shown in the content management DB6005 and the execution event history management DB8008.

[0181] As shown in FIG. 28C, the data file d1 contains 10 text data (transcripts). Further, each text data (0 to 9) contains a sequence number (seq_num) indicating the order in which the content processing occurred, text data (text), the start date and time (start time) of the content processing, the end date and time (end_time) of the content processing, and a content processing ID (id). These pieces of information are the same as the information corresponding to the respective items in the content management DB6005 and the execution event history management DB8006. Note that the number of text data contained in the data file d1 is not limited to this.

[0182] In this way, the shared support server 6 and the schedule management server 8 can associate text data with time information (start date and time and end date and time of content processing) related to the processing of the text data and manage it in a data file d1 as shown in FIG. 28C.

[0183] As described above, the electronic blackboard 2 can transmit the execution event ID of the event in a predetermined project and the content generated by the execution of the corresponding event to the schedule management server 8. Further, the schedule management server 8 can store the received content in the execution event history management DB8008 for each execution event ID associated with the project ID. As a result, the shared system 1 can store the content generated by the event for each project by allowing the user to select which project the event belongs to at the start of the event.

[0184] (Registration process of action item) Here, with reference to FIGS. 30 to 33, the processing when the generated content is an action item will be described. FIG. 30 is a flowchart showing the registration process of the action item. FIG. 31 is an example of a screen for recognizing the action item. FIG. 32 is an example of a screen showing a list of candidates for executing the action item. FIG. 33 is an example of a screen showing a calendar for selecting the expiration date of the action item.

[0185] First, in FIG. 30, when the user presses the icon r1, the reception unit 22 receives a registration request for the action item (step S71-1). Next, as shown in FIG. 31, after the user uses the electronic pen 2500 to draw an action item (here, "submit minutes") on the drawing screen 260a of the electronic blackboard 2, and then surrounds the image (drawn image) 261 showing the content of the action item with a line (designated area) 262, the reception unit 22 receives the designation of a predetermined area including the image 261, and the recognition unit 26 recognizes the image 261 included in the predetermined area (step S71-2). Although FIG. 31 illustrates an example where the image (drawn image) 261 is surrounded by a line (designated area) 262, the recognition unit 26 may be configured to recognize an area at an arbitrary distance from the designated area 262 as the image 261. The designation of the predetermined area is performed, for example, by drawing a predetermined shape such as a circle or a polygon with the electronic pen 2500.

[0186] Next, as shown in FIG. 32, the display control unit 24 displays a list of candidate executors 265 for the action item on the drawing screen 260b (step S71-3). Then, when the user uses the electronic pen 2500 to select a specific executor for executing the action item, the reception unit 22 receives the selection of the specific executor (step S71-4).

[0187] Next, as shown in FIG. 33, the display control unit 24 displays a calendar 267 for receiving the specification of the execution deadline of the action item on the drawing screen 260c (step S71-5). Then, when the user selects a specific deadline using the electronic pen 2500, the reception unit 22 receives the selection of the specific deadline (step S71-6). Note that the calendar 267 is an example of a deadline specification screen. The deadline specification screen may be a date list or the like on which the day of the week and the like are not described.

[0188] Through such processing, the electronic blackboard 2 transmits, as a content registration request, content registration request information indicating a registration request for an action item to the shared support server 6. This content registration request information includes an execution event ID indicating the event in which the action item occurred, the user ID of the executor of the action item selected in step S71-4, the image data of the action item recognized in step S71-2 (here, the image data of "submit minutes"), and the execution deadline of the action item received in step S71-6. That is, the transmission / reception unit 21 transmits the data of the image within the predetermined area as image data indicating the content of the action item generated by the event being executed. Thereby, the transmission / reception unit 61 of the shared support server 6 receives the content registration request information. Note that the processing after the shared support server 6 receives the content registration request information is the same as the processing shown in FIGS. 27 and 29, and thus the description thereof is omitted.

[0189] <Event end processing> Subsequently, with reference to FIGS. 34 to 38, the processing for ending the event being executed will be described. FIGS. 34 and 35 are sequence diagrams showing the event end processing. FIG. 36 is a diagram showing an event end screen on the electronic blackboard. FIG. 37 is a diagram showing a file data upload screen on the electronic blackboard. FIG. 38 is a diagram showing a file data upload completion screen on the electronic blackboard.

[0190] In FIG. 34, when user A closes the screen displayed on the display 220, the reception unit 22 receives the end of the executed event (step S301).

[0191] Then, the transmission / reception unit 21 transmits to the shared support server 6 execution event start / end information indicating the start date / time and end date / time of the executed event, and file data registration request information indicating a registration request for file data (step S302). This execution event start / end information includes an execution event ID, an event name, the start date / time and end date / time of the event. Also, this file data registration request information includes an execution event ID, the user ID of the transmission source, and file data. Thereby, the transmission / reception unit 61 of the shared support server 6 receives the execution event start / end information and the file data registration request information.

[0192] Next, the generation unit 64 of the shared support server 6 generates a unique content processing ID for identifying the content processing generated by the event (step S303). Also, the generation unit 64 generates a URL of content data indicating the content of the content (step S304). Then, the storage / read processing unit 69 associates and manages, for each execution event ID received in step S302, the type of content processing, the start date / time and end date / time of the content processing, the content processing ID generated in step S303, and the URL indicating the storage destination of the content generated in step S304 with respect to the content management DB6005 (see FIG. 10(B)) (step 305).

[0193] Next, the storage / read processing unit 69 of the shared support server 6 reads out the corresponding project ID by searching the execution event management DB6004 (see FIG. 10(A)) using the execution event ID received in step S72 as a search key (step S306). Next, the storage / read processing unit 69 reads out the corresponding organization ID by searching the user authentication management DB6001 (see FIG. 9(A)) using the user ID of the content transmission source as a search key (step S307).

[0194] Next, the memory / read processing unit 69 searches the access management DB6002 (see FIG. 9(B)) using the organization ID read in step S92 as a search key, and reads out the corresponding access ID and access password (step S308).

[0195] Next, in FIG. 35, the transmission / reception unit 61 transmits to the schedule management server 8 the execution event start / end information received in step S302 and the file data registration request information indicating the registration request of the file data (step S309). This file data registration request information includes the project ID read in step S306, the execution event ID received in step S302, the user ID of the transmission source and the file data, the content processing ID generated in step S303, the URL of the file data generated in step S304, the access ID and access password read in step S308, and the start date / time and end date / time of the content processing. As a result, the transmission / reception unit 81 of the schedule management server 8 receives the execution event start / end information and the file data registration request information.

[0196] Next, in the schedule management server 8, the authentication unit 82 authenticates the shared support server 6 using the access ID and access password (step S310). Since this authentication is the same process as step S38, the description is omitted. Here, the case of being valid will be continued with the description.

[0197] Next, the memory / read processing unit 89 of the schedule management server 8 stores the execution event start / end information received in step S309 in the execution event management DB8009 (see FIG. 14(B)) (step S311). In this case, the memory / read processing unit 89 adds the execution event start / end information for one record to the execution event management table of the execution event management DB8009.

[0198] Next, the memory / read processing unit 89 stores and manages various data (information) received in step S309 in the execution event history management DB 8008 (step S312). The memory / read processing unit 89 stores various data (information) including file data in association with the project ID and execution event ID received in step S309 in the execution event history management DB 8008 (see FIG. 14(A)). Thereby, the schedule management server 8 manages the same file data as that on the shared support server 6 side.

[0199] Next, the transceiver unit 81 transmits file data registration information indicating that file data has been registered to the shared support server 6 (step S313). Thereby, the transceiver unit 61 of the shared support server 6 receives the file data registration information.

[0200] Then, the transceiver unit 61 of the shared support server 6 transmits the file data registration information transmitted from the schedule management server 8 to the electronic blackboard 2 (step S314). Thereby, the transceiver unit 21 of the electronic blackboard 2 receives the file data registration information.

[0201] Next, when the memory / read processing unit 29 of the electronic blackboard 2 receives the file data registration information by the transceiver unit 21, it deletes the file data stored in a specific storage area of the storage unit 2000 (step S315). Thereby, the electronic blackboard 2 can reduce the risk of leakage of conference information and the like by deleting the file data already transmitted to the shared support server 6.

[0202] Here, the screen transition of the screen displayed on the electronic blackboard 2 when ending the event being executed will be described. First, in step S301, when the reception unit 22 receives the end of the event being executed, the display control unit 24 causes the event end screen 270 as shown in FIG. 36 to be displayed on the display 220. This event end screen 270 includes a toolbar area 271, a file display area 272, a file upload selection area 273, a "Close" button 278 to be pressed when proceeding with the end process of the event being executed, and a "CANCEL" button 279 to be pressed when canceling the end process of the event being executed. In the toolbar area 271, images indicating the icons r1, r2, r3 as shown in FIG. 26 are displayed. The file display area 272 displays file data images 272a, 272b, 272c for identifying the file data of the material files stored in a specific storage area of the storage unit 2000. The file upload selection area 273 includes a checkbox for receiving a selection as to whether to upload the file data corresponding to the file data image displayed in the file display area 272 to the shared support server 6.

[0203] When the reception unit 22 receives the selection of the "Close" button 278 in a state where the file upload selection area 273 is selected, the display control unit 24 causes a file upload screen 280a as shown in FIG. 37 to be displayed on the display 220. This file upload screen 280a is displayed on the display 220 when the file data stored in a specific storage area of the storage unit 2000 is being uploaded to the shared support server 6. The file upload screen 280a includes the event name 281 of the event to be ended, the end date and time 282 of the event, a file data upload progress display area 283, and a "CANCEL" button 288 to be pressed when interrupting (canceling) the upload of the file data. In the upload progress display area 283, the number of file data to be uploaded (for example, "3" shown in FIG. 37) and the number of file data for which the upload has been completed (for example, "0" shown in FIG. 37) are displayed.

[0204] When the upload of the file data is completed, the display control unit 24 causes an upload completion screen 280b as shown in FIG. 38 to be displayed. The upload completion screen 280b includes a "Close" button 288 to be pressed when ending the event. When the upload completion screen 280b is displayed on the display 220, as shown in step S315, the storage / read processing unit 29 of the electronic blackboard 2 deletes the file data for which the upload has been completed from a specific storage area of the storage unit 2000.

[0205] On the other hand, when the upload of the file data fails while the file upload screen 280a is being displayed on the display 220, the display control unit 24 displays information (e.g., file name) for identifying the file data for which the upload has failed. Thereby, for example, when a failure such as a failure of the communication network 10 occurs, the participants in the event can take home the file data for which the upload has failed by printing the file data created or edited in the event or storing the file data in the USB memory 2600 connected to the electronic blackboard 2.

[0206] Also, when file data remains in a specific storage area of the storage unit 2000 even after the event has ended, the storage / read processing unit 29 of the electronic blackboard 2 deletes the file data stored in the specific storage area, for example, at the start of the next event using the electronic blackboard 2. Thereby, the electronic blackboard 2 can reduce the risk of leakage of conference information and the like.

[0207] <Execution event history viewing process> Next, the browsing process of the execution event history will be described with reference to FIGS. 39 to 45. FIGS. 39 and 40 are sequence diagrams showing the browsing process of the execution event history. FIG. 41 is a diagram showing a project list screen on the PC. FIG. 42 is a diagram showing an execution event list screen on the PC. FIGS. 43 and 44 are diagrams showing an execution event history screen. FIG. 45 is a diagram showing an action item screen on the PC. Note that since the processes in steps S111 to S117 in FIG. 39 are the same as the processes in steps S11 to S17 in FIG. 16, the description thereof will be omitted.

[0208] In the initial screen 540 shown in FIG. 18, when the user presses the "Execution Event History Browsing" button 543, the reception unit 52 receives the browsing of the execution event history (step S118).

[0209] Then, the transmission / reception unit 51 transmits execution event history browsing request information indicating an execution event history browsing request to the schedule management server 8 (step S119). As a result, the transmission / reception unit 81 of the schedule management server 8 receives the execution event history browsing request information.

[0210] Next, the storage / reading processing unit 89 of the schedule management server 8 searches the project member management DB 8007 using the user ID and organization ID received in step S113 as search keys, and reads out the corresponding project ID and project name (step S120). Then, the transmission / reception unit 81 transmits the project ID and project name to the PC 5 (step S121).

[0211] Next, the generation unit 56 of the PC 5 generates a project list screen 560 as shown in FIG. 41 using the project ID and project name received in step S121 (step S122). Then, on the PC 5, the display control unit 54 causes the project list screen 560 generated by the generation unit 56 to be displayed on the display 508 (step S123). This project list screen 560 shows the same content as the project list screen 240 displayed on the electronic blackboard 2 as shown in FIG. 24. That is, the project icons 561 to 566 and the buttons 568 and 569 in FIG. 41 correspond to the project icons 241 to 246 and the buttons 248 and 249 in FIG. 24, respectively.

[0212] Next, in FIG. 40, when user A presses the project icon 561 with, for example, the mouse 512 or the like, the reception unit 52 receives the selection of the project indicated by the project icon 561 (step S124).

[0213] Next, the transmission / reception unit 51 of the PC 5 transmits the project ID of the project selected in step S124 to the schedule management server 8 (step S125). As a result, the schedule management server 8 receives the project ID.

[0214] Next, in the schedule management server 8, the storage / reading processing unit 89 searches the execution event history management DB 8008 using the project ID received in step S125 as a search key to read the corresponding execution event ID (step S126). In this case, the storage / reading processing unit 89 reads all the execution event IDs associated with the project ID received in step S125. Then, the storage / reading processing unit 89 searches the execution event management DB 8009 using the execution event ID read in step S126 as a search key to read the corresponding execution event start / end information (step S127). This execution event start / end information includes the execution event ID, event name, start date / time, and end date / time.

[0215] Then, the transmission / reception unit 81 transmits the execution event ID, event name, start date / time, and end date / time read in step S127 to the PC 5 (step S128). As a result, the transmission / reception unit 51 of the PC 5 receives the execution event ID, event name, start date / time, and end date / time.

[0216] Next, the generation unit 56 of the PC 5 generates an execution event list screen 570 as shown in FIG. 42 using the data (information) received in step S128 (step S129). Then, on the PC 5, the display control unit 54 causes the display 508 to display the execution event list screen 570 generated by the generation unit 56 (step S130). As shown in FIG. 42, the execution event list screen 570 includes information 571 to 573 indicating execution events. For example, the information 571 to 573 indicating execution events includes the execution event name, the start date / time of the event, and the end date / time of the event. The information 571 to 573 indicating execution events is an example of a history information selection area that accepts a selection for displaying history information indicating the execution content of the event. Also, in the lower right part of the execution event list screen 570, there is a "Close" button 575 that is pressed when closing the execution event list screen 570. Furthermore, in the lower left part of the execution event list screen 570, there is an "Action Item" button 577 that is pressed when browsing action items. The "Action Item" button 577 is an example of an action item selection area that accepts a selection for displaying action items.

[0217] Next, the reception unit 52 of the PC 5 receives the selection of the execution event included in the execution event list screen 570 (step S131). Specifically, when the user selects the information indicating the execution event included in the execution event list screen 570, the reception unit 52 receives the selection of the information indicating the execution event. Then, the transmission / reception unit 51 of the PC 5 transmits the execution event ID of the execution event selected by the reception unit 52 to the schedule management server 8 (step S132). As a result, the transmission / reception unit 81 of the schedule management server 8 receives the execution event ID.

[0218] Next, the storage / reading processing unit 89 of the schedule management server 8 reads the corresponding execution event history information by searching the execution event history management DB 8008 using the execution event ID received in step S132 as a search key (step S133). This execution event history information includes the content processing ID, the type of content processing, the start date and time of the content processing, and the end date and time.

[0219] Also, the storage / reading processing unit 89 of the schedule management server 8 reads the corresponding related information by searching the related information management DB 8010 using the execution event ID received in step S132 as a search key (step S134). This related information includes the generation time zone of the content, the content processing ID, the type of content processing, and the sequence No. The generation time zone of the content included in the related information is an example of time information.

[0220] Then, the storage / reading processing unit 89 reads the content data indicating the content of the execution event history from the storage destination of the content data indicating the content of the execution event history read in step S133 (step S135). Then, the transmission / reception unit 81 transmits the read related information and content data to the PC 5 (step S136). As a result, the transmission / reception unit 51 of the PC 5 receives the related information and content data.

[0221] Next, the sound control unit 58 of the schedule management server 8 sets the playback position of the sound data, which is the content data received in step S136 (step S137). In this case, the sound control unit 58 sets, as the playback position of the sound data, the playback position corresponding to the sound data associated with the content generation time zone "00:00" among the received related information.

[0222] Next, the generation unit 56 of the PC 5 generates an execution event history screen 580 as shown in FIG. 43 using the related information and the content data received in step S136 (step S138). More specifically, the generation unit 56 generates the execution event history screen 580 so that the images related to the text data, which is the received content data, are displayed in the text data display area 582 in the order of the sequence No. included in the received related information. Further, the generation unit 56 generates the execution event history screen 580 so that the images related to the screen data, which is the received content data, are displayed in the snapshot display area 583 in the order of the sequence No. included in the received related information. Furthermore, the generation unit 56 generates the execution event history screen 580 so that the playback position specific image 581p included in the playback position display area 581 is displayed at the position related to the sound data set in step S137 during the entire playback time of the sound data.

[0223] Then, the display control unit 54 of the PC 5 causes the execution event history screen 580 generated by the generation unit 56 to be displayed on the display 508 (step S139). Also, the sound control unit 58 of the PC 5 plays the sound data of the playback location set in step S137. As shown in FIG. 43, on the execution event history screen 580, the data of the content generated by the executed event is shown for each type of content processing. The execution event history screen 580 includes a playback location display area 581 for displaying the playback location of the recording data, a text data display area 582 for displaying voice text, a snapshot display area 583 for displaying snapshots, an "action item" button 584 to be pressed when browsing action items, a "document file" button 585 to be pressed when browsing the documents used in the event, and a "pagination button" 588 to be pressed when displaying a pagination display area 589 described later.

[0224] Here, the playback location display area 581 includes a playback location specific image 581p indicating the playback location in the total playback time, a slider 581b indicating the playback history (playback process) of the recording data, and a playback time display image 581t indicating the playback time in the total playback time. Also, the playback location specific image 581p and the slider 581b constitute a seek bar. The seek bar is for displaying the playback location of the recording data and is also an operation area for designating the playback location of the recording data. The user can visually grasp where in the recording data is being played from the start to the end by the positions of the playback location specific image 581p and the slider 581b. Also, the user can move the playback location specific image 581p using an input means such as the mouse 507 and thereby play the recording data from an arbitrary playback location. The playback location specific image 581p is an example of a first image. Also, the seek bar including the playback location specific image 581p and the slider 581b is an example of a second image.

[0225] In addition, in the text data display area 582, images 582a, 582b, 582c, 582d, 582e indicating text data are displayed in time series from top to bottom. Similarly, in the snapshot display area 583, snapshot (capture) images 583a, 583b, 583c are displayed in time series from top to bottom. In this way, the text data display area 582 and the snapshot display area 583 are displayed in parallel in time series. Thereby, for example, if the user views image 583b, the user can easily grasp the time period during which the pie chart was explained in the meeting, and thus can easily search for images 582c, 582d indicating the text data displayed in parallel with this image 583b. In this case, images 583a, 583b, 583c are snapshot images of the entire display 220. In this way, on the execution event history screen 580, images of corresponding content are displayed in the order of generation time in specific display areas for each type of content (text data, snapshot).

[0226] Also, FIG. 44 is an image displayed on the display 508 when the user presses the "pagination" button 588 shown in FIG. 43. Here, "pagination" is a navigation provided to move between pages when information is expanded and divided into multiple pages. For example, in the case of a web page, "pagination" means dividing a page with a large amount of content into multiple pages and arranging links to each page. By using "pagination", it becomes easier for the user to access and easier for the user to grasp which page they are viewing. Note that "pagination" is also referred to as "page splitting", "page forwarding", "paging", and "pager".

[0227] When the user presses the "Pagination" button 588, the reception unit 52 receives the pressing of the "Pagination" button 588. As a result, as shown in FIG. 44, the display control unit 54 causes a pagination display area 589 to be displayed at the lower part of the execution event history screen 580. In this pagination display area 589, for example, the time zone in which content is generated in an event such as a meeting (here, every 5 minutes) is shown, and the time zone that is the same as the time of the playback position of the playback position specific image 581p is surrounded by a frame (elapsed time display area 589j). Note that until the pressing of the "Pagination" button 588 is received, the reception unit 52 is in a waiting state for the pressing.

[0228] As a result, the sharing system 1 can easily share the execution event history generated by events belonging to the same project among members belonging to the same project. In addition, the sharing system 1 uses the project registration screen 520 displayed on the PC 5 to register or change the members belonging to the project, so that the user can omit processing such as viewing settings for each executed event in order to view the execution event history managed for each project.

[0229] On the other hand, in step S131, when the reception unit 52 of the PC 5 receives the selection of the "Action Item" button 577 on the execution event list screen 570, the generation unit 56 generates an action item screen 590 as shown in FIG. 45. Then, the display control unit 54 causes the action item screen 590 generated by the generation unit 56 to be displayed on the display 508. As shown in FIG. 45, the action item screen 590 includes action item information 591 to 594. For example, the action item information 591 includes an image showing the content of the action item identified in FIG. 31, the user name selected in FIG. 32, and the due date specified in FIG. 33. Also, in the lower right part of the action item screen 590, there is a "Close" button 599 to be pressed when closing the action item screen 590. All the action items associated with the project ID for which selection was received in step S124 are displayed on the action item screen 590 shown in FIG. 45. In the execution event history screen 580 shown in FIG. 43, when the reception unit 52 receives the selection of the "Action Item" button 584, similarly, the display control unit 54 causes the action item screen 590 to be displayed on the display 508.

[0230] <Selection Process of Execution Event History> Subsequently, with reference to FIGS. 46 to 51, the selection process of the execution event history will be described. FIGS. 46 and 49 to 51 are sequence diagrams showing the selection process of the execution event history. FIGS. 47 and 48 are diagrams showing the execution event history screen on the PC.

[0231] First, the reception unit 52 of the PC 5 receives the selection of a specific image included in the text data display area 582 of the execution event history screen 580 (step S151). Specifically, for example, as shown in FIG. 47, when the user selects an image (e.g., image 582t) indicating specific text data included in the text data display area 582 of the execution event history screen 580 using the mouse pointer 586, the reception unit 52 receives the selection of the specific image included in the text data display area 582.

[0232] The transceiver unit 51 of the PC 5 transmits the information related to the selection received by the reception unit 52 to the schedule management server 8 (step S152). Here, the selection information includes the content processing ID related to the specific image (e.g., image 582t) for which the selection was received. Thereby, the transceiver unit 81 of the schedule management server 8 receives the selection information transmitted from the PC 5.

[0233] Next, the storage / reading processing unit 89 of the schedule management server 8 reads the corresponding related information by searching the related information management DB 8010 using the content processing ID included in the selection information received in step S152 as a search key (step S153). More specifically, the storage / reading processing unit 89 reads the related information associated with the content processing ID included in the selection information from the related information management DB 8010. This related information includes the generation time zone of the content, the content processing ID, the type of content processing, and the sequence No. The generation time zone of the content included in the related information is an example of time information. Thereby, the schedule management server 8 can identify the content that occurred in the same time zone as the image data for which the user input was received from the data (information) related to various contents included in the related information.

[0234] In addition, the storage / readout processing unit 89 of the schedule management server 8 searches the execution event history management DB 8008 using the content processing ID included in the related information read in step S153 as a search key, and reads out the corresponding execution event history information (step S154). This execution event history information includes the content processing ID, the type of content processing, and the content of the content.

[0235] Then, the storage / readout processing unit 89 reads out content data indicating the content of the execution event history from the storage destination of the content data indicating the content included in the execution event history information read in step S154 (step S155). Then, the transmission / reception unit 81 transmits the read related information and content data to the PC 5 (step S156). As a result, the transmission / reception unit 51 of the PC 5 receives the related information and content data.

[0236] Next, the sound control unit 58 of the PC 5 sets the playback position corresponding to the sound data, which is the content data received in step S156, as the playback position (start time of the sound data) of the sound data (step S157).

[0237] Next, the generation unit 56 of the PC 5 generates an execution event history screen 580 based on the content data received in step S156 (step S158). Specifically, the generation unit 56 generates the execution event history screen 580 so that an image related to the text data, which is the received content data, is displayed in the text data display area 582. In addition, the generation unit 56 generates the execution event history screen 580 so that an image related to the screen data, which is the received content data, is displayed in the snapshot display area 583. Further, the generation unit 56 generates the execution event history screen 580 so that the playback position specific image 581p included in the playback position display area 581 is displayed at the playback position related to the total playback time of the sound data, which is the received content data.

[0238] Furthermore, the generation unit 56 generates an execution event history screen 580 such that the reproduction location specific image 581p included in the reproduction location display area 581 is displayed at the position related to the audio data set in step S136 during the entire reproduction time of the audio data.

[0239] The display control unit 54 causes the display 508 to display the execution event history screen 580 generated in step S158 (step S159). In step S159, the execution event history screen 580 displayed on the display 508 is shown in FIG. 48. In the execution event history screen 580 shown in FIG. 48, the position of the reproduction location specific image 581p included in the reproduction location display area 581 has moved to the time position corresponding to the image 582t selected by the user. In this way, when a specific image is selected from among the images related to the text data obtained by text-converting the audio data, the PC 5 moves the reproduction location specific image 581p to the time position indicated by the generation time zone of the content related to the selected specific image. Thereby, the PC 5 can reproduce the audio data at the reproduction location corresponding to the selected image simply by the user selecting the image related to the text data for which the user wishes to confirm the voice. Further, the PC 5 can easily let the user grasp which reproduction location in the entire reproduction time the audio data related to is being reproduced by moving the reproduction location specific image 581p in correspondence with the movement of the image related to the text data.

[0240] In addition, the processes shown in FIGS. 49 to 51 are processes when content data (execution event history) of a different type from the text data (execution event history) shown in FIG. 46 is selected. FIG. 49 shows the process when an image related to the screen data displayed in the snapshot display area 583 is selected. First, the reception unit 52 of the PC 5 receives the selection of a specific image included in the snapshot display area 583 of the execution event history screen 580 (step S161). Specifically, for example, when the user selects a specific image (for example, the snapshot image 583r) included in the snapshot display area 583 of the execution event history screen 580 using the mouse pointer 586, the reception unit 52 receives the selection of the specific image included in the snapshot display area 583. Since the processes of steps S162 to S169 are the same as the processes of steps S152 to S159 shown in FIG. 46 respectively, the description thereof is omitted.

[0241] FIG. 50 shows the process when an image related to the screen data displayed in the pagination display area 589 is selected. First, the reception unit 52 of the PC 5 receives the selection of an image indicating a specific occurrence time zone included in the pagination display area 589 of the execution event history screen 580 (step S171). Specifically, for example, when the user selects a specific elapsed time display image (for example, the elapsed time display area 589j) included in the pagination display area 589 of the execution event history screen 580 using the mouse pointer 586, the reception unit 52 receives the selection of the specific image included in the snapshot display area 583.

[0242] The transceiver unit 51 of the PC 5 transmits the selection information related to the selection received by the reception unit 52 to the schedule management server 8 (step S172). Here, the selection information includes the content occurrence time zone related to the elapsed time display image for which the selection was received. Thereby, the transceiver unit 81 of the schedule management server 8 receives the selection information transmitted from the PC 5.

[0243] Next, the storage / reading processing unit 89 of the schedule management server 8 searches the related information management DB8010 using the content generation time zone included in the selection information received in step S172 as a search key, and reads out the corresponding related information (step S173). More specifically, the storage / reading processing unit 89 reads out the related information associated with the content generation time zone included in the selection information from the related information management DB8010. This related information includes the content generation time zone, the content processing ID, the type of content processing, and the sequence No. Thereby, the schedule management server 8 can specify the content generated in the time zone shown in the elapsed time display image for which the user's selection has been received from the data (information) related to various contents included in the related information. Note that the processing of steps S174 to S179 is the same as the processing of steps S154 to S159 shown in FIG. 46, respectively, and thus the description thereof is omitted.

[0244] Furthermore, FIG. 51 shows the processing when the reproduction location specifying image 581p displayed in the reproduction location display area 581 is selected. First, the reception unit 52 of the PC 5 receives the movement of the reproduction location specifying image 581p included in the reproduction location display area 581 of the execution event history screen 580 (step S181). Specifically, for example, when the user moves the reproduction location specifying image 581p included in the reproduction location display area 581 of the execution event history screen 580 using the mouse pointer 586, the reception unit 52 receives the movement of the reproduction location specifying image 581p included in the reproduction location display area 581.

[0245] The transmission / reception unit 51 of the PC 5 transmits the selection information related to the movement received by the reception unit 52 to the schedule management server 8 (step S182). Here, the selection information includes the content processing ID related to the audio data of the reproduction location of the received movement destination. Thereby, the transmission / reception unit 81 of the schedule management server 8 receives the selection information transmitted from the PC 5. Note that the processing of steps S182 to S189 is the same as the processing of steps S152 to S159 shown in FIG. 46, respectively, and thus the description thereof is omitted.

[0246] In FIGS. 39 to 51, the case where the execution event history is viewed by the PC 5 has been described. However, in FIG. 26, even when the user presses the icon r2, the execution event history can be viewed on the electronic blackboard 2 by the same process. Further, the processes and screen examples shown in FIGS. 39 to 51 may perform the same processes and display the same screens not only on the PC 5 but also on the electronic blackboard 2, the video conferencing terminal 3, or the car navigation device 4.

[0247] Furthermore, in FIGS. 39 to 51, the case where the image data related to each screen to be displayed on the PC 5 is generated by the generation unit 56 of the PC 5 has been described. However, this image data may be generated by the schedule management server 8. In this case, in the schedule management server 8, the image data is generated by the generation unit 83, and the generated image data is transmitted to the PC 5 by the transmission / reception unit 81.

[0248] <<Main Effects of the Embodiment>> According to this embodiment, as shown in FIGS. 46 to 48, the PC 5 It is a display terminal capable of communicating with a schedule management server 8 (an example of a management system) that manages audio data and text data converted from the audio data. Further, the transmission / reception unit 51 of the PC 5 receives audio data, text data, and time information regarding the generation time of the text data from the schedule management server 8. The display control unit 54 (an example of display control means) causes the received text data to be displayed on the display 508 (an example of a display unit) in the order of the generation time of the text data, and causes a reproduction position specific image 581p (an example of a first image) indicating the reproduction position in the total reproduction time of the received audio data to be displayed on the display 508. The reception unit 52 (an example of reception means) receives a selection for specific text data among the displayed text data. Then, the display control unit 54 of the PC 5 causes the reproduction position specific image 581p to be displayed at the position of the time indicated by the time information corresponding to the specific text data for which the selection has been received. Thereby, when the user of the PC 5 wants to reproduce the recording data corresponding to the displayed text data by looking at the displayed text data, the reproduction position of the recording data to be reproduced can be easily found.

[0249] Also, according to the present embodiment, as shown in FIGS. 46 to 51, the PC5 is a display terminal capable of communicating with a schedule management server 8 (an example of a management system) that manages audio data related to content generated in the executed event and content image data related to the content. Further, the transmission / reception unit 51 of the PC5 receives audio data, content image data, and time information related to the generation time of the content from the schedule management server 8. The display control unit 54 (an example of display control means) displays the content image related to the content image data on the display 508 in the order of the generation time of the content based on the received time information, and also displays a reproduction location specific image 581p (an example of a first image) indicating the reproduction location in the total reproduction time of the audio data on the display 508. The reception unit 52 (an example of reception means) receives a selection for a specific content image among the displayed content images. Then, the display control unit 54 of the PC5 displays the reproduction location specific image 581p at the time position indicated by the time information corresponding to the specific content image for which the selection has been received. Thereby, when the user of the PC5 wants to play the recording data corresponding to the displayed image by looking at the displayed image, the user can easily find the reproduction location of the recording data to be played.

[0250] Furthermore, according to the present embodiment, as shown in FIGS. 46 to 51, the audio control unit 58 (an example of reproduction means) of the PC5 (an example of a display terminal) plays the audio data related to the time indicated by the time information corresponding to a specific content image, and the display control unit 54 (an example of display control means) displays the reproduction location of the played audio data as a reproduction location specific image 581p (an example of a first image). Thereby, the PC5 can play the audio data at the reproduction location corresponding to the image simply by the user selecting the content image.

[0251] Also, according to this embodiment, as shown in FIGS. 46 to 51, the display control unit 54 (an example of display control means) of the PC 5 (an example of a display terminal) displays images related to text data and screen data, which are content image data, in parallel in the order of generation time. Thereby, for example, when the user views the image 583b, the user can easily grasp the time zone when the pie chart was explained in the meeting, and thus can easily find the images 582c and 582d showing the text data displayed in parallel with this image 583b.

[0252] Furthermore, according to this embodiment, as shown in FIGS. 46 to 51, the display control unit 54 (an example of display control means) of the PC 5 (an example of a display terminal) includes a reproduction position specifying image 581p (an example of a first image), and displays a seek bar (an example of a second image) capable of changing the reproduction position during the entire reproduction time of the audio data on the display 508 (an example of a display unit). Thereby, the PC 5 can easily let the user grasp which reproduction position of the audio data is being reproduced out of the entire reproduction time by moving the reproduction position specifying image 581p in correspondence with the movement of the image related to the text data.

[0253] <<Supplementary Note>> Each function of the embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" in this specification includes a processor programmed to execute each function by software like a processor implemented by an electronic circuit, an ASIC (Application Specific Integrated Circuit) designed to execute each function described above, a DSP (digital signal processor), an FPGA (field programmable gate array), a SOC (System on a chip), a GPU (Graphics Processing Unit), and devices such as conventional circuit modules.

Explanation of Reference Numerals

[0254] 1 Shared System 2 Electronic Blackboard (An Example of Communication Terminal) 3 Video Conference Terminal (An Example of Communication Terminal) 4 Car Navigation Device (An Example of Communication Terminal) 5 PC (An Example of Display Terminal) 6 Shared Support Server 8 Schedule Management Server (An Example of Management System) 9 Speech-Text Conversion Server 10 Communication Network 21 Transmission / Reception Unit 22 Reception Unit 24 Display Control Unit 29 Memory / Reading Processing Unit 51 Transmission / Reception Unit (An Example of Reception Means) 52 Reception Unit (An Example of Reception Means) 54 Display Control Unit (An Example of Display Control Means) 56 Generation Unit (An Example of Generation Means) 58 Audio Control Unit (An Example of Reproduction Means) 508 Display (An Example of Display Unit) 581p Reproduction Location Specifying Image (An Example of the First Image) 81 Transmission / Reception Unit (An Example of Transmission Means)

Prior Art Documents

Patent Documents

[0255]

Patent Document 1

Patent Document 2

Patent Document 3

Claims

1. A display terminal communicable with a management system that manages execution event history information including execution event identification information for identifying an execution event participated by a user, a storage destination of sound data generated in the execution event and text data converted from the sound data, and time information regarding the sound data and the text data, wherein: transmission means for transmitting, from the display terminal to the management system, execution event identification information of an execution event selected by the user on the display terminal from among one or more execution events participated by the user specified based on user identification information for identifying the user of the display terminal, so that the management system can receive, based on the execution event history information associated with the execution event identification information of the selected execution event, the text data and the time information regarding the text data; display control means for causing the display unit to display the received text data in chronological order based on the time information, and for causing the display unit to display a first image indicating a reproduction position in the entire reproduction time of the sound data at a first position set as the reproduction position of the sound data in a display area of the display unit; reception means for receiving a selection for predetermined text data among the displayed text data; wherein the display control means causes the display unit to display the first image at a second position corresponding to the time indicated by the time information corresponding to the predetermined text data for which the selection has been received, starting from the first position; the reception means is a display terminal that receives the sound data from the management system.

2. The display terminal according to claim 1, further comprising: generation means for generating a display screen including the text data and the first image; wherein the display control means causes the display unit to display the generated display screen.

3. The display terminal according to claim 2, wherein: the display control means causes the display unit to display the text data in a specific display area of the display screen; the reception means receives a selection for the predetermined text data indicated in the specific display area; the display control means causes the display unit to display the first image at a position corresponding to the time indicated by the time information corresponding to the predetermined text data.

4. The display terminal according to any one of claims 1 to 3, wherein: A playback means for playing the sound data related to the time indicated by the time information corresponding to the predetermined text data is provided. The display control means is a display terminal that displays the playback position of the sound data being played as the first image.

5. The playback means plays the time indicated by the time information corresponding to the predetermined text data as the start time of the sound data. The display terminal according to claim 4, characterized in that.

6. The display control means causes the display unit to display a second image including the first image and indicating the playback history of the sound data. The display terminal according to any one of claims 1 to 5.

7. The second image is a seek bar capable of changing the playback position during the entire playback time of the sound data. The display terminal according to claim 6.

8. An execution event identification information for identifying an execution event participated by a user, a storage destination of sound data generated in the execution event and text data converted from the sound data, and time information regarding the sound data and the text data. A management system for managing execution event history information including, A display terminal capable of communicating with the management system, a shared system comprising: The management system is By transmitting the execution event identification information of the execution event selected by the user on the display terminal from among one or more execution events participated by the user specified based on the user identification information for identifying the user of the display terminal from the display terminal to the management system, based on the execution event history information associated with the execution event identification information of the selected execution event in the management system, the text data, and the time information regarding the text data are transmitted to the display terminal. It has a transmission means for The display terminal is A receiving means for receiving the sound data, the text data, and the time information from the management system, The received text data is displayed on the display unit in chronological order based on the time information, and a first image indicating the playback position during the entire playback time of the sound data is displayed at a first position set as the playback position of the sound data in the display area of the display unit. Display control means for Among the displayed text data, a receiving means for accepting a selection for predetermined text data is provided. A shared system in which the display control means displays the first image at a second position of the time indicated by the time information corresponding to the predetermined text data for which the selection has been received, from the first position.

9. A display control method executed by a display terminal capable of communicating with a management system that manages execution event history information including execution event identification information for identifying an execution event in which a user participated, a storage destination of sound data generated in the execution event, text data converted from the sound data, and time information regarding the sound data and the text data, By transmitting, from the display terminal to the management system, the execution event identification information of the execution event selected by the user on the display terminal from among one or more execution events in which the user identified based on user identification information for identifying the user of the display terminal participated, the text data, and the time information regarding the text data, identified based on the execution event history information associated with the execution event identification information of the selected execution event in the management system, a receiving step of receiving from the management system; A first display control step of causing the display unit to display the received text data in chronological order based on the time information, and displaying a first image indicating a reproduction position in the entire reproduction time of the sound data at a first position set as the reproduction position of the sound data in a display area of the display unit; A reception step of receiving a selection for predetermined text data among the displayed text data; A second display control step of displaying the first image at a second position of the time indicated by the time information corresponding to the predetermined text data for which the selection has been received, from the first position; is executed, The receiving step is a display control method for receiving the sound data from the management system.

10. A program executed by a display terminal capable of communicating with a management system that manages execution event history information including execution event identification information for identifying an execution event in which a user participated, a storage destination of sound data generated in the execution event, text data converted from the sound data, and time information regarding the sound data and the text data, causing a computer to By transmitting, from the display terminal to the management system, the execution event identification information of the execution event selected by the user on the display terminal from among one or more execution events participated in by the user identified based on the user identification information for identifying the user of the display terminal, the text data and the time information regarding the text data, which are specified based on the execution event history information associated with the execution event identification information of the selected execution event in the management system, a receiving step of receiving from the management system; A first display control step of causing the received text data to be displayed on a display unit in chronological order based on the time information, and causing a first image indicating a reproduction position in the total reproduction time of the sound data to be displayed at a first position set as the reproduction position of the sound data in a display area of the display unit; A receiving step of receiving a selection for predetermined text data among the displayed text data; A second display control step of causing the first image to be displayed at a second position of the time indicated by the time information corresponding to the predetermined text data for which the selection has been received, from the first position; To execute; The receiving step is a program for receiving the sound data from the management system.

Images