Remote conference system, output image control method, and output image control program

Abstract

To provide an output image control program for improving convenience for participants of a conference.SOLUTION: A remote conference system for remotely implementing a conference using a plurality of operation devices that are individually operated by a plurality of participants comprises: a first image control unit 255 for outputting a first image shared by the plurality of participants so that the first image is displayed on each of the plurality of operation devices; an output destination determination unit 265 for determining an output destination on the basis of a state of a request device of the plurality of operation devices that is operated by a request person; and a second image control unit 267 for outputting a second image differing from the first image so that the second image is output from the output destination determined by the output destination determination unit 265 in a state in which the first image is displayed on the request device.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a remote conference system, an output image control method, and an output image control program, and in particular to a remote conference system that remotely holds a conference using multiple operating devices operated by multiple participants, an output image control method executed in the remote conference system, and an output image control program that causes a computer to execute the output image control method. [Background technology]

[0002] In recent years, remote conference systems have become popular, allowing multiple participants in different locations to hold a conference via computer. In these remote conference systems, when one of the participants gives a presentation, the image of the page used by the presenter for the presentation is displayed on the computers operated by all the participants. In this case, participants may wish to view an image different from the page used by the presenter in addition to the page used for the presentation.

[0003] Japanese Patent Application Laid-Open No. 9-101767 describes a terminal device connected via a network that switches between a state in which a page selected by a participant from among the conference materials is displayed, and a synchronized state in which the same page as the conference materials being viewed by another participant selected by the participant, such as a presenter, is displayed.

[0004] However, the technology described in JP-A-9-101767 cannot simultaneously display the page being viewed by the presenter and the page selected by the participant. Furthermore, participants must select a page to display from thumbnails of multiple pages, making the selection difficult. This creates a problem in that it is difficult to display the appropriate page in accordance with the progress of the conference. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 9-101767 Summary of the Invention [Problem to be solved by the invention]

[0006] An object of the present invention is to provide a remote conference system that improves convenience for conference participants.

[0007] Another object of the present invention is to provide an output image control method that improves convenience for conference participants.

[0008] It is still another object of the present invention to provide an output image control program that improves convenience for conference participants. [Means for solving the problem]

[0009] According to one aspect of the present invention, a remote conference system is a remote conference system that remotely holds a conference using a plurality of operation devices operated by a plurality of participants, and includes a first image control means that outputs a first image shared by the plurality of participants so that the first image is displayed on each of the plurality of operation devices, an output destination determination means that determines an output destination based on the state of a requesting device operated by a requester among the plurality of operation devices, and a second image control means that outputs a second image such that a second image different from the first image is output from the output destination determined by the output destination determination means while the first image is displayed on the requesting device.

[0010] According to this aspect, a first image shared by multiple participants is displayed on each of multiple operation devices, and while the first image is displayed on the requesting device operated by the requester, a second image, separate from the first image, is output from an output destination determined based on the status of the requesting device. Thus, the first image shared by multiple participants and the second image are output to the requesting device. Furthermore, because the second image is output from an output destination determined based on the status of the requesting device, the requester can view both the first image and the second image. As a result, a remote conference system that improves convenience for conference participants can be provided.

[0011] Preferably, when the requesting device includes a plurality of display devices, the output destination determination means determines, as the output destination, a second display device among the plurality of display devices that is different from the first display device on which the first image is displayed.

[0012] According to this aspect, the participant operating the requesting device can simultaneously view the first image displayed on the first display device and the second image displayed on the second display device.

[0013] Preferably, when the requesting apparatus is connected to an image forming apparatus, the output destination determining unit determines the image forming apparatus as the output destination.

[0014] According to this aspect, a participant operating a requesting device can simultaneously view the first image displayed on the requesting device and the second image formed by the image forming device.

[0015] According to another aspect of the present invention, there is provided a remote conference system for remotely holding a conference using a plurality of operation devices each operated by a plurality of participants, the system comprising: a first image control means for outputting a first image shared by the plurality of participants so that the first image is displayed on each of the plurality of operation devices; an image determination means for determining a second image based on the state of the conference; and a second image control means for outputting the second image so that the second image is output by a requesting device operated by a requester among the plurality of operation devices when the first image is displayed on the requesting device.

[0016] According to this aspect, a first image shared by multiple participants and a second image determined based on the conference status are output to the requesting device, thereby providing a remote conference system that improves convenience for conference participants.

[0017] Preferably, the image determining means determines the second image from among images associated with the first image.

[0018] According to this aspect, a second image related to a first image is output, so that the second image necessary for understanding the content of the first image can be output.

[0019] Preferably, the image determining means determines an image identified by a reserved word included in the first image as the second image.

[0020] According to this aspect, the second image is identified by the reserved word included in the first image, so that the second image can be easily identified.

[0021] Preferably, the image determining means determines the second image from among a plurality of images previously displayed as the first image.

[0022] According to this aspect, the second image is determined from among a plurality of images displayed in the past, so that the shared first image and the previously displayed image can be viewed, thereby allowing the two images to be compared.

[0023] Preferably, the image determining means determines the second image based on the time when the first image was previously displayed.

[0024] According to this aspect, the second image is determined based on the time when the first image was displayed, so that the second image can be easily determined from multiple images. Also, it becomes possible to view the shared first image and images displayed in the past.

[0025] Preferably, the image determination means determines, as the second image, an image that was displayed as the first image while a predetermined participant was speaking, from among a plurality of images that were previously displayed as the first image.

[0026] According to this aspect, the second image is determined based on the participant's speech, so the second image can be easily determined from multiple images. Also, since the first image and an image associated with a specific participant can be viewed, the two images can be compared.

[0027] Preferably, the device further includes an output destination determination means for determining an output destination based on the state of the requesting device, and the second image control means outputs the second image so that the second image is output from the output destination determined by the output destination determination means.

[0028] According to this aspect, the second image is output from an output destination determined based on the state of the requesting device, allowing the requester to view the first image and the second image.

[0029] Preferably, when the requesting device includes a plurality of display devices, the output destination determination means determines, as the output destination, a second display device among the plurality of display devices that is different from the first display device on which the first image is displayed.

[0030] According to this aspect, the participant operating the requesting device can simultaneously view the first image displayed on the display device and the second image formed by the image forming device.

[0031] Preferably, when the requesting apparatus is connected to an image forming apparatus, the output destination determining unit determines the image forming apparatus as the output destination.

[0032] According to this aspect, a participant operating a requesting device can simultaneously view the first image displayed on the requesting device and the second image formed by the image forming device.

[0033] According to yet another aspect of the present invention, an output image control method is an output image control method executed in a remote conference system that remotely holds a conference using a plurality of operation devices each operated by a plurality of participants, and includes a first image control step of outputting a first image shared by the plurality of participants so that the first image is displayed on each of the plurality of operation devices; an output destination determination step of determining an output destination based on the state of a requesting device operated by a requester among the plurality of operation devices; and a second image control step of outputting a second image so that a second image different from the first image is output from the output destination determined in the output destination determination step while the first image is displayed on the requesting device.

[0034] According to this aspect, it is possible to provide an output image control method that improves convenience for conference participants.

[0035] According to yet another aspect of the present invention, an output image control method is an output image control method executed in a remote conference system that remotely holds a conference using a plurality of operation devices each operated by a plurality of participants, and includes a first image control step of outputting a first image shared by the plurality of participants so that the first image is displayed on each of the plurality of operation devices, an image determination step of determining a second image based on the state of the conference, and a second image control step of outputting the second image so that the second image is output by a requesting device operated by a requester among the plurality of operation devices while the first image is displayed on the requesting device.

[0036] According to this aspect, it is possible to provide an output image control method that improves convenience for conference participants.

[0037] According to yet another aspect of the present invention, an output image control program is an output image control program executed by a computer that controls a remote conference system that remotely holds a conference using multiple operation devices each operated by multiple participants, and causes the computer to execute a first image control step of outputting a first image shared by the multiple participants so that the first image is displayed on each of the multiple operation devices, an output destination determination step of determining an output destination based on the state of a requesting device operated by a requester among the multiple operation devices, and a second image control step of outputting a second image so that a second image different from the first image is output from the output destination determined in the output destination determination step while the first image is displayed on the requesting device.

[0038] According to this aspect, it is possible to provide an output image control program that improves convenience for conference participants.

[0039] According to yet another aspect of the present invention, an output image control program is an output image control program executed by a computer that controls a remote conference system that remotely holds a conference using multiple operation devices each operated by multiple participants, and causes the computer to execute a first image control step of outputting a first image so that a first image shared by the multiple participants is displayed on each of the multiple operation devices, an image determination step of determining a second image based on the state of the conference, and a second image control step of outputting the second image so that the second image is output by a requesting device operated by a requester among the multiple operation devices while the first image is displayed on the requesting device.

[0040] According to this aspect, it is possible to provide an output image control program that improves convenience for conference participants. [Brief explanation of the drawings]

[0041] [Figure 1] 1 is a diagram illustrating an example of a system configuration of a remote conference system according to an embodiment of the present invention. [Figure 2]FIG. 2 is a block diagram illustrating an example of a hardware configuration of a server. [Figure 3] FIG. 2 is a block diagram showing an example of the hardware configuration of a PC. [Figure 4] FIG. 2 is a diagram illustrating an example of functions of a CPU included in a server according to the present embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of a first association table. [Figure 6] FIG. 10 is a diagram illustrating an example of a second association table. [Figure 7] FIG. 10 is a diagram illustrating an example of a speaker table. [Figure 8] FIG. 10 is a diagram illustrating an example of a display time table. [Figure 9] FIG. 10 is a diagram illustrating an example of a request setting screen. [Figure 10] FIG. 10 is a diagram illustrating an example of a display time setting screen. [Figure 11] FIG. 10 is a diagram illustrating an example of a participant selection screen. [Figure 12] FIG. 1 is a first diagram showing an example of a display state. [Figure 13] FIG. 2 is a second diagram showing an example of a display state. [Figure 14] FIG. 10 is a diagram illustrating an example of an output state. [Figure 15] 10 is a flowchart showing an example of the flow of an output image control process. [Figure 16] 10 is a flowchart illustrating an example of the flow of a requested image determination process. [Figure 17] 10 is a flowchart illustrating an example of the flow of an output destination determination process. DETAILED DESCRIPTION OF THE INVENTION

[0042] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same components are designated by the same reference numerals. The names and functions of these components are also the same. Therefore, detailed description thereof will not be repeated.

[0043] Fig. 1 is a diagram showing an example of a system configuration of a remote conference system according to one embodiment of the present invention. Referring to Fig. 1, remote conference system 1 includes server 200 and personal computers (hereinafter referred to as "PCs") 100-1, 100-2, 100-3, 100-4 to 100-N, where N is a positive integer greater than or equal to 5. Server 200 and PCs 100-1 to 100-N are connected to Internet 5 and are capable of communicating with each other.

[0044] Each of PC100-1 to PC100-N is equipped with a camera, a microphone for collecting sound, and a speaker for outputting sound. Each of PC100-1 to PC100-N is a general computer, and since their main hardware configurations and functions are the same, PC100-1 to PC100-N will be collectively referred to as PC100 here, and the description will focus on PC100 unless otherwise specified.

[0045] Instead of the PCs 100-1 to 100-N, an information communication device such as a PDA (Personal Digital Assistance) or a smartphone may be used as long as it has a camera, a microphone, a speaker, and a communication function.

[0046] PC 100-1 includes two display devices. The display devices are liquid crystal displays (LCDs) or organic electroluminescence (EL) panels. PC 100-2 and PC 100-3 are connected to MFPs (Multi Function Peripherals) 300-2 and 300-3, respectively. MFPs 300-2 and 300-3 are examples of image forming devices. Note that image forming devices such as laser printers, inkjet printers, and facsimile machines may be used instead of MFPs 300-2 and 300-3.

[0047] Furthermore, the network to which server 200 and PCs 100-1 to 100-N are connected is not limited to Internet 5, and may be any other network as long as server 200 and PCs 100-1 to 100-N can communicate with each other. As the network, for example, a local area network (LAN) or a wide area network (WAN) is used.

[0048] In the remote conference system 1 of this embodiment, participants participate in a conference by operating one of PCs 100-1 to 100-N. In the remote conference system 1 of this embodiment, a case where a presenter is included among the multiple participants will be described as an example. The presenter gives a presentation to the other participants using conference materials consisting of multiple pages. Therefore, pages selected by the presenter from the presentation materials are displayed on the PCs operated by all the other participants, and the presenter's voice is output from the PCs operated by all the other participants. Here, a case where the participant operating PC 100-N is the presenter will be described as an example.

[0049] A program for participating in a conference is installed on each of PCs 100-1 to 100-N, and the conference is held by each of PCs 100-1 to 100-N communicating with server 200. The program installed on each of PCs 100-1 to 100-N may be a dedicated program for communicating with server 200, or may be a general browser program if server 200 provides a web service.

[0050] A remote conference system is realized by executing a remote conference program by server 200. Server 200 communicates with PCs 100-1 to 100-N, and transmits data received from each of PCs 100-1 to 100-N to each of the other PCs 100-1 to 100-N.

[0051] Data transmitted and received between each of PCs 100-1 to 100-N and server 200 includes audio information indicating audio, video information indicating video, and application data. The data transmitted and received between each of PCs 100-1 to 100-N and server 200 may be compressed data or may not be compressed data.

[0052] Server 200 controls the data to be transmitted to each of PCs 100-1 to N. For example, server 200 transmits audio data received from each of PCs 100-1 to N to all of PCs 100-1 to N. Server 200 also aggregates video data received from each of PCs 100-1 to N and transmits the video information to each of PCs 100-1 to N.

[0053] Server 200 determines and transmits the video and application data to be transmitted in response to a request from each of PCs 100-1 to 100-N. Therefore, the images displayed on PCs 100-1 to 100-N may be the same or different. Each of PCs 100-1 to 100-N may process and display the video information received from server 200. In this case, the processing of processing the video information by server 200 is not required, and the load on server 200 is reduced.

[0054] Although the remote conference system 1 in this embodiment will be described as including the server 200, it may not include the server 200. In this case, each of the PCs 100-1 to 100-N has the functions of the server 200 described below.

[0055] Fig. 2 is a block diagram showing an example of a hardware configuration of a server. Referring to Fig. 2, server 200 is a computer that performs arithmetic processing and includes a central processing unit (CPU) 201 for controlling the entire server 200, a ROM (Read Only Memory) 202 for storing programs to be executed by CPU 201, a RAM (Random Access Memory) 203 used as a work area for CPU 201, an HDD 204 for storing data in a non-volatile manner, a communication unit 205 that connects CPU 201 to the Internet 5, a display unit 206 that displays images, an operation unit 207 that accepts operation inputs, and an external storage device 210, each of which is connected to a bus 213.

[0056] Communication unit 205 is an interface for connecting server 200 to Internet 5. Therefore, CPU 201 can communicate with PCs 100-1 to 100-N connected to Internet 5 via communication unit 205.

[0057] The external storage device 210 is equipped with a CD-ROM (Compact Disk Read Only Memory) 211. The CPU 201 controls the external storage device 210 to read data stored in the CD-ROM 211.

[0058] In this embodiment, CPU 201 executes a program stored in ROM 202 or HDD 204. CPU 201 may also control external storage device 210 to read a program to be executed by CPU 201 from CD-ROM 211, store the read program in RAM 203, and execute it.

[0059] Furthermore, the CPU 201 downloads a program from a computer connected to the Internet 5 and stores it in the HDD 204. When a computer connected to the Internet 5 writes a program to the HDD 204, the program is stored in the HDD 204. The CPU 201 may load the program stored in the HDD 204 into the RAM 203 and execute it.

[0060] The recording medium for storing the program to be executed by CPU 201 is not limited to CD-ROM 211, but may be a flexible disk, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile Disc)), IC card, optical card, mask ROM, EPROM (Erasable Programmable ROM) or other semiconductor memory. The program here includes not only a program that can be directly executed by CPU 201, but also a source program, a compressed program, an encrypted program, etc.

[0061] Fig. 3 is a block diagram showing an example of the hardware configuration of a PC. Referring to Fig. 3, PC 100 is a computer that performs arithmetic processing and includes a CPU 101 for controlling the entire PC 100, a ROM 102 for storing programs executed by CPU 101, a RAM 103 used as a work area for CPU 101, an HDD 104 for non-volatilely storing data, a communication unit 105 that connects CPU 101 to Internet 5, a display unit 106 for displaying images, an operation unit 107 that accepts operation inputs from participants who are users, a camera 108 that captures images of participants, a speaker 109 that outputs audio, a microphone 110 that collects the audio of the operator, and an external storage device 111, all of which are connected to a bus 112.

[0062] A CD-ROM 111A is attached to the external storage device 111. The CPU 101 controls the external storage device 111 to read data stored on the CD-ROM 111A.

[0063] A module in which at least two of the camera 108, the speaker 109, and the microphone 110 are integrated may be connected to the PC 100. The module includes, for example, a headset in which the speaker 109 and the microphone 110 are integrated.

[0064] Fig. 4 is a diagram showing an example of functions of a CPU included in a server according to the present embodiment. The functions shown in Fig. 4 are realized by CPU 201 included in server 200 as CPU 201 executes an output image control program stored in ROM 202, HDD 204, or CD-ROM 211. Referring to Fig. 4, CPU 201 includes participant information acquisition unit 251, conference material acquisition unit 253, first image control unit 255, request reception unit 257, conference state acquisition unit 259, image determination unit 261, device information acquisition unit 263, output destination determination unit 265, and second image control unit 267.

[0065] Participant information acquisition unit 251 acquires information about participants attending a conference. Participants participate in a conference by operating one of PCs 100-1 to 100-N. If a participant inputs a user ID as identification information for identifying the participant when joining a conference, the participant's user ID is acquired from each of PCs 100-1 to 100-N. When a user ID is received from one of PCs 100-1 to 100-N, participant information acquisition unit 251 associates the user ID with device identification information for identifying the device from PCs 100-1 to 100-N that sent the user ID. Specifically, participant information acquisition unit 251 generates a participant record including the user ID and the device identification information, and stores a participant table including a plurality of participant records in HDD 204.

[0066] Conference material acquisition unit 253 acquires data of materials used in a conference as presentation data. Specifically, when conference material acquisition unit 253 receives presentation data from a PC operated by a presenter among PCs 100-1 to 100-N, it stores the presentation data in HDD 204. The presentation data is application data or image data. Here, the presentation data includes multiple pages.

[0067] First image control unit 255 transmits an image of a page (hereinafter referred to as a "presentation page") selected by the presenter from the presentation data to each of PCs 100-1 to 100-N as a first image. As a result, the first image is output on each of PCs 100-1 to 100-N. Typically, the first image is displayed on a display device provided in each of PCs 100-1 to 100-N. As a result, the first image is shared by all of the participants. Specifically, first image control unit 255 controls communication unit 205 to transmit the first image to each of PCs 100-1 to 100-N in response to receiving an instruction to select a presentation page from PC 100-N operated by the presenter. First image control unit 255 transmits the first image to each of PCs 100-1 to 100-N every time an instruction to select a presentation page is received from PC 100-N operated by the presenter. Therefore, as the presenter switches presentation pages, the image of the presentation page switched by the presenter is displayed as the first image on each of PCs 100-1 to 100-N. In addition, first image control unit 255 may transmit presentation data to each of PCs 100-1 to 100-N in advance, and transmit a command to each of PCs 100-1 to 100-N to display the first image each time an instruction to select a presentation page is received from PC 100-N operated by the presenter.

[0068] Conference status acquisition unit 259 acquires the conference status. The conference status includes presentation page status information and participant status information. Presentation page status information is information indicating the status in which a presentation page is displayed, and includes page identification information for identifying the presentation page and the time the presentation page is displayed. Participant status information is information indicating the status in which a participant has spoken, and includes participant identification information for identifying the participant and the time the participant has spoken.

[0069] Conference state acquisition unit 259 determines the time when the first image determined for the presentation page by first image control unit 255 is transmitted as the display start time, and determines the time when the next first image determined for the presentation page is transmitted by first image control unit 255 as the display end time. Each time the presentation page is switched, conference state acquisition unit 259 generates a page record that associates the display start time when the display of the presentation page started with the display end time when the display of the presentation page ended. Conference state acquisition unit 259 adds the page record to a presentation page table stored in HDD 204.

[0070] Conference state acquisition unit 259 acquires audio information indicating the audio of a plurality of participants from each of PCs 100-1 to 100-N operated by the participants. Each of PCs 100-1 to 100-N transmits to server 200 audio information obtained by converting audio collected by microphone 110 into a digital signal and video information obtained by converting video of the participant's face captured by camera 108 into a digital signal. Conference state acquisition unit 259 stores the audio information and video information received by communication unit 205 from each of PCs 100-1 to 100-N in HDD 204 in association with the participant. The audio information and video information may be stored in association with PCs 100-1 to 100-N, or may be stored in association with the participant operating PCs 100-1 to 100-N. When the participant operates one of PCs 100-1 to 100-N to log in to the remote conference system, the participant is authenticated and identified by server 200.

[0071] The conference state acquisition unit 259 identifies the participant identification information of the participants based on the audio information received from any of the PCs 100-1 to 100-N, calculates the speech start time, speech end time, and speech duration, and generates an audio record that associates these. The conference state acquisition unit 259 adds the audio record to an audio table stored in the HDD 204.

[0072] Request receiving unit 257 receives a request from any one of the multiple participants. The request here includes a request to display a second image different from the first image. Request receiving unit 257 controls communication unit 205 to receive the participant's request when a signal indicating the participant's request is received from any one of PCs 100-1 to 100-N. In response to communication unit 205 receiving the signal indicating the participant's request, request receiving unit 257 outputs a determination instruction to image determination unit 261 and output destination determination unit 265.

[0073] The image determination unit 261 determines the second image in response to a determination instruction input from the request receiving unit 257. The image determination unit 261 includes a related image determination unit 271 and a display history determination unit 273. The related image determination unit 271 determines the second image from among images related to the first image. The related image determination unit 271 determines an image identified by a reserved word included in the first image as an image related to the first image. The reserved word is a demonstrative word that indicates image identification information in combination with image identification information for identifying other images, or the image identification information itself. The demonstrative word is, for example, a verb such as "reference," "show," or "represent." The image identification information includes a file name and a page number. The image identification information also includes names such as figure numbers, table numbers, and graph numbers assigned to figures, tables, graphs, etc. The image identification information may also include a network address indicating a location on a network, such as a uniform resource locator (URL).

[0074] The related image determination unit 271 extracts a reserved word from the first image and determines the image identified by the reserved word as the second image. Specifically, when the related image determination unit 271 extracts a reference word from the first image, it identifies image identification information indicated by the reference word and determines the image of the page identified by the image identification information as the second image. For example, if the first image contains the character string "See page 2," the related image determination unit 271 extracts the reference word "See" from the first image and determines "page 2" as the image identification information identified by the reference word. For example, if the first image contains the character string "Figure 2," the related image determination unit 271 determines "Figure 2" as the image identification information from the first image. Here, a case will be described in which the image identification information identifies an image included in presentation data that includes the first image. If the image identification information identifies an image included in data other than the presentation data, the related image determination unit 271 searches data stored in a predetermined range to determine the second image. For example, if there are multiple presentation data, the related image determination unit 271 determines the folder in which the multiple presentation data are stored as the search target. Also, the image identification information may be a network address such as a URL indicating a location on the Internet. The related image determination unit 271 determines the image of the web page identified by the URL as the second image.

[0075] The related image determination unit 271 may create an association table when it receives presentation data. The related image determination unit 271 selects presentation data as the processing target, starting from the first page, and extracts reserved words from the selected pages. If a reserved word is extracted, the related image determination unit 271 determines the page identified by the extracted reserved word. Then, the related image determination unit 271 generates an association record that associates the page identification information of the processing target page with the page identified by the reserved word, and adds the record to the association table stored in the HDD 204.

[0076] FIG. 5 is a diagram showing an example of the first association table. The first association table is one of the association tables, and includes a referencing page field and a referenced page field. The referencing page field is set with page identification information of the page that the related image determination unit 271 has targeted for processing, and the related destination page field is set with page identification information of the page that the related image determination unit 271 has identified using a reserved word. For example, in the first record, page 5 is set in the referencing page field, and page 10 is set in the referenced page field. This indicates that page 10 is identified by the reserved word included in page 5, and that page 10 is associated with page 5.

[0077] FIG. 6 is a diagram showing an example of the second association table. The second association table is one of the association tables and includes a referencing page field, a reference figure / table number field, and a figure number page field. The page identification information of the page processed by the related image determination unit 271 is set, the reference figure / table number field is set to image identification information, and the figure number page field is set to page identification information of the page containing the image identified by the image identification information. For example, in the first record, the referencing page field is set to page 2, the reference figure / table number field is set to "Figure 3," and the figure number page field is set to page 5. This identifies page 5, which contains the image identification information "Figure 3," a reserved word included in page 2, and indicates that page 5 is associated with page 2.

[0078] Returning to FIG. 4, display history determination unit 273 determines a second image from among a plurality of images previously transmitted by first image control unit 255 to PCs 100-1 to 100-N as first images. The plurality of images previously transmitted by first image control unit 255 to PCs 100-1 to 100-N as first images are displayed on each of PCs 100-1 to 100-N. Each of the plurality of images previously transmitted by first image control unit 255 as a first image is referred to as a "displayed image." Display history determination unit 273 determines a second image from among a plurality of displayed images based on the time it has been displayed. For example, display history determination unit 273 determines the displayed image that has been displayed for the longest time as the second image. The time for which a plurality of displayed images have been displayed is calculated by referring to a presentation page table stored in HDD 204.

[0079] The display history determination unit 273 determines, from among a plurality of displayed images, a displayed image that was displayed while a predetermined participant was speaking as the second image. For example, the display history determination unit 273 determines, as the second image, a displayed image that was displayed while participant A was speaking. The duration that participant A was speaking is calculated by referring to the voice table stored in the HDD 204.

[0080] The display history determination unit 273 may create a speaker table and a display time table while the conference is in progress. Each time a presentation page is displayed, the display history determination unit 273 generates a speaker record and a display time record and adds them to the speaker table and the display time table stored in the HDD 204, respectively.

[0081] FIG. 7 is a diagram showing an example of a speaker table. Referring to FIG. 7, the speaker table includes a speaker field and a comment page field. The comment page field contains page identification information for identifying the presentation page, and the speaker field contains participant identification information of a participant who spoke while the presentation page was displayed. For example, in the first record, "Sato" is set in the speaker field and "1" is set in the comment page field. This indicates that a participant with user identification information "Sato" spoke while the first page was displayed as the presentation page.

[0082] FIG. 8 is a diagram showing an example of a display time table. Referring to FIG. 8, the display time table includes a presentation page field, a start time field, an end time field, and a display time (seconds) field. The presentation page field contains page identification information for the presentation page, the start time field contains the time when the presentation page started to be displayed, the end time field contains the time when the presentation page stopped to be displayed, and the display time (seconds) field contains the time from the start time to the end time. For example, in a first record, "1" is set in the presentation page field, "10:00:00" is set in the start time field, "10:00:20" is set in the end time field, and "20" is set in the display time (seconds) field. This indicates that the first page was displayed as the presentation page for 20 seconds, from 10:10:00 to 10:10:20.

[0083] The request received by the request receiving unit 257 may include a related page designation, a time designation for displaying an image with a long display time, and a comment page designation including user identification information for identifying a specific user. In this case, the request received by the request receiving unit 257 may include a user designation. The user designation includes a related page designation for displaying a related image, a time designation for displaying an image with a long display time, and a comment page designation for specifying a specific user. The request receiving unit 257 outputs a determination instruction including the user designation to the image determination unit 261. For example, if the determination instruction includes a related page designation for displaying a related image, the related image determination unit 271 determines the second image. If the determination instruction includes a time designation for displaying an image with a long display time, the display history determination unit 273 determines the second image based on the display time. If the determination instruction includes a comment page designation including user identification information for identifying a specific user, the display history determination unit 273 determines the second image displayed while the user was making a comment.

[0084] FIG. 9 is a diagram showing an example of a request setting screen. The request setting screen is displayed by inputting a request instruction into PCs 100-1 to 100-N operated by multiple participants in the conference. FIG. 9 shows a state in which the first image is displayed and the request setting screen is displayed superimposed on a part of the first image. The request setting screen includes four selection menus. When an option indicating the words "related page" is selected by a participant, a signal indicating a request including a related page designation is transmitted to server 200. When request receiving unit 257 receives a signal indicating a request including a related page designation from any of PCs 100-1 to 100-N, it outputs a determination instruction including a related page designation to image determination unit 261.

[0085] When a participant selects an option displaying the words "required time page," a signal indicating a request including a time specification is transmitted to server 200. When request receiving unit 257 receives a signal indicating a request including a time specification from any of PCs 100-1 to 100-N, request receiving unit 257 outputs a determination instruction including a time specification to image determination unit 261. In this case, the time specification may be such that the time to be displayed can be specified.

[0086] When a participant selects an option marked with the words "comment page," a participant list is displayed, and when the participant selects one of the participants, a signal indicating a request including a comment page designation is sent to the server 200. The comment list includes participant identification information for all participants in the conference or all participants who have spoken. The comment page designation includes the participant identification information of the speaker selected by the participant. When the request receiving unit 257 receives a signal indicating a request including a comment page designation from any of the PCs 100-1 to 100-N, it outputs a determination instruction including a comment page designation to the image determination unit 261.

[0087] When a participant selects an option displaying the words "another page," an area for specifying a page is displayed, and when the participant specifies a page, a signal indicating a request including page identification information of the specified page is transmitted to server 200. When request receiving unit 257 receives a signal indicating a request including page identification information from any of PCs 100-1 to 100-N, it outputs a determination instruction including last page identification information to image determination unit 261. Image determination unit 261 determines the image of the page specified by the page identification information as the second image.

[0088] 10 is a diagram showing an example of a display time setting screen. Referring to FIG. 10, the display time setting screen includes a selection menu for a plurality of display times, here 10 minutes, 8 minutes, 3 minutes, and 1 minute. When the 10-minute option is selected, a page with a display time of 10 minutes or more is determined to be the second image. When the 8-minute option is selected, a page with a display time of 8 minutes or more but less than 10 minutes is determined to be the second image. When the 3-minute option is selected, a page with a display time of 3 minutes or more but less than 8 minutes is determined to be the second image. When the 1-minute option is selected, a page with a display time of 1 minute or more is determined to be the second image.

[0089] 11 is a diagram showing an example of a participant selection screen. Referring to FIG. 11, the participant selection screen includes participant identification information “Sato”, “Suzuki”, “Tanaka”, and “Yamada” displayed in a selectable state.

[0090] Returning to Fig. 4, device information acquisition unit 263 acquires device information for each of PCs 100-1 to 100-N. The device information includes the number of display devices and whether an image forming device is connected. Here, PC 100-1 is connected to two display devices, and the other PCs 100-2 to 100-N are connected to one display device. Furthermore, PCs 100-2 and 100-3 are connected to MFPs 300-2 and 3, respectively, and the other PCs 100-1, 100-3 to 100-N are not connected to an image forming device.

[0091] Output destination determination unit 265 determines the output destination of the second image from among PCs 100-1 to 100-N. Output destination determination unit 265 determines the requesting device that has transmitted the request accepted by request acceptance unit 257 as the output destination. Here, an example will be described in which PC 100-1 is the requesting device. Furthermore, output destination determination unit 265 determines the output destination based on the device information of the requesting device. The device information of PC 100-1, which is the requesting device and is acquired by device information acquisition unit 263, indicates that two display devices are included. In this case, output destination determination unit 265 determines the second display device, which is different from the first display device on which the first image is displayed, as the output destination. Note that when one of PCs 100-2 and 100-3, for example, PC 100-2, is determined as the output destination, the device information of PC 100-2 indicates that MFP 300-2 is connected. In this case, output destination determination unit 265 determines MFP 300-2 as the output destination.

[0092] Second image control unit 267 outputs the second image determined by image determination unit 261 to the output destination determined by output destination determination unit 265. Specifically, second image control unit 267 transmits the second image to the requesting device and causes it to be output from the output destination. For example, second image control unit 267 controls communication unit 205 to output an output instruction to the requesting device. The output instruction includes the second image and an output method for the second image. When second image control unit 267 outputs the output instruction to PC 100-1, the output method is a command to display on the second display device. When second image control unit 267 outputs the output instruction to either PC 100-2 or 100-3, the output method is a command to form an image on MFP 300-2 or 3.

[0093] Fig. 12 is a first diagram showing an example of a display state. Fig. 12 shows the display state of each of PCs 100-4 to 100-N. Each of PCs 100-4 to 100-N is equipped with one display device. Therefore, a window including first image 401 and a window including second image 403 are displayed side by side on one display device.

[0094] Fig. 13 is a second diagram showing an example of a display state. Fig. 13 shows the display state of PC 100-1. PC 100-1 is equipped with two display devices. Therefore, a window including first image 401 is displayed on one display device, and a window including second image 403 is displayed on a display device other than the display device on which first image 401 is displayed.

[0095] Fig. 14 is a diagram showing an example of the output state. Fig. 14 shows the output state of PCs 100-2 and 100-3. PC 100-2 has one display device and MFP 300-2, and PC 100-3 has one display device and MFP 300-3. Therefore, a window including first image 401 is displayed on one display device, second image 403 is output to MFPs 300-2 and 3, and an image of the second image is formed on a recording medium.

[0096] 15 is a flowchart showing an example of the flow of output image control processing. The output image control processing is processing carried out by CPU 201 included in server 200 as CPU 201 executes an output image control program stored in ROM 202, HDD 204, or CD-ROM 211. Referring to FIG. 15, CPU 201 included in server 200 acquires participant information (step S01), and proceeds to step S02. CPU 201 acquires information about participants attending the conference. When communication unit 205 receives a user ID transmitted from any of PCs 100-1 to 100-N, it generates a participant record including the user ID and device identification information of the device among PCs 100-1 to 100-N that transmitted the user ID, and adds the record to a participant table stored in HDD 204.

[0097] In step S02, the conference materials are acquired, and the process proceeds to step S03. When communication unit 205 receives presentation data from one of PCs 100-1 to 100-N, the presentation data is acquired. A presenter is selected from among the multiple participants, and when the presenter inputs a command to send the presentation data to a PC operated by the presenter among PCs 100-1 to 100-N, the presentation data is sent from that PC. The presentation data received by communication unit 205 is stored in HDD 204. Here, an example will be described in which the presentation data includes multiple pages.

[0098] In step S03, it is determined whether the first image has been determined. If the first image has been determined, the process proceeds to step S04; if not, the process returns to step S03. If a presentation page of the presentation data is selected by the presenter, the image of that presentation page is determined to be the first image. In step S04, the first image is transmitted to all participants, and the process proceeds to step S05. At this stage, the first image is displayed on each of PCs 100-1 to 100-N.

[0099] In step S05, it is determined whether a request has been accepted. If communication unit 205 receives a request from any of PCs 100-1 to 100-N, it is determined that the request has been accepted. The request here includes displaying a second image different from the first image. If communication unit 205 receives a signal indicating a request from a participant from any of PCs 100-1 to 100-N, the request is accepted. If the request has been accepted, the process proceeds to step S06; if not, the process returns to step S03.

[0100] In step S06, a requested image determination process is executed, and the process proceeds to step S07. The requested image determination process will be described in detail later, but this process determines a second image that is requested to be output by one of the participants. When the requested image determination process is executed, the second image is determined.

[0101] In step S07, the device information is acquired, and the process proceeds to step S08. The device information of the device is acquired from the requesting device that has transmitted a signal indicating a request from a participant among PCs 100-1 to 100-N. Communication unit 205 is controlled to request the requesting device to transmit its device information, and when communication unit 205 receives the device information from the requesting device, the device information is acquired.

[0102] In step S08, an output destination determination process is executed, and the process proceeds to step S09. The output destination determination process will be described in detail later, but is a process for determining how the second image is to be output.

[0103] In step S09, an output instruction is sent to the requesting device, and the process proceeds to step S10. The output instruction is sent from communication unit 205 to the requesting device. The output instruction includes the second image and a command indicating how to output the second image. In step S10, it is determined whether the conference has ended. If the conference has ended, the process ends, but if not, the process returns to step S03.

[0104] 16 is a flowchart showing an example of the flow of the requested image determination process. The requested image determination process is a process executed in step S06 of the output image control process. Referring to FIG. 16, it is determined whether the request indicates a related page designation (step S21). If the request indicates a related page designation, the process proceeds to step S22, but if not, the process proceeds to step S23.

[0105] In step S22, an image related to the first image is determined as the second image, and the process returns to the output image control process. An image identified by a reserved word included in the first image is determined as the image related to the first image. The reserved word is a demonstrative word or image identification information itself. Demonstrative words are verbs such as "see," "show," and "represent." The image identification information includes a file name and a page number. The image identification information also includes names such as figure numbers, table numbers, and graph numbers assigned to figures, tables, graphs, etc., and a network address indicating a location on a network. Specifically, the reserved word is extracted from the first image, and the image identified by the reserved word is determined as the second image.

[0106] In step S23, it is determined whether the request indicates a comment page designation. If the request indicates a comment page designation, the process proceeds to step S24; if not, the process proceeds to step S25. In step S24, the displayed image displayed while the designated user was speaking is determined to be the second image, and the process returns to the output image control process. The displayed image is the image transmitted to PCs 100-1 to 100-N as the first image and displayed. The designated user is a participant identified by the user identification information included in the comment page designation indicated in the request. The audio table stored in HDD 204 is referenced to identify the time when the designated user is speaking, and the presentation page table stored in HDD 204 is referenced to identify the time when the presented image will be displayed. Then, the displayed image displayed while the designated user was speaking is identified.

[0107] In step S25, it is determined whether the request indicates a time specification. If the request indicates a time specification, the process proceeds to step S26; if not, the process proceeds to step S27. In step S26, the displayed image that has been displayed for the specified time is determined to be the second image, and the process returns to the output image control process. The presentation page table stored in HDD 204 is referenced to identify the time for which the presented image will be displayed.

[0108] In step S27, the specified page is determined to be the second image, and the process returns to the output image control process. The image of the specified page, identified by the page identification information included in the request, among the multiple pages included in the presentation data is determined to be the second image.

[0109] 17 is a flowchart showing an example of the flow of the output destination determination process. The output destination determination process is a process executed in step S08 of the output image control process. With reference to FIG. 17, device information of the requesting device is determined (step S31), and the process proceeds to step S32. The device information of each of PCs 100-1 to 100-N was acquired in step S07 of the output image control process. The device information of the requesting device among PCs 100-1 to 100-N is determined.

[0110] In step S32, it is determined whether the requesting device has multiple display devices. If the requesting device has multiple display devices, the process proceeds to step S33; if not, the process proceeds to step S35. In step S33, the first display device on which the first image is to be displayed is identified, and the process proceeds to step S34. The first display device is identified by querying the requesting device about the display device on which the first image is being displayed. In step S34, the second display device is determined as the output destination, and the process returns to the output image control process. The second display device is a display device different from the first display device among the multiple display devices included in the requesting device.

[0111] In step S35, it is determined whether the requesting device is connected to the image forming device. If the requesting device is connected to the image forming device, the process proceeds to step S36. If not, the process returns to the output image control process. In step S36, the image forming device is determined as the output destination, and the process returns to the output image control process.

[0112] Although the first display device is specified in step S33 and the second display device is determined as the output destination in step S24, a display device on which the first image is not being displayed may also be determined as the output destination. In this case, the requesting device is determined as the output destination, and a command indicating that the display device on which the first image is not being displayed is the output destination is transmitted to the requesting device.

[0113] <First Modification> In the remote conference system 1 in the above-described embodiment, the CPU 201 included in the server 200 includes the image determination unit 261 and the output destination determination unit 265. In the remote conference system 1 in the first modification, the CPU 201 included in the server 200 includes either the image determination unit 261 or the output destination determination unit 265.

[0114] When CPU 201 included in server 200 includes image determination unit 261 but does not include output destination determination unit 265, server 200 does not further include device information acquisition unit 263. In this case, server 200 transmits the second image determined by image determination unit 261 to the requesting device.

[0115] If CPU 201 included in server 200 includes output destination determination unit 265, it does not include image determination unit 261. In this case, server 200 transmits the second image specified by the participant operating the requesting device to the requesting device.

[0116] As described above, remote conference system 1 in this embodiment remotely holds a conference using PCs 100-1 to 100-N operated by multiple participants. Server 200 outputs a first image shared by the multiple participants so that it is displayed on each of PCs 100-1 to 100-N, determines an output destination based on the status of a requesting device operated by a requester among PCs 100-1 to 100-N, and outputs a second image so that a second image different from the first image is output from the determined output destination while the first image is displayed on the requesting device. Therefore, the first image shared by the multiple participants is displayed on each of PCs 100-1 to 100-N, and the second image is output from the requesting device operated by the requester while the first image is displayed. Therefore, the first image and the second image are output to the requesting device. Furthermore, since the second image is output from an output destination determined based on the status of the requesting device, the requester can view the first image and the second image.

[0117] Furthermore, in the remote conference system 1, when the PC 100-1 is the requesting device, the server 200 determines the second display device, which is different from the first display device on which the first image is displayed, as the output destination. This allows the participant operating the PC 100-1 to simultaneously view the first image displayed on the first display device and the second image displayed on the second display device.

[0118] Furthermore, in the remote conference system 1, when PC 100-2 is the requesting device, for example, server 200 determines MFP 300-2 connected to PC 100-2 as the output destination. This allows a participant operating PC 100-2 to simultaneously view the first image displayed on the display device and the second image formed on paper.

[0119] Furthermore, in remote conference system 1 according to the present embodiment, server 200 outputs a first image shared by multiple participants so that it is displayed on each of PCs 100-1 to 100-N, determines a second image based on the state of the conference, and outputs the second image so that the requesting device operated by the requester among PCs 100-1 to 100-N outputs the second image while the first image is being displayed on that device. Thus, the first image and the second image determined based on the state of the conference are output to the requesting device. This improves convenience for conference participants.

[0120] Furthermore, the server 200 determines the second image from among images related to the first image, and therefore can output the second image necessary to understand the content of the first image.

[0121] Furthermore, the server 200 determines the image identified by the reserved word included in the first image as the second image, which makes it easier to identify the second image from the first image.

[0122] The server 200 also determines the second image from among multiple images previously displayed as the first image, making it possible to view the currently shared first image and the previously displayed image, allowing the two images to be compared.

[0123] Furthermore, the server 200 determines the second image from among the multiple images displayed as the first image based on the time at which they were displayed as the first image. Therefore, since the second image is determined based on the time of display, it becomes easy to narrow down the second image from the multiple images. Furthermore, since it is possible to view the currently shared first image and an image displayed in the past, the two images can be compared.

[0124] Furthermore, the server 200 determines the image that was displayed as the first image while a specific participant was speaking from among multiple images previously displayed as the first image as the second image. Therefore, the second image is determined based on the situation in which the participant was speaking, so the second image can be easily determined from multiple images. Furthermore, participants can view the first image and images related to the specific participant.

[0125] Furthermore, the server 200 determines the output destination based on the status of the requesting device, and outputs the second image so that the second image is output from the determined output destination. Since the second image is output from the output destination determined based on the status of the requesting device, the second image can be easily determined. Furthermore, participants can view the first image and images associated with a specific participant.

[0126] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0127] 1 Remote conference system, 5 Internet, 100-1 to N PCs, 200 Server, 300-2, 3 MFP, 108 Camera, 109 Speaker, 110 Microphone, 201 CPU, 202 ROM, 203 RAM, 204 HDD, 205 Communication unit, 206 Display unit, 207 Operation unit, 210 External storage device, 211 CD-ROM, 213 Bus, 251 Participant information acquisition unit, 253 Conference material acquisition unit, 255 First image control unit, 257 Request acceptance unit, 259 Conference status acquisition unit, 261 Image determination unit, 263 Device information acquisition unit, 265 Output destination determination unit, 267 Second image control unit, 271 Related image determination unit, 273 Display history determination unit.

Claims

1. A remote conference system that remotely holds a conference using a plurality of operation devices operated by a plurality of participants, a first image control means for outputting a first image shared by the plurality of participants so that the first image is displayed on each of the plurality of operation devices; related image information creating means for creating information related to the first image; an image determination means for determining a second image by referring to the information created by the related image information creation means; a second image control means for causing the image determination means to determine the second image in response to a request operation by the requesting device when the first image is displayed on a requesting device operated by a requester among the plurality of operation devices, and for outputting the determined second image.

2. The related image information creating means creates image information associated with the first image and identified by a reserved word included in the first image, The remote conference system according to claim 1 , wherein the image determination means determines an image associated with the first image as the second image in response to a request operation by the requesting device.

3. The related image information creating means creates time information when the first image is displayed in association with the first image, 2. The remote conference system according to claim 1, wherein the image determination means determines the second image based on a time input by the requesting device.

4. The related image information creating means creates speaker information about the speaker's statement in association with the first image, 2. The remote conference system according to claim 1, wherein the image determining unit determines the second image based on the speaker input by the request device.

5. A remote conference system that remotely holds a conference using multiple operation devices operated by multiple participants, a first image control means for outputting a first image shared by the plurality of participants so that the first image is displayed on each of the plurality of operation devices; an image determination means for determining a second image based on a state of the conference; a second image control means for outputting the second image such that the second image is output by a requesting device operated by a requester among the plurality of operation devices while the first image is being displayed on the requesting device, The image determination means determines the second image based on the time when the first image was previously displayed.

6. 1. An output image control method executed in a remote conference system in which a conference is remotely held using a plurality of operation devices operated by a plurality of participants, the method comprising: a first image control step of outputting a first image shared by the plurality of participants so that the first image is displayed on each of the plurality of operation devices; an associated image information creating step of creating information associated with the first image; an image determination step of determining a second image by referring to the information created in the related image information creation step; and a second image control step of outputting the second image determined in the image determination step in response to a request operation by a requesting device operated by a requester among the plurality of operating devices while the first image is displayed on the requesting device.

7. An output image control program executed on a computer that controls a remote conference system that remotely holds a conference using a plurality of operation devices operated by a plurality of participants, the program comprising: a first image control step of outputting a first image shared by the plurality of participants so that the first image is displayed on each of the plurality of operation devices; an associated image information creating step of creating information associated with the first image; an image determination step of determining a second image by referring to the information created in the related image information creation step; and a second image control step of outputting the second image determined in the image determination step in response to a request operation by a requesting device operated by a requester among the plurality of operating devices while the first image is displayed on the requesting device.

Images