Photographing system, server, photographing method, and program

Abstract

To provide an imaging system or the like allowing for easily performing imaging by using participant images in an online conference and sharing the photographs after obtaining the consent of the participants.SOLUTION: In an imaging system 1, a server 3 captures an image from a video of each participant in an online conference on the basis of imaging instruction information from a user terminal 5, and saves the image as a captured image. The server 3 sends an approval request as to whether the imaged captured image can be shared to the participant who is the subject, and makes the captured image sharable among the participants (sharers) who has responded as sharable when the participant who is the subject responds with approval. The server manages the captured image as non-shareable when no approval is obtained from the participant who is the subject.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a photography system, a server, a terminal, a photography method, and a program, and more particularly to online photography. [Background technology]

[0002] Traditionally, when many people gather at a meeting, group photos are taken as a memento of the event and distributed or sold. However, it is difficult to take group photos if there is no opportunity to gather together. In recent years, online conference systems have been provided that enable multiple users to hold conferences while in remote locations by connecting their PCs or smartphones via a network and sending and receiving voice and images. This type of conference system is used not only in business situations but also for gatherings with friends, and is used as a tool for people in remote locations to easily gather and talk while looking at each other's faces.

[0003] In a typical online conference system, a computer terminal or a smartphone equipped with a camera, a microphone, a speaker, a display, etc. is used to capture images of participants and acquire their voices, which are then transmitted to the terminals of the other participants via a server. The terminals of the other participants receive the images (videos) of each participant and display them on a display screen, while outputting the audio of each participant from a speaker. The display screen of each terminal is provided with windows for the number of participants, and the video transmitted from each participant is displayed in each window. Also, as shown in Patent Document 1, a video conference system has been proposed that displays an image representing a virtual conference room on a display screen. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 10-056626 Summary of the Invention [Problem to be solved by the invention]

[0005] However, since online meetings do not involve meeting in person, it is difficult to take group photos. Although it is possible for participants to capture the screen (print screen) using the functions of their devices, the screen is captured as is, making the photo unimpressive and including unnecessary parts of the screen. There are also issues with it being difficult to capture the photo at the right time when people are having fun, and with the possibility of capturing moments that people do not want to be photographed.

[0006] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to provide a photography system etc. that makes it possible to easily photograph participants when gathering online and to share the photographs with the consent of the participants. [Means for solving the problem]

[0007] A first invention for solving the above-mentioned problem is a photography system in which an online conference can be held between a plurality of terminals communicatively connected to a server via a network, the server comprising: an online conference management means for managing entry and exit to the online conference, receiving video and audio transmitted from the terminal of each participant participating in the online conference and transmitting them to the terminals of the other participants; a capture means for extracting an image from the video of the subject participant and saving it as a captured image; an approval request means for sending an approval request as to whether or not the captured image can be shared to the terminal of the subject participant; and a sharing management means for making the captured image shareable with other subject participants if approval is obtained from the subject participant, and making the captured image unshareable if approval is not obtained, and the terminal comprises a response means for responding to the approval request from the server with the approval or disapproval.

[0008] According to the first invention, the video of each participant in an online conference is used to cut out (capture) an image of the participant, and when an approval response is received from the subject participant, the captured image can be shared among the participants (sharers) who responded that the captured image can be shared. This makes it possible to easily take pictures during an online conference and share photos with the consent of the participants.

[0009] In the first invention, the terminal includes a shooting instruction means for instructing to specify a participant as a subject on a screen displaying the video of the participant of the online conference and to shoot the participant, and a transmission means for transmitting identification information of the participant specified by the shooting instruction means and information on the shooting date and time to the server as shooting instruction information, and the capture means of the server cuts out an image from the video of the participant as a subject based on the shooting instruction information transmitted from the terminal and saves it as a captured image. This allows the participant to operate the terminal to specify the participant as a subject and shoot at a desired timing.

[0010] In the first aspect of the present invention, the capture means may monitor the video of each participant, and when a predetermined condition is met, cut out an image from the video of each participant and save it as a captured image. This allows the server side to determine the timing of shooting and take the image. For example, even when the conversation is lively, it is possible to take the image at a good timing without forgetting.

[0011] The server further includes an image providing means for providing the captured image made sharable by the sharing management means so that it can be used among the shared participants. The image providing means also makes the captured image available even after the online conference has ended. This allows the shared captured image to be used (displayed or saved) on each terminal with the consent of the subject. By making the image available even after the conference has ended, it can be kept as a memento or record of the conference.

[0012] It is preferable that the server includes a group photo generating means for generating a group photo by combining the captured images that have been made sharable by the sharing management means, and the image providing means provides the group photo to the participants included in the group photo. In this way, a group photo of the participants can be created and shared. Since the group photo is generated on the server side, the participants do not need to perform image processing or the like, and can easily obtain the group photo.

[0013] It is preferable that the server includes a transmission means for transmitting the capture images to the terminals of the participants during the online conference, and that the terminals include an image list display means for displaying a list of the capture images received from the server. It is also preferable that the image list display means hides capture images that are not shareable by the sharing management means. This allows the participants to check the captured images that have been taken on their own terminals. It is also possible to prevent images that are not shareable from being displayed in the image list. The image list display means displays a message indicating that capture images that are not shareable by the sharing management means are not shareable.

[0014] The server preferably includes a notification unit that transmits a notification to the terminal of the participant who is the subject of the video when the video is captured by the capture unit, so that the participant can know that he or she has been photographed.

[0015] It is preferable that the group photo generating means extracts keywords from the conversation information during the online conference, obtains an image related to the extracted keyword, and sets the obtained image as a background image of the group photo. This makes it possible to add a background that represents the atmosphere of the conference to the group photo, thereby improving the quality of the group photo.

[0016] A second invention is a server that is communicatively connected to terminals of participants in an online conference via a network, and is characterized in that it comprises: an online conference management means that manages entry and exit to the online conference, and receives video and audio transmitted from the terminal of each participant participating in the online conference and transmits them to the terminals of the other participants; a capture means that cuts out an image from the video of the subject participant and saves it as a captured image; an approval request means that sends an approval request to the terminal of the subject participant as to whether the captured image can be shared; and a sharing management means that, if approval is obtained from the subject participant, makes the captured image shareable with other subject participants, and, if approval is not obtained, makes the captured image unshareable.

[0017] According to the second invention, a server in the imaging system of the first invention can be provided.

[0020] No. 3 The invention is a photographing method in a system in which an online conference can be held between a plurality of terminals communicatively connected to a server via a network, comprising the steps of: the server managing entry and exit to the online conference, receiving video and audio transmitted from the terminal of each participant participating in the online conference, and transmitting the video and audio to the terminals of the other participants; the server extracting an image from the video of the subject participant and saving it as a captured image; the server sending an approval request to the terminal of the subject participant as to whether or not the captured image can be shared; the terminal responding to the approval request from the server with whether or not approval is granted; and the server making the captured image shareable with other subject participants if approval is obtained from the subject participant, and making the captured image unshareable if approval is not obtained.

[0021] No. 3The invention uses the video of each participant in an online conference to cut out (capture) an image of the participant, and when an approval response is received from the subject participant, the captured image can be shared among the participants (sharers) who responded. This makes it possible to easily take photos during an online conference and share them with the consent of the participants.

[0022] No. 4 The invention is a program for causing a computer communicatively connected to the terminals of participants in an online conference via a network to function as: online conference management means for managing entry and exit to the online conference, and receiving video and audio transmitted from the terminals of each participant participating in the online conference and transmitting them to the terminals of the other participants; capture means for cutting out an image from the video of the subject participant and saving it as a captured image; approval request means for sending an approval request to the terminal of the subject participant as to whether or not the captured image can be shared; and sharing management means for making the captured image shareable with other subject participants if approval is obtained from the subject participant, and making the captured image unshareable if approval is not obtained.

[0023] No. 4 According to the present invention, it is possible to cause a computer to function as the server of the imaging system of the first invention or the server of the second invention.

[0024] No. 5 The invention is a program for causing a computer used as a terminal of a participant in an online conference, which is communicatively connected to a server via a network, to function as a receiving means for receiving an approval request from the server as to whether or not a capture image taken of the participant as the subject in the online conference can be shared, and a responding means for responding to the approval request with the approval or disapproval.

[0025] No. 5 According to the present invention, a computer is connected to the end of the photographing system of the first invention. End and It is possible to make it function as such. Effect of the Invention

[0026] The present invention makes it possible to provide a photography system or the like that can easily take pictures using participant images in an online conference and share the pictures with the consent of the participants. [Brief description of the drawings]

[0027] [Figure 1] A block diagram showing the system configuration of a photographing system 1 according to the present invention. [Diagram 2] Figure showing an example of the hardware configuration of Server 3 [Diagram 3] FIG. 2 is a diagram showing an example of a hardware configuration of a user terminal 5. [Figure 4] A block diagram showing the functional configuration of the photography system 1 (the server 3 and the user terminal 5). [Diagram 5] FIG. 2 shows the configuration of an online conference processing unit 51 and an online conference management unit 31. [Figure 6] Flowchart showing the flow of online conference processing [Figure 7] FIG. 6 shows an example of a conference screen 61 displayed on a user terminal 5. [Figure 8] Flowchart showing the flow of image capture processing in an online conference [Figure 9] A diagram showing an example of image list 67 [Figure 10] FIG. 3 shows the configuration of shared management information 361. [Figure 11] Flowchart showing the flow of image provision processing after exit processing (end of conference) [Figure 12] A diagram showing examples of group photographs 81 and 82 [Figure 13] 11 is a flowchart showing the flow of a photographing process in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

[0029] [First embodiment] FIG. 1 is a diagram showing the system configuration of an imaging system 1 according to a first embodiment of the present invention.

[0030] As shown in FIG. 1, the photography system 1 is configured by a server 3 and user terminals 5 (5a, 5b, 5c, 5d, 5e) communicatively connected via a network 7. In the example of FIG. 1, an example is shown in which users of five user terminals 5a to 5e become participants in an online conference, but the number of terminals that can participate in an online conference is not limited to five and can be any number of terminals. In the following description, when there is no need to distinguish between the user terminals 5a to 5e, they are referred to as user terminals 5. In addition, in the present invention, a conference includes not only a business conference, but also a meeting, lecture, seminar, class, etc. in which users casually talk with each other.

[0031] The server 3 is a computer that performs communication processing of video and audio data between the user terminals 5 in an online conference held among a plurality of user terminals 5, and also performs entry and exit processing of users participating in the conference.

[0032] The user terminal 5 is a terminal used by a participant of an online conference, and is a computer that can be communicatively connected to the server 3 via a network 7. The user terminal 5 is equipped with a speaker 508, a microphone 507, and a camera 506. For example, the user terminal 5 may be a smartphone with built-in speakers 508, microphone 507, and camera 506, or may be a PC connected to an external camera, speaker, microphone, or headset.

[0033] The network 7 includes a LAN (Local Area Network), a WAN (Wide Area Network) that is connected over a wider area, or a public communication line such as the Internet, a base station, etc. The network 7 may be wired or wireless.

[0034] Fig. 2 is a diagram showing an example of a hardware configuration of the server 3. As shown in Fig. 2, the server 3 can be realized by a computer configured by connecting a control unit 301, a storage unit 302, and a communication unit 303 via a bus 309 or the like. However, the present invention is not limited to this, and various configurations can be used as appropriate.

[0035] The control unit 301 is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The CPU calls up a program stored in the storage unit 302, the ROM, a recording medium, etc., into a work memory area on the RAM and executes it, drives and controls each device connected via a bus 309, and realizes the processing performed by the server 3.

[0036] The storage unit 302 is a flash memory, a HDD (Hard Disk Drive), etc., and stores programs executed by the control unit 301, data necessary for executing the programs, an OS (Operating System), etc. As for the programs, a control program equivalent to an OS and an application program for causing the server 3 to execute processes related to online conferences and image capture processes during online conferences are stored. These program codes are read out as necessary by the control unit 301 and transferred to the RAM, and are read out by the CPU and executed as various means. Details of the processes executed by the server 3 will be described later.

[0037] The communication unit 303 has a communication control device, a communication port, etc., and is a communication interface that mediates communication between the server 3 and the network 7, and is capable of controlling communication with the user terminal 5 and other computers via the network 7. The bus 309 is a path that mediates the transmission and reception of control signals, data signals, etc. between each unit.

[0038] Fig. 3 is a diagram showing a hardware configuration of the user terminal 5. As shown in Fig. 3, the user terminal 5 can be realized by a computer configured by connecting a control unit 501, a storage unit 502, a communication unit 503, a display unit 504, an input unit 505, a camera 506, a microphone 507, and a speaker 508 via a bus 509. As described above, the camera 506, the microphone 507, and the speaker 508 may be externally attached via an interface. The configuration of the user terminal 5 is not limited to the configuration of Fig. 3, and various configurations can be used as appropriate.

[0039] The functions of the control unit 501, the storage unit 502, and the communication unit 503 are approximately the same as those of the control unit 301, the storage unit 302, and the communication unit 303 described above.

[0040] The display unit 504 is composed of a display such as a liquid crystal panel and a logic circuit (such as a video adapter) for executing display processing in cooperation with the display, and causes the display to display the display data input under the control of the control unit 501. Note that the display unit 504 may be a touch panel display in which an input device such as a touch panel is integrally provided on the display screen. The input unit 505 includes, for example, a touch panel, a keyboard, a pointing device such as a mouse, an input device such as a numeric keypad, and the like, and inputs input data to the control unit 501 .

[0041] The camera 506 is a camera that captures an image of the user, and is configured with an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor).

[0042] The microphone 507 picks up voice and inputs it to the control unit 501 . The speaker 508 outputs the sound input from the control unit 501 .

[0043] Fig. 4 is a block diagram showing a functional configuration of the photography system 1 according to the embodiment of the present invention. As shown in Fig. 4, the server 3 includes an online conference management unit 31, a photography instruction receiving unit 32, a capture unit 33, a capture notification unit 34, an approval request unit 35, a sharing management unit 36, a group photo generation unit 37, an image provision unit 38, etc. The user terminal 5 includes an online conference processing unit 51, a photography instruction unit 52, a photography instruction transmission unit 53, an approval / non-approval response unit 54, an image list display unit 55, etc.

[0044] FIG. 5 is a diagram illustrating the online conference management unit 31 of the server 3 and the online conference processing unit 51 of the user terminal 5. As shown in FIG. The online conference management unit 31 includes a communication unit 303, an entry / exit processing unit 31a, an ID setting unit 31b, a decoding unit 31c, a video acquisition unit 31d, an audio acquisition unit 31e, an audio generation unit 31f, a conference screen generation unit 31g, an audio output unit 31h, a video output unit 31i, an encoding unit 31j, and the like.

[0045] The entry / exit processing unit 31a manages entry (participation) and exit from a virtually set conference room in an online conference. The entry / exit processing unit 31a sets up a virtual conference room in response to a request from the user terminal 5 of the user who will be the host of the conference, assigns a conference ID, and accepts the setting of the conference name. When a request to participate in a conference is received from the user terminal 5, the entry / exit processing unit 31a performs login processing for the user terminal 5, and if the login is successful, provides a conference screen 61 and permits participation (entry) in the conference. In the login processing, input of the name, affiliation, etc. of the user to be used in the conference is accepted. When a request to leave is received from the user terminal 5, the entry / exit processing unit 31a ends transmission and reception of the conference screen 61 for that user terminal 5. The ID setting unit 31b sets a different participant ID (identification information) for each participant (user terminal 5) who has entered the conference, and deletes the participant ID when the participant leaves the conference.

[0046] The communication unit 303 of the server 3 transmits and receives audio data and video data to and from a plurality of user terminals 5. The audio data and video data are superimposed and transmitted and received as compressed and encoded data.

[0047] The decoding unit 31c decodes the compressed and encoded data received by the communication unit 303, and separates and extracts audio data and video data.

[0048] The video acquisition unit 31d acquires the video data extracted by the decoding unit 31c, and inputs it to the conference screen generation unit 31g. The voice acquisition unit 31e acquires the voice data extracted by the decoding unit 31c, performs signal processing such as noise removal, and inputs the data to the voice generation unit 31f.

[0049] The voice generating unit 31f superimposes the voices of the respective user terminals 5 acquired by the voice acquiring unit 31e, and inputs the superimposed voices to the encoding unit 31j.

[0050] The conference screen generating unit 31g generates a conference screen 61 in which the video of each participant acquired from each user terminal 5 is laid out, for each user terminal 5. The video output unit 31i inputs the conference screen 61 generated by the conference screen generating unit 31g to the encoding unit 31j.

[0051] The encoding unit 31j superimposes and compresses the audio data input from the audio output unit 31h and the conference screen data input from the video output unit 31i, and transmits the superimposed data to the user terminal 5 via the communication unit 303.

[0052] The online conference processing unit 51 of the user terminal 5 includes an encoding unit 51a, a decoding unit 51b, and the like. The encoding unit 51a superimposes the video input from the camera 506 and the audio input from the microphone 507, and performs a process of compression encoding. The decoding unit 51b performs a process of decoding the superimposed data of the audio and video (conference screen 61) received from the server 3, and separates it into audio data and video data. The audio data is output from the speaker 508. The video data is input to the display unit 504 and displayed in synchronization with the audio.

[0053] Returning to the explanation of Figure 4. The photographing instruction unit 52 of the user terminal 5 designates a participant as a subject on a screen (conference screen 61) on which the video of the participants of the online conference is displayed, and instructs the user to photograph the participant. The photographing instruction transmission unit 53 transmits identification information (participant ID) of the participant (photographed user) designated by the photographing instruction unit 52 and information on the photographing date and time to the server 3 as photographing instruction information.

[0054] The photographing instruction receiving unit 32 of the server 3 receives the photographing instruction information transmitted from the user terminal 5. The photographing instruction information includes the participant ID of the participant to be photographed, and photographing date and time information which is the date and time when the photographing instruction was input.

[0055] When the capture unit 33 of the server 3 receives the shooting instruction information, it issues a new shooting ID, cuts out an image from the video of the participant as the subject based on the received shooting instruction information, and saves it as a captured image in a predetermined image folder in the storage unit 302. The capture unit 33 assigns a shooting ID to each captured image and saves it. When shooting instructions are given with multiple participants as subjects, a captured image is obtained for each participant, but the same shooting ID is assigned to each captured image taken at the same time.

[0056] When a video is captured by the capture unit 33, the capture notification unit 34 of the server 3 transmits a notification indicating that the video has been captured to the user terminal 5 of the subject user. Upon receiving the notification, the user terminal 5 notifies the user of the notification by a shutter sound, an effect display of the corresponding window, or the like.

[0057] The approval request unit 35 of the server 3 transmits an approval request as to whether or not the captured image can be shared to the user terminal 5 of the subject user.

[0058] The approval / rejection response unit 54 of the user terminal 5 accepts an input of approval / rejection of sharing in response to an approval request from the server 3, and transmits the same to the server 3 (response).

[0059] If approval is obtained from the user terminal 5 of the subject user, the sharing management unit 36 ​​of the server 3 allows the capture image to be shared with other subject users (sharers). If approval is not obtained, the capture image cannot be shared. The sharing management unit 36 ​​generates sharing management information 361 shown in FIG. 10, for example, and stores it in the storage unit 302. The sharing management information 361 stores, for each capture image, the shooting ID, shooting date and time, sharer, approval information, etc. The sharing management information 361 will be described later.

[0060] The image list display unit 55 of the user terminal 5 obtains capture images taken during the online conference from the server 3, and displays them as a list as an image list 67. The image list display unit 55 hides capture images that are not shareable by the sharing management unit 36, and deletes them from the image list 67. In addition, the user terminal 5 that issued the shooting instruction displays a sharing prohibited icon 67e for capture images that are not shareable, indicating that they cannot be shared.

[0061] The group photo generating unit 37 generates a group photo 81 (82) by combining the shareable captured images. For example, the group photo generating unit 37 generates the group photo 81 by arranging the captured image of a certain participant in the center and arranging the captured images of the other participants around it. In addition, the name of the subject of the captured image included in the group photo 81 is obtained from the login information and displayed near the captured image. In addition, the group photo generating unit 37 obtains the title of the group photo 81 from the name of the conference and displays it in the group photo 81. In addition, the group photo generating unit 37 extracts keywords from the information of the conversation during the online conference, obtains images related to the extracted keywords, and sets the obtained images as the background images of the group photo 81 (82). The method of obtaining the images is, for example, to obtain them by searching the Web based on the extracted keywords. Alternatively, a method of registering background images tagged with each keyword in the storage unit 302 in advance is considered.

[0062] The image providing unit 38 provides the shareable captured image so that it can be used (displayed, saved, etc.) among the participants who shared the image. Providing the image includes, for example, sending the image by email to the user terminal 5 or sharing it in the cloud. In addition, the image providing unit 38 provides the group photo 81 (82) generated by the group photo generating unit 37 to the participants included in the group photo 81 (82).

[0063] Next, an online conference process executed between the user terminal 5 and the server 3 will be described with reference to Fig. 6. Note that a conference application program is stored in the storage unit 502 of the user terminal 5, which causes the user terminal 5 to execute the process described below. Similarly, a conference management application program is stored in the storage unit 302 of the server 3, which causes the server 3 to execute the process described below. The flowchart in Fig. 6 illustrates a case where a user terminal 5a (user A), a user terminal 5B (user B), and a user terminal 5c (user C) participate in a conference.

[0064] 6, the user terminals 5a, 5b, and 5c access the server 3 and perform an entry process with the server 3. In the entry process, when each of the user terminals 5a, 5b, and 5c transmits login information (user ID, authentication information, etc.) to the server 3, the entry / exit processing unit 31a of the server 3 performs an authentication process, and if the authentication is successful, the user is allowed to enter a predetermined virtual conference room (steps S101 to S104).

[0065] The ID setting unit 31b of the server 3 sets a participant ID for each user (user terminal 5a, 5b, 5c) who has entered the conference room (step S105).

[0066] When entry is permitted, each user terminal 5a, 5b, 5c compresses and encodes the camera image (video) captured by the camera 506 and the audio input from the microphone 507, and transmits the compressed and encoded data to the server 3 (steps S106, S107, and S108). When the server 3 receives the compressed and encoded data of the camera image (video) and audio transmitted from each user terminal 5a, 5b, 5c of the conference participants, it decodes the data to separate and extract the image data and audio data (step S109). In this way, the video of each participant (users A, B, C) is input to the video acquisition unit 31d of the server 3, and when a participant speaks, the voice of the speaker is input to the audio acquisition unit 31e of the server 3.

[0067] The voice generating unit 31f of the server 3 performs voice signal processing such as noise removal processing on the input voice of the speaker to generate voice for output. Also, the conference screen generating unit 31g of the server 3 generates a conference screen 61 in which the images input from each of the user terminals 5a, 5b, and 5c are arranged in each window (step S110).

[0068] FIG. 7 is a diagram showing an example of a conference screen 61. The conference screen 61 is provided with windows 62, 63, 64, and 65 for displaying the video of each participant. In the example of FIG. 7, four user terminals 5 participate in the conference, and four windows 62, 63, 64, and 65 are provided. The video input from the user terminals 5a to 5d of the participants to the server 3 is displayed, such as the video of user A in the window 62, the video of user B in the window 63, the video of user C in the window 64, and the video of user D in the window 65. The windows 62, 63, 64, and 65 may display the participant ID or the name of the user, or may be anonymous. The arrangement and size of the windows 62, 63, 64, and 65 may be arbitrary, and the layout may be changed in each user terminal 5. Also, the conference screen 61 may be generated on the user terminal 5 side, rather than on the server 3. In that case, the server 3 transmits the video input from each user terminal 5 to each user terminal 5 by linking it with the participant ID. The user terminal 5 displays the video of each participant in a window 62, 63, 64, or 65 of the corresponding participant ID.

[0069] 7, an image list 67 and a capture button 68 are displayed on the conference screen 61. The image list 67 and the capture button 68 will be described later.

[0070] The audio output unit 31h of the server 3 inputs the output audio generated in step S110 to the encoding unit 31j. The video output unit 31i inputs the conference screen 61 generated in step S110 to the encoding unit 31j. The encoding unit 31j superimposes the audio and video, compresses and encodes them (step S111), and transmits them to each user terminal 5 via the communication unit 303 (step S112).

[0071] Each user terminal 5 receives the video (conference screen 61) and audio transmitted from the server 3 (steps S113, S115, S117), and displays them on the display unit 504 and outputs them from the speaker 508 (steps S114, S116, S118).

[0072] Next, the photographing process in an online conference will be described with reference to the flowchart in Fig. 8. Fig. 8 describes the flow of the process when a participant (user A) designates a participant (user B) as a subject and photographs the participant on his / her user terminal 5a. User C is a participant in the online conference who is not the subject.

[0073] When user A wants to photograph user B as a subject, user A designates user B as the subject on the conference screen 61 displayed on the user terminal 5a (step S201), and in this state presses the capture button 68 (step S202). The subject designation operation is, for example, an instruction (click, touch operation) of a window displaying the image of the user to be designated. The user terminal 5a displays the designated window (windows 62, 63, 64 in the conference screen 61 of FIG. 7) so that it can be identified by highlighting, inverting, blinking, or the like. In addition, it is preferable that the capture button 68 can be switched between capturing a video 68a (moving image) and a photo 68b (still image). The example of FIG. 7 means capturing a photo 68b.

[0074] The control unit 501 of the user terminal 5a transmits the participant ID of the designated subject (subject user B) together with the photographing date and time, which is the time when the photographing button 68 was pressed, to the server 3 (step S203).

[0075] Upon receiving the shooting instruction information, the server 3 issues a shooting ID, and based on the shooting instruction information, cuts out (captures) an image of the corresponding shooting date and time from the video of the subject user B, and records it in a predetermined image folder in the storage unit 302 (step S204). The server 3 notifies the user terminal 5b of the subject user B that the image has been captured (photographed) (step S205). Upon receiving the notification from the server 3, the user terminal 5b of the subject user B notifies the user that the image has been photographed (step S206). The notification method may be, for example, sounding a shutter sound, or displaying an effect in the windows 62, 63, and 64 of the subject user B on the conference screen 61, as shown by reference numeral 66 in FIG. 7.

[0076] The server 3 generates a thumbnail image of the image captured in step S204 (step S207), and transmits the shooting ID, shooting date and time, and a thumbnail image of the captured image to the corresponding participant ID (subject user B), as well as transmitting an approval request as to whether the image may be shared between the photographer and the participants who became the subject (step S208). Note that the thumbnail image is transmitted in step S208 in order to reduce the amount of data, and the original captured image may also be transmitted.

[0077] The user terminal 5b of subject user B displays the shooting ID 67a, shooting date and time 67b, and captured image 67c (thumbnail image) received from the server 3 in an image list 67 as shown in FIG. 9, and also displays an “Approve” button 67g and a “Reject” button 67h, and accepts a response indicating whether or not to approve (step S209).

[0078] FIG. 9 is a diagram showing an example of the image list 67. As shown in FIG. 9, the image list 67 displays a shooting ID 67a, a shooting date and time 67b, and a capture image 67c (thumbnail image) in a list format, and each row displays either a shareable icon 67d or a share-prohibited icon 67e. Those without an icon display are those for which an approval / rejection response has not been obtained. In addition, if the user is the subject, an "Approve (OK)" button 67g and a "Reject" button 67h are displayed in the field of the corresponding shooting ID 67a. When the "Approve (OK)" button 67g is pressed, the display is switched to an approved display 67f such as "OK! I did it!". When the "Reject" button 67h is pressed, the capture image 67c (thumbnail image) of the shooting ID 67a is erased (hidden) from the image list 67. The shareable icon 67d is displayed when approval has been obtained from all participants (sharers) who were subjects with the same shooting ID. If even one of the participants (sharers) who were subjects of the same shooting ID refuses sharing, a sharing prohibition icon 67e will be displayed in the row of that shooting ID on the photographer's user terminal 5a, and the capture image 67c (thumbnail image) of that shooting ID 67a will be erased (hidden) from the image list 67 on the user terminals 5b, 5c, ... other than the photographer.

[0079] When a response of either approval or rejection is input to the user terminal 5b of the subject user B, the user terminal 5b transmits the shooting ID and approval information (approval or rejection) to the server 3 (step S210). The server 3 records the received approval information in the shared management information 361 (step S211).

[0080] FIG. 10 is a diagram showing an example of the sharing management information 361. As shown in FIG. 10, the sharing management information 361 records, for each "capture image", a "shooting ID", a "shooting date and time", a "sharer", "approval information", and a "sharing prohibition flag". For example, in the shooting of the shooting ID "#1", three subjects (users A, B, and C) are specified and photographed simultaneously (February 1, 2021, 19:03). In this case, the subject users "A, B, and C" are recorded in the "sharer" column. In the "approval information" column, if no response is received from the subject user, a value indicating non-approval ("not yet") is recorded, and if a response is received from the user terminal 5, "approval" or "rejection" is recorded according to the content.

[0081] In FIG. 10, in the shooting with shooting ID "#2", three subjects (users A, B, and C) were specified and photographed simultaneously (February 1, 2021, 19:10). The sharers are subject users A, B, and C, and responses of "approval" have been obtained from subject user A and subject user B, but a "rejection" response has been obtained from subject user C. In this case, since permission has not been obtained from all of the sharers, the server 3 sets a sharing prohibition flag on the captured image with the corresponding shooting ID "#2".

[0082] If the server 3 receives responses of "approval" from all of the subject users, the server 3 allows the capture image to be shared, and displays the capture image 67c (thumbnail image) and the shareable icon 67d in the image list 67 of all the sharers (steps S212, S214). If even one of the subject users does not approve ("reject"), the user terminals 5 of subject user B and another user C hide the capture image 67c (thumbnail image) of the corresponding shooting ID (steps S212, S215), and the user terminal 5 of the photographer displays the capture image 67c (thumbnail image) in the image list 67, but displays the sharing prohibited icon 67e (steps S212, S213).

[0083] Next, the process after the conference is ended will be described with reference to FIG. As shown in FIG. 11, when the server 3 receives a request to leave from each of the user terminals 5a, 5b, and 5c (steps S301, S302, and S303), it performs a process of leaving the room (step S304). When all the participants leave the room, the conference ends. After that, the server 3 extracts captured images approved by all the participants, i.e., images without a share prohibition flag, from the shared management information 361 (step S305), and generates a group photo 81 using original images of the extracted captured images (step S306). Alternatively, the server 3 provides the original images of the captured images and the generated group photo 81 to the sharers in a shareable manner (step S307). For example, the shareable captured images and the generated group photo 81 are sent to the user terminals 5 of the sharers so that they can be displayed and saved, or are uploaded to a cloud (image server) so that they can be downloaded, displayed, and saved on the user terminals 5 of the sharers.

[0084] The server 3 stores the images that cannot be shared for a predetermined period of time, and then deletes them from the image folder (step S308).

[0085] FIG. 12 is a diagram showing examples of group photographs 81 and 82. The group photo generating unit 37 of the server 3 generates a group photo by combining sharable captured images. For example, as shown in FIG. 12(a), the group photo generating unit 37 generates a group photo 81 for each participant, in which a captured image 81a of a certain participant is arranged in the center and captured images 81b, 81c, 81d, and 81e of other participants are arranged around it. In addition, the names 81f, 81g, 81h, and 81i of the subjects of the captured images 81b to 81e included in the group photo 81 may be obtained from the login information and laid out nearby, or a title 81k of the group photo 81 may be determined from the name of the conference and laid out in the group photo 81. In addition, the group photo generating unit 37 may extract keywords from information of conversations during an online conference, and obtain images related to the extracted keywords from a web search or the storage unit 302, and set them as a background image 82m of the group photo 82, as shown in FIG. 12(b).

[0086] The image providing unit 38 provides (by email, sharing in the cloud, etc.) the shareable captured image so that it can be used (displayed, saved, etc.) among the shared participants. The image providing unit 38 provides the group photos 81, 82 generated by the group photo generating unit 37 to the participants included in the group photos 81, 82.

[0087] As described above, in the photographing system 1 of the first embodiment, when a participant designates a participant as a subject on the screen of the user terminal 5 displaying the video of each participant in an online conference and operates the photographing button 68, the identification information (participant ID) of the designated participant and information on the photographing date and time are transmitted to the server 3 as photographing instruction information, and the server 3 cuts out an image from the video of the participant as a subject based on the photographing instruction information and saves it as a captured image. The server 3 transmits an approval request to the subject user as to whether the photographed captured image can be shared, and when an approval response is obtained from the subject user, the server 3 makes the captured image shareable among the participants (sharers) who responded with the approval. When approval is not obtained from the subject user, the server 3 manages the captured image as not shareable. This makes it possible to easily photograph each participant during an online conference. In addition, the captured image can be shared and used by the sharers even after the conference ends after obtaining the approval of the photographed participant. In addition, the server 3 can generate group photos 81 and 82, and provide the generated group photos 81 and 82 to each sharer as shareable.

[0088] [Second embodiment] Next, a second embodiment of the present invention will be described. Fig. 13 is a flow chart showing the photographing process in the photographing system 1 according to the second embodiment. The system configuration of the photographing system 1, the server 3, and the hardware configuration of the user terminal 5 in the second embodiment are similar to the system configuration of the photographing system 1, the server 3, and the hardware configuration of the user terminal 5 in the first embodiment. In addition, the process of entering and leaving an online conference and the process of transmitting and receiving video and audio during the conference (Figs. 6 and 11) are similar to those in the first embodiment.

[0089] The second embodiment differs from the imaging system 1 of the first embodiment in that the server 3 records a capture image when a predetermined condition is met, regardless of an imaging instruction from the user terminal 5. The server 3 monitors the video of each participant in an online conference, and when a predetermined condition is met, cuts out an image from the video of each participant and saves it as a capture image.

[0090] As shown in the flowchart of FIG. 13, the server 3 monitors the video transmitted from the user terminals 5a, 5b, and 5c of each participant in the online conference (step S401). For example, when all participants perform the same action (step S402; Yes), the server 3 cuts out an image of that moment from the video of each participant (users A, B, and C) and saves it as a captured image in a predetermined image folder of the storage unit 302 (step S403). The server 3 notifies the user terminals 5a, 5b, and 5c of the subject users A, B, and C that the image has been captured (photographed) (step S404). When the user terminals 5a, 5b, and 5c of the subject users A, B, and C receive the notification from the server 3, they notify the user terminals 5a, 5b, and 5c that the image has been photographed (steps S405, S406, and S407). The notification method is the same as in the first embodiment, for example, by sounding a shutter sound or displaying the subject user windows 62, 63, 64, and 65 of the conference screen 61 in FIG. 7 with an effect.

[0091] The server 3 generates a thumbnail image of the image photographed (captured) in step S403 (step S408), and transmits the photographing ID, photographing date and time, and the captured image (thumbnail image) to each of the subject users A, B, and C, as well as a request for approval as to whether the image may be shared among the subject participants A, B, and C (step S409).

[0092] The user terminals 5a, 5b, 5c of the subject users A, B, C display the shooting ID 67a, shooting date and time 67b, and capture image 67c (thumbnail image) received from the server 3 in the image list 67, as well as displaying an "Approve" button 67g and a "Reject" button 67h, and accept a response indicating whether or not to approve (steps S410, S411, S412).

[0093] When a response of either approval or rejection is input to the user terminals 5a, 5b, 5c of the subject users A, B, C, the user terminals 5a, 5b, 5c transmit the shooting ID and approval information (approval or rejection) to the server 3 (steps S413, S414, S415). The server 3 records the received approval information in the shared management information 361 (step S416).

[0094] If the server 3 receives responses of "approval" from all of the subject users A, B, and C, it makes the captured image 67c sharable, and displays the captured image 67c (thumbnail image) and a shareable icon 67d in the image list 67 of all the sharers. If any one of the subject users A, B, and C is not approved ("rejected"), the captured image 67c (thumbnail image) of the corresponding shooting ID is hidden on the user terminal 5 of each of the subject users A, B, and C (steps S418 to S421). The processing after the end of the conference is the same as in the first embodiment.

[0095] In the process of Fig. 13, the server 3 records the captured image under the condition that all participants perform the same action, but the condition is not limited to this. For example, the server 3 monitors the video of each participant, analyzes facial expressions and body movements, and when all participants make the same expression (e.g., laughing) at the same time or perform the same gesture (e.g., making a toast) at the same time, captures an image of that moment from the video of each participant.

[0096] In this way, the server 3 determines the timing to capture and takes the picture, so the participants do not miss the opportunity to take the picture and can concentrate on the conference. Also, each participant is notified that a picture has been taken and can respond with approval / rejection on whether or not to share it, so only the images they like can be shared.

[0097] Although the preferred embodiments of the imaging system and the like according to the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that a person skilled in the art can come up with various modified or altered examples within the scope of the technical ideas disclosed in this application, and it is understood that these also naturally fall within the technical scope of the present invention. [Explanation of symbols]

[0098] 1. Shooting system 3. Server 5, 5a to 5e: User terminal 6……………Conference room 7………………Network 10....Photographic equipment 31………………Online Conference Management Department 32: Shooting instruction receiving unit 33………………Capture section 33....Image acquisition section 34………………Capture notification section 35………………Approval request section 36………………Shared Management Department 37.....Group photo generation section 38………………Image provided by 39....Image output section 51...Online conference processing unit 52....Shooting instruction section 53....Shooting instruction transmission unit 54: Approval response section 55....Image list display section 61………………Meeting screen 62~65………Window 66.....Shooting effects 67………………Image list 67c……………Capture image (thumbnail image) 67d……………Shareable icon 67c……………No sharing icon 67g……………Approval button 67h……………Reject button 68....Shooting button 361…………Shared management information

Claims

1. A system capable of conducting online conferences between multiple terminals connected to a server via a network, The server, An online conference management means for managing entry and exit to the online conference, receiving video and audio transmitted from a terminal of each participant participating in the online conference, and transmitting the video and audio to terminals of other participants; A capture means for extracting an image from a video of a participant as a subject and saving it as a captured image; an approval request means for transmitting an approval request as to whether the captured image can be shared to a terminal of a participant who is the subject of the image; a sharing management means for allowing the captured image to be shared with other participants who are subjects when approval is obtained from the participant who is the subject, and for preventing the captured image from being shared when approval is not obtained, The terminal includes: A response means for responding to an approval request from the server with the approval or disapproval of the request. An imaging system comprising:

2. The terminal includes: A photographing instruction means for instructing to photograph a participant by designating the participant as a subject on a screen on which the video of the participant of the online conference is displayed; a transmission means for transmitting identification information of the participant designated by the photography instruction means and information on the photography date and time as photography instruction information to the server, The capture means of the server includes:

2. The photographing system according to claim 1, wherein an image is cut out from a video of the participant as a subject based on photographing instruction information transmitted from the terminal, and saved as a captured image.

3. The capture means includes:

3. The photographing system according to claim 1, wherein the video of each participant is monitored, and when a predetermined condition is satisfied, an image is cut out from the video of each participant and saved as a captured image.

4. The server, 4. The photographing system according to claim 1, further comprising an image providing unit that provides the captured image that has been made sharable by the sharing management unit so that the captured image can be used among participants who have shared the captured image.

5. 5. The photographing system according to claim 4, wherein the image providing means makes the captured image available even after the online conference has ended.

6. The server, a group photo generating unit that generates a group photo by combining the captured images that have been made sharable by the sharing management unit; 5. The photographing system according to claim 4, wherein the image providing means provides the group photograph to participants included in the group photograph.

7. the server includes a transmission means for transmitting the captured image to a terminal of each participant during the online conference; 7. The photographing system according to claim 1, wherein the terminal comprises an image list display unit that displays a list of the captured images received from the server.

8. 8. The photographing system according to claim 7, wherein the image list display means does not display captured images that are determined by the sharing management means to be unsharable.

9. 8. The photographing system according to claim 7, wherein the image list display means displays a message indicating that a captured image that has been determined by the sharing management means to be unsharable is unsharable.

10. The server, 10. The photographing system according to claim 1, further comprising a notification unit that transmits a notification to a terminal of a participant who is a subject of the photographing when the image is captured by the capture unit.

11. The group photo generating means includes: The photographing system according to claim 6, further comprising: extracting keywords from information about conversations during the online conference; acquiring an image related to the extracted keyword; and setting the acquired image as a background image of the group photograph.

12. A server communicatively connected to terminals of participants of an online conference via a network, An online conference management means for managing entry and exit to the online conference, receiving video and audio transmitted from a terminal of each participant participating in the online conference, and transmitting the video and audio to terminals of other participants; A capture means for extracting an image from a video of a participant as a subject and saving it as a captured image; an approval request means for transmitting an approval request as to whether the captured image can be shared to a terminal of a participant who is the subject of the image; a sharing management means for allowing the captured image to be shared with other participants who are subjects when approval is obtained from the participant who is the subject, and for preventing the captured image from being shared when approval is not obtained; A server comprising:

13. A method for capturing images in a system capable of conducting an online conference between a plurality of terminals communicatively connected to a server via a network, comprising: The server manages entry and exit to the online conference, receives video and audio transmitted from a terminal of each participant participating in the online conference, and transmits the video and audio to the terminals of other participants; The server extracts an image from the video of the participant as a subject and stores it as a captured image; a step of the server transmitting an approval request for whether the captured image can be shared to a terminal of the participant who is the subject of the image; a step of the terminal responding to an approval request from the server by indicating whether approval is granted; a step of allowing the server to share the captured image with other participants who are subjects when approval is obtained from the participant who is the subject, and not allowing the captured image to be shared when approval is not obtained; A photographing method comprising:

14. A computer that is connected to the terminals of online conference participants via a network, an online conference management means for managing entry and exit to the online conference, receiving video and audio transmitted from the terminal of each participant participating in the online conference, and transmitting the video and audio to the terminals of other participants; A capture means for extracting an image from a video of a participant as a subject and saving it as a captured image; an approval request means for transmitting an approval request as to whether the captured image can be shared to a terminal of a participant who is the subject of the image; a sharing management means for allowing the captured image to be shared with other participants who are subjects when approval is obtained from the participant who is the subject, and for preventing the captured image from being shared when approval is not obtained; A program to function as a

15. A computer used as a terminal for a participant of an online conference that is connected to a server via a network, A receiving means for receiving, from the server, an approval request as to whether or not a capture image taken of the user as a subject can be shared in an online conference; A response means for responding to the approval request with respect to whether approval is possible or not; A program to function as a

Images