Display control system, display control method, and display control program

Abstract

To provide a display control system, a display control method, and a display control program capable of appropriately switching a display image displayed on a conference terminal according to the contents of a conference.SOLUTION: A display control system is provided with: a conference acquisition processing unit that acquires conference information related to a conference; a video acquisition processing unit that acquires video information corresponding to a video of a user participating in the conference; a determination processing unit that determines the type of an image to be displayed on a conference terminal based on the conference information acquired by the conference acquisition processing unit; and an output processing unit that outputs an image corresponding to the video information acquired by the video acquisition processing unit and the type of the image determined by the determination processing unit.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a display control system, a display control method, and display control.

Background Art

[0002] In recent years, a video conferencing system (online conferencing system) that conducts a conference by transmitting and receiving voices, videos, files, etc. via a network at locations separated from each other has been used. For example, in the video conferencing system, face images of participants participating in the conference are displayed on each other's terminals (conference terminals). Conventionally, in the video conferencing system, a technique has been proposed in which a photographed image and a character image (avatar image) are switched according to the size of the display area and displayed on the conference terminal (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technology, there is a problem that a character image is displayed in a conference where it is not appropriate to use a character image, or a photographed image is displayed in a conference where it is not desired to use a photographed image. Thus, in the conventional technology, it is difficult to appropriately switch the display image according to the content of the conference.

[0005] An object of the present invention is to provide a display control system, a display control method, and a display control program capable of appropriately switching a display image displayed on a conference terminal according to the content of a conference.

Means for Solving the Problems

[0006] A display control system according to one aspect of the present invention is a display control system that causes a conference terminal to display an image corresponding to a user in a conference in which the user participates, including a conference acquisition processing unit that acquires conference information regarding the conference, a video acquisition processing unit that acquires video information corresponding to video of the user participating in the conference, a determination processing unit that determines a type of image to be displayed on the conference terminal based on the conference information acquired by the conference acquisition processing unit, and an output processing unit that outputs an image corresponding to the video information acquired by the video acquisition processing unit and the type of image determined by the determination processing unit to the conference terminal.

[0007] A display control method according to another aspect of the present invention is a display control method that causes a conference terminal to display an image corresponding to a user in a conference in which the user participates, and one or more processors execute a conference acquisition step of acquiring conference information regarding the conference, a video acquisition step of acquiring video information corresponding to video of the user participating in the conference, a determination step of determining a type of image to be displayed on the conference terminal based on the conference information acquired by the conference acquisition step, and an output step of outputting an image corresponding to the video information acquired by the video acquisition step and the type of image determined by the determination step to the conference terminal.

[0008] A display control program according to another aspect of the present invention is a display control program that causes a conference terminal to display an image corresponding to a user in a conference in which the user participates, and is a program for causing one or more processors to execute a conference acquisition step of acquiring conference information regarding the conference, a video acquisition step of acquiring video information corresponding to video of the user participating in the conference, a determination step of determining a type of image to be displayed on the conference terminal based on the conference information acquired by the conference acquisition step, and an output step of outputting an image corresponding to the video information acquired by the video acquisition step and the type of image determined by the determination step to the conference terminal.

Advantages of the Invention

[0009] According to the present invention, it is possible to provide a display control system, a display control method, and a display control program capable of appropriately switching a display image displayed on a conference terminal according to the content of a conference.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Note that the following embodiments are an example of embodying the present invention and do not have the character of limiting the technical scope of the present invention.

[0012] The display control system according to the present invention can be applied to, for example, a video conference (online meeting) in which a plurality of users participate at different locations (such as a conference room, a personal seat, or a home). The conference system according to the present embodiment is an example of the display control system of the present invention. In the conference system according to the present embodiment, for example, a user terminal used by a user participating in a conference is arranged in each conference room.

[0013] [Conference System 100] FIG. 1 is a diagram showing a schematic configuration of a conference system 100 according to an embodiment of the present invention. The conference system 100 includes a user terminal 1, a camera device 2 (see FIG. 2), and a schedule server 3. The conference system 100 is a system that supports a conference in which a plurality of users participate. Also, in a conference in which a user participates, the conference system 100 displays an image corresponding to the user on a conference terminal (such as the user terminal 1).

[0014] FIG. 1 shows three different locations. Here, an example is shown in which users A, B, and C participate in a video conference using the user terminal 1 at different locations. Each user terminal 1 can conduct a video conference by transmitting and receiving camera images and voices to and from other user terminals 1 by executing a conference application installed on its own terminal. Note that a microphone - speaker device and a camera device that can be connected to the user terminal 1 are connected to each location, and are configured to enable two - way communication of voice and camera images. The microphone - speaker device and the camera device may be built into the user terminal 1.

[0015] The schedule server 3 manages conference information D1 regarding the schedule (reservation) of a conference.

[0016] FIG. 3 is a diagram showing an example of the meeting information D1. In the meeting information D1, for each meeting, meeting reservation information in which information such as the holding date, holding time, meeting room ID, meeting name, reserving user ID, meeting URL, participants, and meeting materials are associated with each other is registered. The meeting room ID is identification information of the meeting room which is the venue of the meeting. The reserving user ID is identification information for identifying the user who reserved the meeting. The meeting URL is access information (connection information) for connecting when starting the meeting. For example, the control unit 11 accesses the meeting URL to start a meeting application (such as a general-purpose meeting application) corresponding to the meeting URL and connect to the connection destination ID (meeting ID). The meeting materials are materials used in the meeting. For example, when the holding of the meeting is determined, the reserving user performs a reservation operation of inputting meeting information including the above-mentioned respective information on his / her own PC (such as the user terminal 1). The control unit 11 registers the meeting reservation information by accessing the meeting information D1 of the schedule server 3 according to the reservation operation of the reserving user.

[0017] The schedule server 3 manages the meeting information D1 for each user. For example, the schedule server 3 registers the meeting information D1 in association with the user ID (such as the login ID).

[0018] In addition to the meeting information D1, the schedule server 3 may manage action schedule information, attendance information, etc. for each user. The action schedule information includes information such as the scheduled going-out information. The attendance information includes information such as working hours, working place (inside the company, at home, etc.), and scheduled vacation.

[0019] As the schedule server 3 according to the present embodiment, a well-known server (such as a cloud server) that provides a schedule management service can be used.

[0020] [User Terminal 1] As shown in FIG. 2, the user terminal 1 includes a control unit 11, a storage unit 12, an operation display unit 13, a communication unit 14, etc. The user terminal 1 is an information processing device such as a personal computer (PC), a smartphone, or a tablet terminal. The user terminal 1 is an example of the conference terminal of the present invention. Each user terminal 1 has the same configuration.

[0021] The communication unit 14 is a communication unit for connecting the user terminal 1 to the network N1 by wire or wirelessly and performing data communication according to a predetermined communication protocol with other devices such as other user terminals 1, a camera device 2, and a schedule server 3 via the network N1. Further, the user terminal 1 may be connected to conference devices (such as a display and a microphone speaker device) by a wired LAN or a wireless LAN via the communication unit 14.

[0022] The operation display unit 13 is a user interface including a display unit such as a liquid crystal display or an organic EL display for displaying various information, and an operation unit such as a mouse, a keyboard, or a touch panel for receiving operations. The operation display unit 13 receives, for example, operations of a user who uses the user terminal 1.

[0023] The storage unit 12 is a non-volatile storage unit such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory for storing various information. The storage unit 12 stores a control program such as a display control program for causing the control unit 11 to execute the display control process (see FIG. 6, etc.) described later. For example, the display control program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a reading device (not shown) such as a CD drive or a DVD drive provided in the user terminal 1 and stored in the storage unit 12. Note that the display control program may be distributed from a cloud server and stored in the storage unit 12.

[0024] In addition, the storage unit 12 stores image type information D2, which is information regarding the types of images to be displayed on the user terminal 1. FIG. 4 shows an example of the image type information D2. In the image type information D2, the types of output images according to conditions are registered. The conditions include each piece of information included in the conference information D1 (see FIG. 3), the type of conference application used in the conference, the type of microphone - speaker device used in the conference, conditions related to the communication means (e.g., Wi - Fi (registered trademark)) connected to the conference, etc. The output images include a captured image, which is the video of the user captured by the camera device 2, and a processed image obtained by processing the video of the user captured by the camera device 2. The processed images include character images (also referred to as avatar images), illustration images, digital human images, virtual background images, etc. For example, images 1 - 5 shown in FIG. 4 are respectively one of an avatar image, an illustration image, a digital human image, a virtual background image, and a 3D avatar image.

[0025] For example, the control unit 11 of the user terminal 1A of user A refers to the image type information D2 and determines the type of image to be displayed on the user terminal 1B of user B that satisfies the conditions related to the conference information D1. A specific example of the method for determining the type of the image will be described later.

[0026] The control unit 11 includes control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic processes. The ROM pre - stores control programs such as BIOS and OS for causing the CPU to execute various processes. The RAM stores various information and is used as a temporary storage memory (working area) for various processes executed by the CPU. Then, the control unit 11 controls the user terminal 1 by causing the CPU to execute various control programs pre - stored in the ROM or the storage unit 12.

[0027] Specifically, the control unit 11 includes various processing units such as a meeting acquisition processing unit 111, a video acquisition processing unit 112, a decision processing unit 113, a generation processing unit 114, and an output processing unit 115. The control unit 11 functions as the various processing units by executing various processes according to the display control program. Also, some or all of the processing units included in the control unit 11 may be configured by electronic circuits. The display control program may be a program for causing a plurality of processors to function as the various processing units.

[0028] The meeting acquisition processing unit 111 acquires meeting information D1 (see FIG. 3). For example, when user A performs a login operation on the user terminal 1A, the meeting acquisition processing unit 111 accesses the schedule server 3 to acquire the meeting information D1 associated with the user ID of user A.

[0029] The video acquisition processing unit 112 acquires video information corresponding to the video of the users participating in the meeting. Here, a camera device 2 is individually connected to each user terminal 1. For example, a camera device 2A is connected to the user terminal 1A, and a camera device 2B is connected to the user terminal 1B (see FIG. 2). The camera device 2A captures user A using the user terminal 1A, and the camera device 2B captures user B using the user terminal 1B. Each camera device 2 may be built into the user terminal 1.

[0030] Also, the camera device 2 may be a device that outputs the captured video (captured image) to the user terminal 1, or may be a device (such as a smartphone) that extracts the feature amount of the user's face based on the captured video and outputs the extracted feature amount data to the user terminal 1. Also, a plurality of camera devices 2 may be connected to one user terminal 1. In this case, the captured image and the feature amount data may be input to the user terminal 1.

[0031] At the start of a meeting, during the meeting, etc., the video acquisition processing unit 112 acquires video information (captured images, feature amount data) from the camera device 2. That is, the video acquisition processing unit 112 acquires a captured image that is a video of the user captured by the camera device 2, or feature amount data extracted based on the video of the user captured by the camera device 2.

[0032] Based on the conference information D1 acquired by the conference acquisition processing unit 111, the determination processing unit 113 determines the type of image to be displayed on another user terminal 1. Specifically, the determination processing unit 113 refers to the image type information D2 (see FIG. 4) and determines the type that satisfies the conditions regarding the conference information D1 (see FIG. 3). For example, the determination processing unit 113 determines the type of image based on at least any one of the conference time, conference participants, content of the conference, importance of the conference, and the location where the user participating in the conference is located, which are included in the conference information D1.

[0033] In addition, the determination processing unit 113 determines the type of image based on at least any one of the type of conference application used in the conference, the type of microphone - speaker device used in the conference, and the communication means connected to the conference.

[0034] In addition, based on the conference information D1, the determination processing unit 113 determines, as the type, a captured image that is a video of the user captured by the camera device 2, or a processed image obtained by processing the video of the user captured by the camera device 2. Note that the processed image includes at least any one of a character image, an illustration image, a digital human image, and a virtual background image.

[0035] For example, when the meeting that user A participates in is a meeting in which external members participate, the determination processing unit 113 determines "captured image" as the type of the image. Also, for example, when the meeting that user A participates in is a meeting outside working hours, the determination processing unit 113 determines "processed image (for example, avatar image)" as the type of the image.

[0036] The generation processing unit 114 generates the processed image from the video information acquired by the video acquisition processing unit 112. Specifically, the generation processing unit 114 generates the processed image corresponding to the type determined by the determination processing unit 113 for the video information acquired by the video acquisition processing unit 112. For example, the generation processing unit 114 generates a character image (avatar image), an illustration image, a digital human image, a virtual background image, etc. based on the captured image captured by the camera device 2. Well-known techniques can be applied to the method of generating the processed image from the captured image.

[0037] The output processing unit 115 outputs an image corresponding to the video information acquired by the video acquisition processing unit 112 and the type of the image determined by the determination processing unit 113 to another user terminal 1. For example, when the meeting that user A participates in is a meeting that external members participate in, the output processing unit 115 of the user terminal 1A outputs the captured image of user A to the user terminal 1B. In this case, the user terminal 1B displays the captured image of user A on the operation display unit 13 of its own terminal.

[0038] Also, for example, when the meeting that user A participates in is a meeting outside working hours, the output processing unit 115 of the user terminal 1A outputs the avatar image of user A to the user terminal 1B. In this case, the user terminal 1B displays the avatar image of user A on the operation display unit 13 of its own terminal.

[0039] The user terminals 1A and 1B execute the same processing as each other, so that, as shown in the meeting application screen of FIG. 5, on each of the user terminals 1A and 1B, the image Pa of user A and the image Pb of user B are displayed. Note that an icon image Sc of the logged-in user may be displayed on the meeting application screen of FIG. 5. Also, the icon image Sc may be an image representing the user's status (on vacation, commuting, working remotely, etc.).

[0040] In addition to the above-mentioned various processes, the control unit 11 executes a conference process. Specifically, the control unit 11 activates a conference application and connects to other user terminals 1. When each user terminal 1 is connected, the control unit 11 starts a conference by transmitting and receiving audio, video, files, etc. Further, the control unit 11 causes a conference application screen (see FIG. 5) to be displayed.

[0041] In addition, the control unit 11 receives various operations (such as a conference start operation, a conference end operation, a screen sharing operation, etc.) from the user on the conference application screen.

[0042] [Display control process] Hereinafter, with reference to FIGS. 6 to 17, an example of the procedure of the display control process executed in the conference system 100 will be described. Each flowchart shown in FIGS. 6 to 17 shows a specific example of the display control process. Here, the display control process executed in the user terminal 1A used by user A will be described. Specifically, the control unit 11 of the user terminal 1A determines the type of image to be displayed on the user terminal 1B based on specific information such as conference information, and executes a process of outputting the determined type of image to the user terminal 1B.

[0043] Note that the present invention can be regarded as an invention of a display control method (an example of the display control method of the present invention) that executes one or more steps included in the display control process. Also, one or more steps included in the display control process described here may be appropriately omitted. Also, the execution order of each step in the display control process may be different within a range that produces the same operational effects.

[0044] Furthermore, here, the case where the control unit 11 of the user terminal 1 executes each step in the display control process is taken as an example for explanation. However, in other embodiments, one or more processors may execute each step in the display control process in a distributed manner.

[0045] [Specific example 1] FIG. 6 is a flowchart corresponding to the display control process according to Specific Example 1.

[0046] In step S11, the control unit 11 acquires conference information and video information. Specifically, when user A performs a login operation, the control unit 11 acquires conference information D1 (see FIG. 3) corresponding to user A from the schedule server 3. In addition, when user A performs a login operation, the control unit 11 acquires video information (for example, a captured image) from the camera device 2A.

[0047] Next, in step S12, the control unit 11 determines whether the current time is within user A's working hours. The working hours may be the working hours specific to user A or the shared working hours defined within the company to which user A belongs. If the current time is within user A's working hours (S12: Yes), the control unit 11 transfers the process to step S13. On the other hand, if the current time is outside user A's working hours (S12: No), the control unit 11 transfers the process to step S121.

[0048] In step S121, the control unit 11 determines "avatar image (character image)" as the type of image to be displayed on the user terminal 1B. After step S121, the control unit 11 transfers the process to step S110.

[0049] In step S13, the control unit 11 determines whether the current time is within the conference time of the conference that user A participates in. If the current time is within user A's conference time (S13: Yes), the control unit 11 transfers the process to step S14. On the other hand, if the current time is outside user A's conference time (S13: No), the control unit 11 transfers the process to step S131.

[0050] In step S131, the control unit 11 determines "illustration image" as the type of image to be displayed on the user terminal 1B. After step S131, the control unit 11 transfers the process to step S110.

[0051] In step S14, the control unit 11 determines whether the meeting participants are only internal members of the company. If the meeting participants are only internal members of the company (S14: Yes), the control unit 11 causes the process to proceed to step S141. On the other hand, if the meeting participants are not only internal members of the company, that is, if the meeting participants include external members of the company (S14: No), the control unit 11 causes the process to proceed to step S15.

[0052] In step S141, the control unit 11 determines "digital human image" as the type of image to be displayed on the user terminal 1B. After step S141, the control unit 11 causes the process to proceed to step S110.

[0053] In step S15, the control unit 11 determines "captured image" as the type of image to be displayed on the user terminal 1B. After step S15, the control unit 11 causes the process to proceed to step S16.

[0054] In step S110, the control unit 11 generates an image of the determined type. Here, the control unit 11 generates an avatar image, an illustration image, or a digital human image based on the captured image. After step S110, the control unit 11 causes the process to proceed to step S16.

[0055] In step S16, the control unit 11 outputs the image of the determined type to the user terminal 1B. Here, the control unit 11 outputs a captured image, an avatar image, an illustration image, or a digital human image to the user terminal 1B. When the user terminal 1B acquires the image from the user terminal 1A, it displays the image on the conference application screen (see FIG. 5).

[0056] [Specific Example 2] FIG. 7 is a flowchart corresponding to the display control process according to Specific Example 2.

[0057] In step S21, the control unit 11 acquires conference information and video information. Specifically, when user A performs a login operation, the control unit 11 acquires conference information D1 corresponding to user A (see FIG. 3) from the schedule server 3. Also, when user A performs a login operation, the control unit 11 acquires video information (e.g., a captured image) from the camera device 2A.

[0058] Next, in step S22, the control unit 11 determines whether the word "outside the company" is included in the conference name (title). If the word "outside the company" is included in the conference name (S22: Yes), the control unit 11 shifts the process to step S221. On the other hand, if the word "outside the company" is not included in the conference name (S22: No), the control unit 11 shifts the process to step S23.

[0059] In step S221, the control unit 11 determines "captured image" as the type of image to be displayed on the user terminal 1B. After step S221, the control unit 11 shifts the process to step S27.

[0060] In step S23, the control unit 11 determines whether an external email address is included in the conference information D1. If an external email address is included in the conference information D1 (S23: Yes), the control unit 11 shifts the process to step S221. On the other hand, if an external email address is not included in the conference information D1 (S23: No), the control unit 11 shifts the process to step S24.

[0061] In step S24, the control unit 11 determines whether "important" is set in the conference flag. If "important" is set in the conference flag (S24: Yes), the control unit 11 shifts the process to step S241. On the other hand, if "important" is not set in the conference flag (S24: No), the control unit 11 shifts the process to step S25.

[0062] In step S241, the control unit 11 determines "virtual background image" as the type of the image to be displayed on the user terminal 1B. After step S241, the control unit 11 causes the process to proceed to step S26.

[0063] In step S25, the control unit 11 determines "avatar image" as the type of the image to be displayed on the user terminal 1B. After step S25, the control unit 11 causes the process to proceed to step S26.

[0064] In step S26, the control unit 11 generates the image of the determined type. Here, the control unit 11 generates a virtual background image or an avatar image based on the captured image. After step S26, the control unit 11 causes the process to proceed to step S27.

[0065] In step S27, the control unit 11 outputs the image of the determined type to the user terminal 1B. Here, the control unit 11 outputs the captured image, the virtual background image, or the avatar image to the user terminal 1B. When the user terminal 1B acquires the image from the user terminal 1A, the user terminal 1B displays the image on the conference application screen (see FIG. 5).

[0066] [Specific Example 3] FIG. 8 is a flowchart corresponding to the display control process according to Specific Example 3.

[0067] In step S31, the control unit 11 acquires conference information and video information. Specifically, when user A performs a login operation, the control unit 11 acquires conference information D1 corresponding to user A (see FIG. 3) from the schedule server 3. In addition, when user A performs a login operation, the control unit 11 acquires video information (for example, a captured image) from the camera device 2A.

[0068] Next, in step S32, the control unit 11 determines whether the meeting information D1 includes address information. If the meeting information D1 includes address information (S32: Yes), the control unit 11 causes the process to proceed to step S321. On the other hand, if the meeting information D1 does not include address information (S32: No), the control unit 11 causes the process to proceed to step S33.

[0069] In step S321, the control unit 11 determines that the type of the image to be displayed on the user terminal 1B is a "captured image". After step S321, the control unit 11 causes the process to proceed to step S37.

[0070] In step S33, the control unit 11 determines whether the location corresponding to the address information is a meeting room. If the location corresponding to the address information is a meeting room (S33: Yes), the control unit 11 causes the process to proceed to step S321. On the other hand, if the location corresponding to the address information is not a meeting room (S33: No), the control unit 11 causes the process to proceed to step S34.

[0071] In step S34, the control unit 11 determines whether the location corresponding to the address information is the user's seat. If the location corresponding to the address information is the user's seat (S34: Yes), the control unit 11 causes the process to proceed to step S341. On the other hand, if the location corresponding to the address information is not the user's seat (S34: No), the control unit 11 causes the process to proceed to step S35.

[0072] In step S341, the control unit 11 determines that the type of the image to be displayed on the user terminal 1B is a "digital human image". After step S341, the control unit 11 causes the process to proceed to step S36.

[0073] In step S35, the control unit 11 determines that the type of the image to be displayed on the user terminal 1B is an "avatar image". After step S35, the control unit 11 causes the process to proceed to step S36.

[0074] In step S36, the control unit 11 generates an image of the determined type. Here, the control unit 11 generates a digital human image or an avatar image based on the captured image. After step S36, the control unit 11 causes the process to proceed to step S37.

[0075] In step S37, the control unit 11 outputs the image of the determined type to the user terminal 1B. Here, the control unit 11 outputs the captured image, the digital human image, or the avatar image to the user terminal 1B. When the user terminal 1B acquires the image from the user terminal 1A, it is displayed on the conference application screen (see FIG. 5).

[0076] [Specific Example 4] FIG. 9 is a flowchart corresponding to the display control process according to Specific Example 4.

[0077] In step S41, the control unit 11 acquires conference information and video information. Specifically, when user A performs a login operation, the control unit 11 acquires conference information D1 corresponding to user A from the schedule server 3 (see FIG. 3). Further, when user A performs a login operation, the control unit 11 acquires video information (for example, a captured image) from the camera device 2A.

[0078] Next, in step S42, the control unit 11 determines whether the conference to be started is a conference pre-registered in the conference information D1. If the conference to be started is a conference pre-registered in the conference information D1 (S42: Yes), the control unit 11 causes the process to proceed to step S421. On the other hand, if the conference to be started is not a conference pre-registered in the conference information D1 (S42: No), the control unit 11 causes the process to proceed to step S43.

[0079] In step S421, the control unit 11 determines "captured image" as the type of image to be displayed on the user terminal 1B. After step S421, the control unit 11 causes the process to proceed to step S45.

[0080] In step S43, the control unit 11 determines "virtual background image" as the type of image to be displayed on the user terminal 1B. After step S43, the control unit 11 causes the process to proceed to step S44.

[0081] In step S44, the control unit 11 generates an image of the determined type. Here, the control unit 11 generates a virtual background image based on the captured image. After step S44, the control unit 11 causes the process to proceed to step S45.

[0082] In step S45, the control unit 11 outputs the image of the determined type to the user terminal 1B. Here, the control unit 11 outputs the captured image or the virtual background image to the user terminal 1B. When the user terminal 1B acquires the image from the user terminal 1A, it displays the image on the conference application screen (see FIG. 5).

[0083] [Specific Example 5] FIG. 10 is a flowchart corresponding to the display control process according to Specific Example 5.

[0084] In step S51, the control unit 11 acquires conference information and video information. Specifically, when user A performs a login operation, the control unit 11 acquires conference information D1 corresponding to user A (see FIG. 3) from the schedule server 3. Also, when user A performs a login operation, the control unit 11 acquires video information (for example, a captured image) from the camera device 2A.

[0085] Next, in step S52, the control unit 11 determines whether the application used for the video conference is an application for business negotiation. If the application used for the video conference is an application for business negotiation (S52: Yes), the control unit 11 causes the process to proceed to step S521. On the other hand, if the application used for the video conference is not an application for business negotiation (S52: No), the control unit 11 causes the process to proceed to step S53.

[0086] In step S521, the control unit 11 determines "captured image" as the type of image to be displayed on the user terminal 1B. After step S521, the control unit 11 causes the process to proceed to step S56.

[0087] In step S53, the control unit 11 determines whether the application used for the video conference is the application for the seminar. If the application used for the video conference is the application for the seminar (S53: Yes), the control unit 11 causes the process to proceed to step S531. On the other hand, if the application used for the video conference is not the application for the seminar (S53: No), the control unit 11 causes the process to proceed to step S54.

[0088] In step S531, the control unit 11 determines "virtual background image" as the type of image to be displayed on the user terminal 1B. After step S531, the control unit 11 causes the process to proceed to step S510.

[0089] In step S54, the control unit 11 determines whether the application used for the video conference is the application for the meeting. If the application used for the video conference is the application for the meeting (S54: Yes), the control unit 11 causes the process to proceed to step S541. On the other hand, if the application used for the video conference is not the application for the meeting (S54: No), the control unit 11 causes the process to proceed to step S55.

[0090] In step S541, the control unit 11 determines "avatar image" as the type of image to be displayed on the user terminal 1B. After step S541, the control unit 11 causes the process to proceed to step S510.

[0091] In step S55, the control unit 11 determines "captured image" as the type of image to be displayed on the user terminal 1B. After step S55, the control unit 11 causes the process to proceed to step S56.

[0092] In step S510, the control unit 11 generates an image of the determined type. Here, the control unit 11 generates a virtual background image or an avatar image based on the captured image. After step S510, the control unit 11 causes the process to proceed to step S56.

[0093] In step S56, the control unit 11 outputs the image of the determined type to the user terminal 1B. Here, the control unit 11 outputs the captured image, the virtual background image, or the avatar image to the user terminal 1B. When the user terminal 1B acquires the image from the user terminal 1A, it displays the image on the application screen used for the video conference (see FIG. 5).

[0094] [Specific Example 6] FIG. 11 is a flowchart corresponding to the display control process according to Specific Example 6.

[0095] In step S61, the control unit 11 acquires information on the microphone - speaker device connected to the user terminal 1A. Also, when user A performs a login operation, the control unit 11 acquires video information (for example, a captured image) from the camera device 2A.

[0096] Next, in step S62, the control unit 11 determines whether the microphone - speaker device is a microphone - speaker device for a conference room. If the microphone - speaker device is a microphone - speaker device for a conference room (S62: Yes), the control unit 11 causes the process to proceed to step S621. On the other hand, if the microphone - speaker device is not a microphone - speaker device for a conference room (S62: No), the control unit 11 causes the process to proceed to step S63.

[0097] In step S621, the control unit 11 determines that the type of the image to be displayed on the user terminal 1B is a "captured image". After step S621, the control unit 11 causes the process to proceed to step S67.

[0098] In step S63, the control unit 11 determines whether the microphone speaker device is a personal microphone speaker device. If the microphone speaker device is a personal microphone speaker device (S63: Yes), the control unit 11 causes the process to proceed to step S631. On the other hand, if the microphone speaker device is not a personal microphone speaker device (S63: No), the control unit 11 causes the process to proceed to step S64.

[0099] In step S631, the control unit 11 determines "virtual background image" as the type of image to be displayed on the user terminal 1B. After step S631, the control unit 11 causes the process to proceed to step S66.

[0100] In step S64, the control unit 11 determines whether the microphone speaker device is a headset device. If the microphone speaker device is a headset device (S64: Yes), the control unit 11 causes the process to proceed to step S641. On the other hand, if the microphone speaker device is not a headset device (S64: No), the control unit 11 causes the process to proceed to step S65.

[0101] In step S641, the control unit 11 determines "digital human image" as the type of image to be displayed on the user terminal 1B. After step S641, the control unit 11 causes the process to proceed to step S66.

[0102] In step S65, the control unit 11 determines "avatar image" as the type of image to be displayed on the user terminal 1B. After step S65, the control unit 11 causes the process to proceed to step S66.

[0103] In step S66, the control unit 11 generates an image of the determined type. Here, the control unit 11 generates a virtual background image, a digital human image, or an avatar image based on the captured image. After step S66, the control unit 11 causes the process to proceed to step S67.

[0104] In step S67, the control unit 11 outputs the determined type of image to the user terminal 1B. Here, the control unit 11 outputs a photographed image, an avatar image, a virtual background image, or a digital human image to the user terminal 1B. When the user terminal 1B acquires the image from the user terminal 1A, it displays the image on the conference application screen (see FIG. 5).

[0105] [Specific Example 7] FIG. 12 is a flowchart corresponding to the display control process according to Specific Example 7.

[0106] In step S71, the control unit 11 acquires Wi-Fi connection destination information and video information. Specifically, when user A performs a login operation, the control unit 11 acquires information on the Wi-Fi connection destination (home, company, facility, etc.). Also, when user A performs a login operation, the control unit 11 acquires video information (for example, a photographed image) from the camera device 2A.

[0107] Next, in step S72, the control unit 11 determines whether the Wi-Fi connection destination is home. If the Wi-Fi connection destination is home (S72: Yes), the control unit 11 transfers the process to step S721. On the other hand, if the Wi-Fi connection destination is not home (S72: No), the control unit 11 transfers the process to step S73.

[0108] In step S721, the control unit 11 determines "avatar image" as the type of image to be displayed on the user terminal 1B. After step S721, the control unit 11 transfers the process to step S710.

[0109] In step S73, the control unit 11 determines whether the Wi-Fi connection destination is within the company. If the Wi-Fi connection destination is within the company (S73: Yes), the control unit 11 transfers the process to step S731. On the other hand, if the Wi-Fi connection destination is not within the company (S73: No), the control unit 11 transfers the process to step S74.

[0110] In step S731, the control unit 11 determines "virtual background image" as the type of image to be displayed on the user terminal 1B. After step S731, the control unit 11 causes the process to proceed to step S710.

[0111] In step S74, the control unit 11 determines whether the Wi-Fi connection destination is a facility (such as a hotel or a café). If the Wi-Fi connection destination is a facility (S74: Yes), the control unit 11 causes the process to proceed to step S741. On the other hand, if the Wi-Fi connection destination is not a facility (S74: No), the control unit 11 causes the process to proceed to step S75.

[0112] In step S741, the control unit 11 determines "digital human image" as the type of image to be displayed on the user terminal 1B. After step S741, the control unit 11 causes the process to proceed to step S710.

[0113] In step S75, the control unit 11 determines "captured image" as the type of image to be displayed on the user terminal 1B. After step S75, the control unit 11 causes the process to proceed to step S76.

[0114] In step S710, the control unit 11 generates an image of the determined type. Here, the control unit 11 generates an avatar image, a virtual background image, or a digital human image based on the captured image. After step S710, the control unit 11 causes the process to proceed to step S76.

[0115] In step S76, the control unit 11 outputs the image of the determined type to the user terminal 1B. Here, the control unit 11 outputs the captured image, the avatar image, the virtual background image, or the digital human image to the user terminal 1B. When the user terminal 1B acquires the image from the user terminal 1A, the user terminal 1B displays the image on the application screen used for the video conference (see FIG. 5).

[0116] [Specific Example 8] FIG. 13 is a flowchart corresponding to the display control process according to Specific Example 8.

[0117] In step S81, the control unit 11 acquires the information of the status icon and the video information. Specifically, when user A performs a login operation, the control unit 11 acquires the information of the icon image Sc (see FIG. 5) representing the status. Also, when user A performs a login operation, the control unit 11 acquires video information (for example, a captured image) from the camera device 2A.

[0118] Next, in step S82, the control unit 11 determines whether the status icon is an icon representing being on vacation. If the status icon is an icon representing being on vacation (S82: Yes), the control unit 11 transfers the process to step S821. On the other hand, if the status icon is not an icon representing being on vacation (S82: No), the control unit 11 transfers the process to step S83.

[0119] In step S821, the control unit 11 determines "avatar image" as the type of image to be displayed on the user terminal 1B. After step S821, the control unit 11 transfers the process to step S810.

[0120] In step S83, the control unit 11 determines whether the status icon is an icon representing being on the way to work. If the status icon is an icon representing being on the way to work (S83: Yes), the control unit 11 transfers the process to step S831. On the other hand, if the status icon is not an icon representing being on the way to work (S83: No), the control unit 11 transfers the process to step S84.

[0121] In step S831, the control unit 11 determines "virtual background image" as the type of image to be displayed on the user terminal 1B. After step S831, the control unit 11 transfers the process to step S810.

[0122] In step S84, the control unit 11 determines whether the status icon is an icon indicating remote work in progress. If the status icon is an icon indicating remote work in progress (S84: Yes), the control unit 11 causes the process to proceed to step S841. On the other hand, if the status icon is not an icon indicating remote work in progress (S84: No), the control unit 11 causes the process to proceed to step S85.

[0123] In step S841, the control unit 11 determines "digital human image" as the type of image to be displayed on the user terminal 1B. After step S841, the control unit 11 causes the process to proceed to step S810.

[0124] In step S85, the control unit 11 determines "captured image" as the type of image to be displayed on the user terminal 1B. After step S85, the control unit 11 causes the process to proceed to step S86.

[0125] In step S810, the control unit 11 generates an image of the determined type. Here, the control unit 11 generates an avatar image, a virtual background image, or a digital human image based on the captured image. After step S810, the control unit 11 causes the process to proceed to step S86.

[0126] In step S86, the control unit 11 outputs the determined type of image to the user terminal 1B. Here, the control unit 11 outputs a captured image, an avatar image, a virtual background image, or a digital human image to the user terminal 1B. When the user terminal 1B acquires the image from the user terminal 1A, it displays the image on the application screen used for the video conference (see FIG. 5).

[0127] [Specific Example 9] FIG. 14 is a flowchart corresponding to the display control process according to Specific Example 9.

[0128] In step S91, the control unit 11 acquires the state of the conference support application and video information. When user A performs a login operation, the control unit 11 acquires video information (for example, a captured image) from the camera device 2A.

[0129] Here, the conference support application is an application that supports video conferencing executed by a general-purpose conference application. The conference support application manages the progress of the conference and accepts user operations. Also, in the conference support application, the user can set the operation mode to the home mode or the office mode. For example, the user sets it to the home mode for a private conference and to the office mode for a work-related conference.

[0130] Next, in step S92, the control unit 11 determines whether the operation mode is set to the home mode. If the operation mode is set to the home mode (S92: Yes), the control unit 11 causes the process to proceed to step S921. On the other hand, if the operation mode is set to the office mode (S92: No), the control unit 11 causes the process to proceed to step S93.

[0131] In step S921, the control unit 11 determines "avatar image" as the type of image to be displayed on the user terminal 1B. After step S921, the control unit 11 causes the process to proceed to step S922.

[0132] In step S922, the control unit 11 generates an image of the determined type. Here, the control unit 11 generates an avatar image based on the captured image. After step S922, the control unit 11 causes the process to proceed to step S94.

[0133] In step S93, the control unit 11 determines "captured image" as the type of image to be displayed on the user terminal 1B. After step S93, the control unit 11 causes the process to proceed to step S94.

[0134] In step S94, the control unit 11 outputs the determined type of image to the user terminal 1B. Here, the control unit 11 outputs the captured image or the avatar image to the user terminal 1B. When the user terminal 1B acquires the image from the user terminal 1A, it is displayed on the application screen used for the video conference (see FIG. 5).

[0135] [Specific Example 10] FIG. 15 is a flowchart corresponding to the display control process according to Specific Example 10.

[0136] In step S101, the control unit 11 acquires the NFC communication history and video information. Specifically, user A taps the NFC at the start of the meeting. When user A taps the NFC, the communication history is stored in the storage unit 12. Also, when user A performs a login operation, the control unit 11 acquires video information (for example, a captured image) from the camera device 2A.

[0137] Next, in step S102, the control unit 11 determines whether the NFC was tapped immediately before the start of the meeting. If the NFC was tapped immediately before the start of the meeting (S102: Yes), the control unit 11 transfers the process to step S106. On the other hand, if the NFC was not tapped immediately before the start of the meeting (S102: No), the control unit 11 transfers the process to step S103.

[0138] In step S106, the control unit 11 determines "captured image" as the type of image to be displayed on the user terminal 1B. After step S106, the control unit 11 transfers the process to step S105.

[0139] In step S103, the control unit 11 determines "avatar image" as the type of image to be displayed on the user terminal 1B. After step S103, the control unit 11 transfers the process to step S104.

[0140] In step S104, the control unit 11 generates an image of the determined type. Here, the control unit 11 generates an avatar image based on the captured image. After step S104, the control unit 11 causes the process to proceed to step S105.

[0141] In step S105, the control unit 11 outputs the image of the determined type to the user terminal 1B. Here, the control unit 11 outputs the captured image or the avatar image to the user terminal 1B. When the user terminal 1B acquires the image from the user terminal 1A, it is displayed on the application screen used for the video conference (see FIG. 5).

[0142] [Specific Example 11] FIG. 16 is a flowchart corresponding to the display control process according to Specific Example 11.

[0143] In step S111, when user A performs a login operation, the control unit 11 acquires video information (for example, a captured image) from the camera device 2A.

[0144] Next, in step S112, the control unit 11 determines whether user A in the captured image is holding a small object in their hand. If user A in the captured image is holding a small object in their hand (S112: Yes), the control unit 11 causes the process to proceed to step S212. On the other hand, if user A in the captured image is not holding a small object in their hand (S112: No), the control unit 11 causes the process to proceed to step S113.

[0145] In step S212, the control unit 11 determines "captured image" as the type of image to be displayed on the user terminal 1B. After step S212, the control unit 11 causes the process to proceed to step S117.

[0146] In step S113, the control unit 11 determines whether user A in the captured image is wearing a business suit. If user A in the captured image is wearing a business suit (S113: Yes), the control unit 11 causes the process to proceed to step S131. On the other hand, if user A in the captured image is not wearing a business suit (S113: No), the control unit 11 causes the process to proceed to step S114.

[0147] In step S131, the control unit 11 determines "virtual background image" as the type of image to be displayed on the user terminal 1B. After step S131, the control unit 11 causes the process to proceed to step S116.

[0148] In step S114, the control unit 11 determines whether user A in the captured image is wearing a motion capture suit. If user A in the captured image is wearing a motion capture suit (S114: Yes), the control unit 11 causes the process to proceed to step S141. On the other hand, if user A in the captured image is not wearing a motion capture suit (S114: No), the control unit 11 causes the process to proceed to step S115.

[0149] In step S141, the control unit 11 determines "3D avatar image" as the type of image to be displayed on the user terminal 1B. After step S141, the control unit 11 causes the process to proceed to step S116.

[0150] In step S115, the control unit 11 determines "avatar image" as the type of image to be displayed on the user terminal 1B. After step S115, the control unit 11 causes the process to proceed to step S116.

[0151] In step S116, the control unit 11 generates an image of the determined type. Here, the control unit 11 generates a virtual background image, a 3D avatar image, or an avatar image based on the captured image. After step S116, the control unit 11 causes the process to proceed to step S117.

[0152] In step S117, the control unit 11 outputs the determined type of image to the user terminal 1B. Here, the control unit 11 outputs a photographed image, a virtual background image, a 3D avatar image, or an avatar image to the user terminal 1B. When the user terminal 1B acquires the image from the user terminal 1A, it is displayed on the conference application screen (see FIG. 5).

[0153] [Specific Example 12] FIG. 17 is a flowchart corresponding to the display control process according to Specific Example 12.

[0154] In step S121, the control unit 11 acquires information on the input device connected to the user terminal 1A. Also, when user A performs a login operation, the control unit 11 acquires video information (for example, a photographed image) from the camera device 2A. The input device includes, for example, the camera device 2.

[0155] Next, in step S122, the control unit 11 determines whether the camera device 2 is connected. If the camera device 2 is connected (S122: Yes), the control unit 11 transfers the process to step S221. On the other hand, if the camera device 2 is not connected (S122: No), the control unit 11 transfers the process to step S123.

[0156] In step S221, the control unit 11 determines "photographed image" as the type of image to be displayed on the user terminal 1B. After step S221, the control unit 11 transfers the process to step S126.

[0157] In step S123, the control unit 11 determines whether a device (camera device 2) capable of acquiring the feature amount of user A's face is connected. The device is, for example, a smartphone. If a device capable of acquiring the feature amount of user A's face is connected (S123: Yes), the control unit 11 causes the process to proceed to step S223. On the other hand, if a device (camera device 2) capable of acquiring the feature amount of user A's face is not connected (S123: No), the control unit 11 causes the process to proceed to step S124.

[0158] In step S223, the control unit 11 determines "digital human image" as the type of image to be displayed on the user terminal 1B. After step S223, the control unit 11 causes the process to proceed to step S125.

[0159] In step S124, the control unit 11 determines "avatar image" as the type of image to be displayed on the user terminal 1B. After step S124, the control unit 11 causes the process to proceed to step S125.

[0160] In step S125, the control unit 11 generates an image of the determined type. Here, the control unit 11 acquires the feature amount data from the camera device 2 and generates a digital human image based on the feature amount data. Also, the control unit 11 generates an avatar image based on the captured image. After step S125, the control unit 11 causes the process to proceed to step S126.

[0161] In step S126, the control unit 11 outputs the image of the determined type to the user terminal 1B. Here, the control unit 11 outputs the captured image, the digital human image, or the avatar image to the user terminal 1B. The user terminal 1B acquires the image from the user terminal 1A and displays it on the conference application screen (see FIG. 5).

[0162] As described above, the control unit 11 determines the type of image to be displayed on the conference terminal (user terminal 1) based on predetermined conditions (for example, conditions 1 to 5 shown in FIG. 4), and outputs an image (such as a captured image or a processed image) corresponding to the determined type of image to the conference terminal. Each of the above conditions is a condition corresponding to each determination process included in the above specific examples 1 to 12.

[0163] According to the above configuration, for example, in a conference where it is not appropriate to use a character image, the captured image (actual video) of the user can be displayed, and in a private conference, a processed image such as a character image can be displayed. In this way, it is possible to appropriately switch the display image displayed on the conference terminal according to the content of the conference.

[0164] The present invention is not limited to the above-described embodiments. As another embodiment, the conference system 100 may include a server connected to each user terminal 1 via the network N1. The server may be configured as, for example, a cloud server. The server may have the functions of the control unit 11 of the user terminal 1 described above. Specifically, the server acquires conference information D1 related to the conference, acquires video information corresponding to the video of the users participating in the conference, determines the type of image to be displayed on the conference terminal based on the acquired conference information, and outputs an image corresponding to the acquired video information and the determined type of image to the conference terminal. That is, the server may include processing units such as a conference acquisition processing unit 111, a video acquisition processing unit 112, a determination processing unit 113, a generation processing unit 114, and an output processing unit 115 (see FIG. 2).

[0165] Further, the server may store the conference information D1 (see FIG. 3) and the image type information D2 (see FIG. 4).

[0166] Further, the server may store a conference application. In this case, each user terminal 1 executes a browser process for accessing the server and operating the conference application.

[0167] As described above, the display control system of the present invention may be configured by the user terminal 1 alone, or may be configured by the server alone, or may be configured by combining the user terminal 1 and the server.

Explanation of Signs

[0168] 1: User terminal 2: Camera device 3: Schedule server 11: Control unit 12: Storage unit 13: Operation display unit 14: Communication unit 100: Conference system 111: Conference acquisition processing unit 112: Video acquisition processing unit 113: Decision processing unit 114: Generation processing unit 115: Output processing unit

Claims

1. A display control system for causing a conference terminal to display an image corresponding to a user in a conference in which the user participates, comprising: a conference acquisition processing unit that acquires conference information regarding the conference; a video acquisition processing unit that acquires video information corresponding to a video of the user taken by a camera during the conference; a determination processing unit that determines the type of image to be displayed on the conference terminal based on the conference information acquired by the conference acquisition processing unit; an output processing unit that outputs, to the conference terminal, an image corresponding to the video information acquired by the video acquisition processing unit and the type of image determined by the determination processing unit; A display control system comprising:

2. The determination processing unit determines the type of the image based on at least any one of information on conference time, conference participants, content of the conference, importance of the conference, and the location where the user participating in the conference is located, which is included in the conference information. The display control system according to claim 1.

3. The determination processing unit determines the type of the image based on at least any one of information on the type of conference application used in the conference, the type of microphone speaker device used in the conference, and the communication means connected to the conference. The display control system according to claim 2.

4. The determination processing unit determines, as the type of the image, a captured image that is a video of the user taken by a camera or a processed image obtained by processing the video of the user taken by the camera, based on the conference information. The display control system according to any one of claims 1 to 3.

5. The processed image includes at least any one of a character image, an illustration image, a digital human image, and a virtual background image. The display control system according to claim 4.

6. The display control system according to claim 5, further comprising a generation processing unit that generates a processed image corresponding to the type of the image determined by the determination processing unit from the video information acquired by the video acquisition processing unit. The display control system according to claim 5.

7. The video acquisition processing unit acquires a captured image that is a video of the user taken by a camera or feature amount data extracted based on the video of the user taken by the camera. The display control system according to any one of claims 1 to 6.

8. A display control method for causing a conference terminal to display an image corresponding to a user in a conference in which the user participates, comprising: one or more processors perform: a conference acquisition step of acquiring conference information regarding the conference; a video acquisition step of acquiring video information corresponding to video of the user participating in the conference; a determination step of determining a type of image to be displayed on the conference terminal based on the conference information acquired in the conference acquisition step; an output step of outputting, to the conference terminal, an image corresponding to the video information acquired in the video acquisition step and the type of image determined in the determination step; A display control method for performing the above steps.

9. A display control program for causing a conference terminal to display an image corresponding to a user in a conference in which the user participates, comprising: a conference acquisition step of acquiring conference information regarding the conference; a video acquisition step of acquiring video information corresponding to video of the user participating in the conference; a determination step of determining a type of image to be displayed on the conference terminal based on the conference information acquired in the conference acquisition step; an output step of outputting, to the conference terminal, an image corresponding to the video information acquired in the video acquisition step and the type of image determined in the determination step; A display control program for causing one or more processors to perform the above steps.

Images