Control system, control method, and computer program

Abstract

To realize a screen display more convenient for a user in a system which provides a plurality of videos.SOLUTION: An embodiment of the present invention includes: a communication unit for receiving data of a plurality of videos provided from another device; and a control unit for allocating the videos received by the communication unit to a plurality of partial regions obtained by dividing the display screen and displaying the videos. The control unit determines allocation of the partial regions and the videos so that the sum of the areas of the display regions of the videos become larger.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for receiving and displaying a plurality of moving images online. [Background technology]

[0002] Conventionally, systems for conducting conversations and conferences via a network have been proposed. In these systems, facial images of participants captured by a camera and images of windows shared by participants are generally displayed in partial areas provided for each user. For example, Patent Document 1 describes a method of extracting an image of a captured facial area of ​​a user and displaying it with a blurred background. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-213013 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventionally, there are systems in which, when multiple members participate, a partial area is provided for each of one or more participating members, and faces and moving images are displayed. There are also systems in which the screen layout of the partial area for each member can be selected from multiple predefined screen layouts. For example, there is a screen layout in which the partial areas of one or more users are set to the same size, and the image of each user is displayed in the partial area. As another example, there is a screen layout in which the partial area of ​​a representative user (for example, the user who is speaking at that time) is set larger than the partial areas of other users. As another example, there is a screen layout in which a partial area for displaying an image of a shared window or desktop is set larger than the partial area of ​​the user. However, sufficient convenience has not always been realized. Such a problem is not necessarily limited to systems for conversations and meetings, but is a common problem in systems that display multiple moving images provided by other users or other systems.

[0005] In view of the above circumstances, an object of the present invention is to provide a technique that enables a screen display that is more convenient for users to see in a system in which a plurality of moving images are provided. [Means for solving the problem]

[0006] One aspect of the present invention is a control system comprising a communication unit that receives data of multiple moving images provided from another device, and a control unit that allocates and displays the moving images received by the communication unit to multiple partial areas obtained by dividing a display screen into multiple parts, wherein the control unit determines the allocation of the partial areas to the moving images so as to maximize the sum of the areas of the display areas of the multiple moving images.

[0007] One aspect of the present invention is the above-mentioned control system, wherein the control unit determines the allocation of the partial regions and the moving images based on an aspect ratio of each moving image and an aspect ratio of each partial region.

[0008] One aspect of the present invention is the above-mentioned control system, wherein the control unit determines the allocation of the partial regions to the moving images so that the order of the aspect ratios of each moving image matches the order of the aspect ratios of each partial region.

[0009] One aspect of the present invention is a control method in which a computer receives data of a plurality of moving images provided from another device, and the computer allocates and displays the received moving images in a plurality of partial areas obtained by dividing a display screen into a plurality of parts, and the computer determines the allocation of the partial areas to the moving images so as to maximize the sum of the areas of the display areas of the plurality of moving images.

[0010] One aspect of the present invention is a computer program for operating a computer as a control system comprising a communication unit that receives data for multiple moving images provided from another device, and a control unit that allocates and displays the moving images received by the communication unit to multiple partial areas obtained by dividing a display screen into multiple parts, the control unit determining the allocation of the partial areas to the moving images so as to maximize the sum of the areas of the display areas for the multiple moving images. Effect of the Invention

[0011] According to the present invention, in a system in which a plurality of moving images are provided, it is possible to realize a screen display that is more convenient for the user. [Brief description of the drawings]

[0012] [Figure 1] 1 is a schematic block diagram showing a system configuration of a conversation control system 100 of the present invention. [Diagram 2] 2 is a schematic block diagram showing a specific example of the functional configuration of a user terminal 10. FIG. [Diagram 3] 2 is a schematic block diagram showing a specific example of the functional configuration of the conversation control device 20. FIG. [Figure 4]1A and 1B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10. [Diagram 5] 1A and 1B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10. [Figure 6] 2A and 2B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10. [Figure 7] 1A and 1B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10. [Figure 8] 2A and 2B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10. [Figure 9] 2A and 2B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10. [Figure 10] 2A and 2B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10. [Figure 11] 1A and 1B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10. [Figure 12] 2A and 2B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10. [Figure 13] 13 is a diagram showing a specific example of a state in which screen sharing is being performed in a reduced partial area 51. FIG. [Figure 14] 13 is a diagram showing a specific example of a state in which screen sharing is being performed in a reduced partial area 51 and a partial area 52. FIG. [Figure 15] 13 is a diagram showing an example of the configuration of a partial region 52. FIG. [Figure 16] 1 is a diagram showing an example of an arrangement of a plurality of partial regions 52 displayed on a display unit 13 of a user terminal 10. FIG. [Figure 17] 1 is a diagram showing an example of an arrangement of a plurality of partial regions 52 displayed on a display unit 13 of a user terminal 10. FIG. [Figure 18] 1 is a diagram showing an example of an arrangement of a plurality of partial regions 52 displayed on a display unit 13 of a user terminal 10. FIG. [Figure 19] 1 is a diagram showing an example of an arrangement of a plurality of partial regions 52 displayed on a display unit 13 of a user terminal 10. FIG. [Figure 20]1 is a diagram showing an example of an arrangement of a plurality of partial regions 52 displayed on a display unit 13 of a user terminal 10. FIG. [Figure 21] FIG. 13 is a diagram showing a specific example of a pattern of the shape of a provided moving image. [Figure 22] FIG. 11 is a diagram showing a specific example of placement candidate information. [Figure 23] FIG. 11 is a diagram showing a specific example of placement candidate information. [Figure 24] FIG. 11 is a diagram showing a specific example of placement candidate information. [Diagram 25] FIG. 13 is a diagram showing a specific example of arrangement candidate information according to a modified example. [Figure 26] FIG. 9 is a diagram illustrating an outline of a hardware configuration example of an information processing device 900 applied to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, specific configuration examples of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing a system configuration of a conversation control system 100 of the present invention. The conversation control system 100 is a specific example of a system in which a plurality of moving images are provided. The conversation control system 100 is a system in which users operating user terminals 10 can have a conversation with each other via a network 90. ​​The conversation control system 100 includes a plurality of user terminals 10 and a conversation control device 20. The plurality of user terminals 10 and the conversation control device 20 are communicatively connected via the network 90. ​​The network 90 may be a network using wireless communication or a network using wired communication. The network 90 may be configured by combining a plurality of networks.

[0014] 2 is a schematic block diagram showing a specific example of the functional configuration of the user terminal 10. The user terminal 10 is configured using information devices such as a smartphone, a tablet, a personal computer, a portable game machine, a stationary game machine, a dedicated device, etc. The user terminal 10 includes a communication unit 11, an operation unit 12, a display unit 13, an audio input unit 14, an audio output unit 15, an imaging unit 16, a storage unit 17, and a control unit 18.

[0015] The communication unit 11 is a communication device. The communication unit 11 may be configured as, for example, a network interface. The communication unit 11 performs data communication with other devices via a network 90 in response to control by the control unit 18. The communication unit 11 may be a device that performs wireless communication or a device that performs wired communication.

[0016] The operation unit 12 is configured using existing input devices such as a keyboard, a pointing device (mouse, tablet, etc.), buttons, and a touch panel. The operation unit 12 is operated by the user when inputting the user's instructions to the user terminal 10. The operation unit 12 may be an interface for connecting the input device to the user terminal 10. In this case, the operation unit 12 inputs an input signal generated in the input device in response to the user's input to the user terminal 10. The operation unit 12 may be configured using a microphone and a voice recognition device. In this case, the operation unit 12 recognizes the words spoken by the user and inputs character string information of the recognition result to the user terminal 10. In this case, the voice recognition process may be executed by the control unit 18. In this case, the microphone constituting the operation unit 12 may be configured integrally with the voice input unit 14 described later. The operation unit 12 may be configured in any manner as long as it is configured to input the user's instructions to the user terminal 10.

[0017] The display unit 13 is an image display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 13 displays image data used in the conversation control system 100. The display unit 13 may be an interface for connecting an image display device to the user terminal 10. In this case, the display unit 13 generates a video signal for displaying the image data and outputs the video signal to the image display device connected to itself.

[0018] The voice input unit 14 is configured using a microphone. The voice input unit 14 may be configured as a microphone itself, or may be configured as an interface for connecting a microphone as an external device to the user terminal 10. The microphone acquires the speech of a user having a conversation. The voice input unit 14 outputs data of the voice acquired by the microphone to the control unit 18.

[0019] The audio output unit 15 is configured using an audio output device such as a speaker, headphones, or earphones. The audio output unit 15 may be configured as the audio output device itself, or may be configured as an interface for connecting the audio output device to the user terminal 10 as an external device. It is desirable for the audio output device to output audio so that the user having a conversation can hear the audio. The audio output unit 15 outputs audio corresponding to an audio signal output by the control unit 18.

[0020] The imaging unit 16 is configured using an imaging device such as a camera. The imaging unit 16 may be an interface for connecting an imaging device such as a camera to the user terminal 10. The imaging unit 16 is preferably installed in a position where it can capture an image of the user's face. In this case, the imaging unit 16 captures an image of the user's face and generates still image or video data (hereinafter referred to as "captured image data"). The imaging unit 16 outputs the generated captured image data to the control unit 18. The captured image data is one aspect of a provided video. The provided video is a video provided from the user terminal 10 or an information device to another user terminal 10. Specific examples of the provided video include a video (captured image data) captured by the imaging unit 16 of the user terminal 10 and a video showing a screen shared in the user terminal 10 (hereinafter referred to as a "shared screen"). The shared screen is, for example, a window selected by the user, a window displaying a selected file, a selected desktop, or the like. In the video showing the shared screen, operations performed by the user on the screen may be reflected at any time. For example, when a user operates the position of a cursor, the display position of the cursor changes on the shared screen in response to the operation.

[0021] The storage unit 17 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 17 stores data used by the control unit 18. The storage unit 17 may function as, for example, a user information storage unit 171. The user information storage unit 171 stores information about the user who operates the user terminal 10 (hereinafter referred to as "user information"). The user information may include, for example, the user's handle name, the user's icon data, and the user's attribute information. The handle name is the name (nickname or real name) that the user uses when having a conversation in the conversation control system 100. The icon data is image data that the user uses when having a conversation in the conversation control system 100. The attribute information may include, for example, information about the user's age, sex, and affiliation.

[0022] The control unit 18 is configured using a processor such as a CPU (Central Processing Unit) and a memory. The control unit 18 functions as a display control unit 181, a conference control unit 182, and a conversation control unit 183 by the processor executing a program. All or part of the functions of the control unit 18 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The above program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a semiconductor storage device (for example, an SSD: Solid State Drive), or a storage device such as a hard disk or a semiconductor storage device built into a computer system. The above program may be transmitted via an electric communication line.

[0023] The display control unit 181 receives display information from the conversation control device 20 via the communication unit 11. The display control unit 181 generates an image signal based on the acquired display information and displays it on the display unit 13. At this time, the display control unit 181 performs display control within the display area of ​​the application in which the application of the conversation control system 100 is running. For example, if the application of the conversation control system 100 is a web browser, display control is performed within the window of the web browser. If the web browser is displayed full screen, display control is performed on the entire display screen of the display unit 13.

[0024] The display information may be, for example, image data indicating the image itself to be displayed. In this case, the entity that generates the image data (image data generation unit) is the display information generation unit 233 of the conversation control device 20. The display information may be, for example, data indicating information required to generate the image to be displayed (for example, the number of divided areas to be displayed, the shape of the image area to be displayed in each divided area, the position, size and display mode of the conference room, and information indicating the position and icon image of the user). In this case, the display control unit 181 generates image data to be displayed on the display unit 13 based on the display information. In this case, the entity that generates the image data (image data generation unit) is the display control unit 181.

[0025] The conference control unit 182 performs control related to a conference virtually established in the conversation control device 20. For example, when a user instructs to set up a new conference room by operating the operation unit 12, the conference control unit 182 performs processing for setting up a new conference room. When a user of the own device (user terminal 10) sets up to transmit captured image data to another user terminal 10, the conference control unit 182 transmits captured image data captured by the imaging unit 16 to the other device. The other device as the transmission destination may be, for example, the conversation control device 20. In this case, the captured image data is transmitted via the conversation control device 20 to the other user terminal 10 connected to the same conference room as the own device (user terminal 10). The other device as the transmission destination may be the other user terminal 10 connected to the same conference room as the own device (user terminal 10).

[0026] The conversation control unit 183 controls a conversation between multiple users participating in the conversation control system 100. A conversation between multiple users may be, for example, a conversation between multiple users participating in a conference room (hereinafter referred to as a "conference room conversation"). When the user of the own device (user terminal 10) participates in a conversation, the conversation control unit 183 controls a conversation between multiple users including the user of the own device. In this case, the conversation control unit 183 transmits, for example, voice data input from the voice input unit 14 to the conversation control device 20 via the communication unit 11. When the conversation control unit 183 receives voice data from the conversation control device 20, it outputs the received voice data from the voice output unit 15. When the user of the own device (user terminal 10) does not participate in the conversation, the conversation control unit 183 controls a conversation between multiple users not including the user of the own device. In this case, the conversation control unit 183 receives voice data of a conversation between other users and outputs it from the voice output unit 15. The voice data output from the voice output unit 15 may be received from another user terminal 10 via the conversation control device 20, or may be received directly from another user terminal 10 connected in accordance with the control of the conversation control device 20.

[0027] 3 is a schematic block diagram showing a specific example of a functional configuration of the conversation control device 20. The conversation control device 20 is configured using an information processing device such as a personal computer or a server device. The conversation control device 20 includes a communication unit 21, a storage unit 22, and a control unit 23.

[0028] The communication unit 21 is a communication device. The communication unit 21 may be configured as, for example, a network interface. The communication unit 21 performs data communication with other devices via the network 90 in accordance with the control of the control unit 23. The communication unit 21 may be a device that performs wireless communication or a device that performs wired communication.

[0029] The storage unit 22 is configured using a storage device such as a magnetic hard disk device, a semiconductor storage device, etc. The storage unit 22 stores data used by the control unit 23. The storage unit 22 may function as a conference room information storage unit 221, for example.

[0030] The conference room information storage unit 221 stores information about conference rooms (hereinafter referred to as "conference room information"). A conference room is a virtual room set up in the conversation control system 100 for users to have conversations. The conference room information may include, for example, information indicating the name set for the conference room. The conference room information may further include information indicating the users currently in the conference room.

[0031] The control unit 23 is configured using a processor such as a CPU and a memory. The control unit 23 functions as a user control unit 231, a conference room control unit 232, a display information generating unit 233, and a conversation control unit 234 by the processor executing a program. All or part of the functions of the control unit 23 may be realized using hardware such as an ASIC, a PLD, or an FPGA. The above program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a semiconductor storage device (e.g., SSD), or a storage device such as a hard disk or a semiconductor storage device built into a computer system. The above program may be transmitted via an electric communication line.

[0032] The user control unit 231 performs control processing related to users. For example, the user control unit 231 may perform processing for logging in (for example, authentication processing) for a user who accesses the conversation control device 20.

[0033] The conference room control unit 232 performs control processing related to the conference room. For example, when the conference room control unit 232 receives an instruction from the user terminal 10 to set up a new conference room, the conference room control unit 232 may generate conference room information based on the received information and register it in the conference room information storage unit 221. When a predetermined operation is performed by a user to participate in a conference room, the conference room control unit 232 performs processing to allow the user to participate in the conference room if a predetermined condition is satisfied. For example, the conference room control unit 232 updates the conference room information storage unit 221 to register that a new user has participated in the conference room.

[0034] The display information generating unit 233 generates information (display information) required to generate an image to be displayed on the user terminal 10. The display information may include, for example, the number of divided areas, the shape of the image area displayed in each divided area, information about the currently installed conference room, and information about the user of each user terminal 10 accessing the conversation control device 20. The display information generating unit 233 transmits the generated display information to the user terminal 10.

[0035] The conversation control unit 234 receives voice data from the user terminal 10. The conversation control unit 234 generates voice data to be output to each user terminal 10 (hereinafter referred to as "conversation voice data") and transmits the conversation voice data to each user terminal 10. The conversation control unit 234 may generate conversation voice data based on, for example, what kind of conversation the user of each user terminal 10 is participating in. For example, the conversation control unit 234 may transmit, to each user's user terminal 10, conversation voice data including voices received from the user terminals 10 of all users participating in the conference room in which that user is present.

[0036] Next, the relationship between the images displayed on the user terminal 10 and the conversation voice data output when each image is displayed will be described.

[0037] FIG. 4 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In FIG. 4, a conference room area image 30, a conversation area 31, a listening area 32, an icon image 41, and an operation area image 501 are displayed on the display unit 13. An image of a virtual space in which a conference room can be set up is displayed on the display unit 13. The virtual space is mainly composed of a conference room area and an area outside the conference room. The conference room area is displayed as a conference room area image 30. The area outside the conference room (hereinafter referred to as the "outdoor area") is the area represented by the symbol 91, and is an area in which icon images 41 of users who are not in the conference room are displayed.

[0038] The conference room area image 30 includes a conversation area 31 and a listening area 32. A user who has an icon image 41 displayed in the conversation area 31 can converse with other users who have an icon image 41 displayed in the conversation area 31 of the same conference room. That is, the voices of the users who have an icon image 41 displayed in the conversation area 31 are transmitted and received between each other. In the following description, entering a conference room means that the icon image 41 is displayed in the conversation area 31. That is, entering a conference room means that a user is in a state in which he or she can converse with other users in the conference room.

[0039] A user whose icon image 41 is displayed in the listening area 32 (hereinafter referred to as a "listening user") can listen to the conversation of a user whose icon image 41 is displayed in the conversation area 31 of the conference room (a user who has entered the conference room). However, the listening user cannot speak in the conference room. In other words, the voice uttered by the listening user is not output in the user terminals 10 of other users who have entered the conference room, or in the user terminals 10 of other listening users in the conference room.

[0040] The conference room area image 30 is displayed as a geometric image showing an area of ​​a predetermined size. In the example of this embodiment, the conference room area image 30 is displayed as a circular image. The name of the conference room 311 is displayed near the center of the conference room area image 30. In the example of FIG. 4, the name of the conference room is "Chat." By displaying the name of the conference room, even a user located in the outdoor area 91 can roughly guess what kind of conversation is taking place in the conference room without entering the conference room.

[0041] An entry button 312 and a details button 313 may be displayed inside the conference room area image 30. The entry button 312 is a button for requesting entry into the conference room. In response to the entry button 312 being operated by a user located in the outdoor area 91, the conference room control unit 232 determines that the user is requesting entry into the conference room. In response to the request, the conference room control unit 232 determines whether or not to permit entry, and controls the user's entry in accordance with the determination result.

[0042] The details button 313 is a button for displaying detailed information about the conference room. In response to the operation of the details button 313, a details display area is displayed on the screen. In the details display area, detailed information about the conference room, and buttons and input frames for performing operations on the conference room are displayed. For example, the name of the conference room, a button for changing the name of the conference room, an input frame for entering the changed name of the conference room, etc. may be displayed.

[0043] The icon image 41 is an image that indicates the virtual position of the user and information about the user. The icon image 41 may be displayed according to icon data included in the user information, for example.

[0044] 5 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In the example of FIG. 5, icon images 41 (41_1 and 41_2) of two users and a conference room area image 30 of one conference room are displayed. No user has yet participated in the conference room indicated by the conference room area image 30. The two users have not participated in any conference room. Therefore, the icon image 41_1 and the icon image 41_2 are displayed in the outside area 91, not inside the conference room area image 30.

[0045] A user who wishes to participate in a conference room can do so (enter) the conference room by performing a specific operation. For example, a user can join the conference room by operating the entry button 312 in the conference room area image 30 in which the user wishes to participate. Note that the operation for entering the room does not need to be limited to the above. If an entry restriction is set for a conference room, only users who are permitted to enter the conference room according to the entry restriction can enter the conference room.

[0046] Fig. 6 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In the example of Fig. 6, one user is participating in a conference room with the theme of chatting. Therefore, an icon image 41 (41_2) of one user is displayed inside the conference room area image 30 showing the conference room with the theme of chatting. At this point, there are no other users participating in the same conference room as the user of icon image 41_2. Therefore, the user of icon image 41_2 cannot talk with other users in the conference room with the theme of chatting.

[0047] When a user of the icon image 41_1 in the outdoor area 91 performs an operation to participate in a conference room with a chat theme, the users participating in the conference room are notified that there is a user who wishes to participate. For example, a predetermined sound (e.g., a door knocking sound) may be output from the user terminal 10 of each user. The users to be notified may be all users participating in the conference room, or only a predetermined portion of users (e.g., the user who is the administrator of the conference room).

[0048] If a predetermined condition is met after the notification, the user of the icon image 41_1 can participate in the conference room with the theme of chatting. The predetermined condition may be, for example, simply that a predetermined time (e.g., 1 second, 3 seconds, 5 seconds, etc.) has passed, or that participation has been permitted by a predetermined user (e.g., the administrator or participant of the conference room), or that a participation condition preset for the conference room has been met. Also, a predetermined condition may not be set. In that case, all users who wish to participate can participate in the conference room (e.g., a conference room with the theme of chatting).

[0049] FIG. 7 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In the example of FIG. 7, two users are participating in a conference room with the theme of chatting. Therefore, icon images 41 (41_1 and 41_2) of the two users are displayed inside the conference room area image 30 showing the conference room with the theme of chatting. At this point, the user of the icon image 41_1 and the user of the icon image 41_2 are participating in the same conference room. Therefore, the user of the icon image 41_1 and the user of the icon image 41_2 can have a conversation in the conference room with the theme of chatting. In other words, users participating in the same conference room can have a conversation using the user terminal 10.

[0050] More specifically, it is as follows. The user terminal 10 of a user participating in a conference room receives and outputs voice data spoken by other users participating in the same conference room. In addition, the user terminal 10 of a user participating in a conference room transmits the voice data spoken by that user to the conversation control device 20. This transmission causes the voice data to be output in the user terminals 10 of the other users participating in the same conference room. When a user of icon image 41_3 in the outdoor area 91 performs an operation to participate in a conference room with a chat theme, after a predetermined condition is met, the user of icon image 41_3 can participate in the conference room with a chat theme.

[0051] FIG. 8 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In the example of FIG. 8, three users participate in a conference room with the theme of chatting. Therefore, icon images 41 (41_1, 41_2, and 41_3) of the three users are displayed inside the conference room area image 30 showing the conference room with the theme of chatting. At this point, the user of the icon image 41_1, the user of the icon image 41_2, and the user of the icon image 41_3 participate in the same conference room. Therefore, the user of the icon image 41_1, the user of the icon image 41_2, and the user of the icon image 41_3 can have a conversation in the conference room with the theme of chatting. The same applies when four or more users participate in the same conference room.

[0052] Fig. 9 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In the example of Fig. 9, six users are participating in a conference room with the theme of chatting. In the example of Fig. 9, the user terminal 10 is operating in a video conversation mode. In the video conversation mode, a reduced partial area 51 is displayed for each user participating in the same conference room (the conference room with the theme of chatting in Fig. 9). The reduced partial area 51 is an area capable of displaying a video image (provided video image) provided from the user terminal 10 of the user associated with that reduced partial area 51.

[0053] Types of video images other than those described above may be displayed as provided video images in the reduced partial areas 51. In the example of Fig. 9, provided video images including the faces of the users captured by the imaging unit 16 of the user terminal 10 are displayed in each reduced partial area 51.

[0054] In the example of Fig. 9, a partial list area 50 is provided in a part of the screen of the display unit 13 (for example, near the right end). A predetermined number or less (seven or less in the example of Fig. 9) of reduced partial areas 51 are arranged and displayed in the partial list area 50. In the example of Fig. 9, reduced partial areas 51 of all five users participating in a conference room with the theme of casual conversation are displayed.

[0055] If there are more than a predetermined number of participants, the reduced partial areas 51 for the number of participants exceeding the predetermined number are not displayed. However, if the number of reduced partial areas 51 displayed in the part list area 50 is reduced for some reason, the reduced partial areas 51 of users that were not displayed until then may be displayed in the part list area 50. The reduced partial areas 51 displayed in the part list area 50 may be displayed in a scrolled manner. For example, in the screen of FIG. 9, when the display of the part list area 50 is scrolled downward, the reduced partial area 51 at the top end is no longer displayed, and the second to seventh reduced partial areas 51_ that were displayed until then are each moved up by one, and the eighth reduced partial area 51 that was not displayed until then may be displayed at the bottom end.

[0056] FIG. 10 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In the example of FIG. 10, an image of a user displayed in the reduced partial area 51_1 is displayed in the partial area 52_1. When a user performs a predetermined operation (e.g., touch, click, etc.) on the reduced partial area 51 displayed on the display unit 13 of the user terminal 10, the operated reduced partial area 51 is displayed as the partial area 52. The partial area 52 has a larger area than the reduced partial area 51. In the partial area 52, the same image as that displayed in the reduced partial area 51 is enlarged and displayed. For example, one or more partial areas 52 are displayed in an area other than the partial list area 50. The reduced partial area 51 corresponding to the user displayed in the partial area 52 may be deleted from the partial list area 50. In the example of FIG. 10, an operation is performed on the reduced partial area 51_1 originally displayed in the partial list area 50, so that the partial area 52_1 is displayed. At that time, the reduced partial area 51_1 is deleted from the partial list area 50. As a result of this deletion, the display positions of the reduced partial regions 51_2 to 51_5 are moved upward one by one.

[0057] FIG. 11 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In the example of FIG. 11, the images of the users displayed in the reduced partial areas 51_1 and 51_4 are displayed in the partial areas 52_1 and 52_4, respectively. Such a display is displayed as a result of, for example, performing an operation on the reduced partial areas 51_1 and 51_4 originally displayed in the partial list area 50. When a plurality of partial areas 52 are displayed, the sizes of the partial areas 52 may be displayed as the same size, or some or all of the partial areas 52 may be displayed as being different in size from the sizes of the other partial areas 52. In the example of FIG. 11, the partial areas 52 are displayed as the same size.

[0058] Fig. 12 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In the example of Fig. 12, as a result of an operation being performed on all reduced partial areas 51 displayed in the partial list area 50, partial areas 52 are displayed for all users. When partial areas 52 are displayed for all users, the partial list area 50 does not need to be provided. In this case, the area that was used as the partial list area 50 is also used as a display area for the partial areas 52. This makes it possible to display the partial areas 52 more widely.

[0059] As shown in Figs. 10 to 12, a partial area individual operation button and a partial area whole operation button 505 are displayed on the display unit 13. The partial area individual operation button includes an individual enlargement button 53 and a reduction button 54. The partial area individual operation button is displayed for each partial area 52. One partial area whole operation button 505 is displayed on the screen of the display unit 13. When the individual enlargement button 53 is operated, the partial area 52 is enlarged and displayed as an area larger than the other partial areas 52. At this time, the other partial areas 52 do not have to be displayed. When the partial area 52 is enlarged and displayed, a back button is also displayed. When the back button is operated, the display state returns to the state immediately before the individual enlargement button 53 was operated. When the reduction button 54 is operated, the partial area 52 for that user is erased, and a reduced partial area 51 is displayed in the partial list area 50.

[0060] The partial area whole operation button 505 may include a full screen button, a close all button, and a hide button. These buttons may be displayed by operating the partial area whole operation button 505, for example. When the full screen button is operated, full screen display is performed with the partial area 52 still displayed. When the close all button is operated, all partial areas 52 are erased, and all reduced partial areas 51 are displayed in the partial list area 50 as much as possible. That is, the display returns to a state in which none of the partial areas 52 is displayed. When the hide button is operated, the display of all partial areas 52 is temporarily erased, and the conference room area image 30 originally displayed in the background is displayed. In this display state, operations on the conference room area image 30 and the icon image 41 are valid. On the other hand, when even one partial area 52 is displayed, operations on the conference room area image 30 and the icon image 41 are invalid. When the hide button is operated, the back button is also displayed. When the back button is operated, the display returns to the state immediately before the hide button was operated.

[0061] 13 is a diagram showing a specific example of a state in which screen sharing is being performed in the reduced partial area 51. In the reduced partial area 51_10, the reduced partial area 51_11, and the reduced partial area 51_12, as a result of a screen sharing operation being performed by a user corresponding to the reduced partial area 51, a moving image of the shared screen is displayed as a provided moving image. In this manner, screen sharing may be performed by a plurality of users at the same time.

[0062] Fig. 14 is a diagram showing a specific example of a state in which screen sharing is being performed in the reduced partial areas 51 and the partial areas 52. In the example of Fig. 14, as a result of a predetermined operation being performed in the reduced partial areas 51_10 and the reduced partial areas 51_12, the images of the shared screens displayed in the reduced partial areas 51_10 and 51_12 are displayed in the partial areas 52_10 and 52_12, respectively. In this manner, screen sharing being performed in parallel by different users may be displayed in a plurality of partial areas 52.

[0063] Next, a description will be given of a process for determining the display contents of the user terminal 10. Specific examples of the display contents to be determined include information indicating the size and display position of the partial regions 52, allocation of provided moving images to each partial region 52, and the size and display position of the image display region 56 in each partial region 52. This determination process may be performed by, for example, the display control unit 181 of each user terminal 10, or by the display information generation unit 233 of the conversation control device 20.

[0064] In the following description, an example will be described in which the display content determination process is performed by the display control unit 181 of each user terminal 10, but the process is similar when performed by the display information generation unit 233. The display control unit 181 that executes the following process may be executed by, for example, an application that is installed in the user terminal 10 in advance. The display control unit 181 that executes the following process may also be realized by downloading an application for executing the following process from a server such as the conversation control device 20 and executing the application every time an information processing device operated by a user operates as the user terminal 10 of the conversation control system 100.

[0065] FIG. 15 is a diagram showing an example of the configuration of a partial area 52. An image display candidate area 55 is provided inside the partial area 52. The image display candidate area 55 indicates the largest area of ​​the partial area 52 in which the provided moving image can be displayed. In each partial area 52, a provided moving image of the same shape (size) as the image display candidate area 55 can be displayed as the maximum image size. The image display candidate area 55 may be the same shape and size as the partial area 52, or may be an area smaller than the partial area 52. In the example of FIG. 15, the height of the image display candidate area 55 is H1 and the width is W1.

[0066] An image display area 56 is set inside the image display candidate area 55. The image display area 56 is an area for displaying the provided moving image transmitted from the user or device assigned to the partial area 52. In FIG. 15, the height of the image display area 56 is H2, and the width is W2. The height H2 and width W2 of the image display area 56 are determined based on the height H3 and width W3 of the provided moving image transmitted from the user or device assigned to the partial area 52. W2 and H2 are determined so that the ratio of W2 and H2 is the same as the ratio of W3 and H3. In other words, the shape of the image display area 56 is determined so that the aspect ratio of the image display area 56 is the same as the aspect ratio of the provided moving image. For example, the image display area 56 may be determined so that the area of ​​the image display area 56 for displaying the provided moving image assigned to the partial area 52 is maximized in each partial area 52. The display control unit 181 may obtain the aspect ratio of the determined common image by any method. For example, the display control unit 181 may receive a provided video from another device and obtain the aspect ratio based on the data of the received provided video. For example, the display control unit 181 may be configured to transmit information indicating the aspect ratio of a video (provided video) to another device when providing the video from its own device to the other device. In this case, the display control unit 181 of each device obtains information indicating the aspect ratio from the device that is the provider of the provided video.

[0067] 16 is a diagram showing an example of the arrangement of a plurality of partial regions 52 displayed on the display unit 13 of the user terminal 10. A plurality of arrangement candidates are determined according to the number of partial regions 52 displayed on the display unit 13. The arrangement candidates are expressed, for example, as [m1,...,mn:horizontal] and [m1,...,mn:vertical], where m and n are integers equal to or greater than 1. In [m1,...,mn:horizontal], the screen of the display unit 13 is divided horizontally into n rows, and the partial regions 52 are arranged so that, counting from the top, m1 partial regions 52 are arranged in the first row, m2 partial regions 52 are arranged in the second row, and mn partial regions 52 are arranged in the nth row, with the partial regions 52 being of the same size in every row. In [m1,...,mn: vertical], the screen of display unit 13 is divided vertically into n columns, and counting from the left, m1 partial regions 52 are arranged in the first column, m2 partial regions 52 in the second column, and mn partial regions 52 in the nth column, with all partial regions 52 being the same size in each column.

[0068] In FIG. 16, n=3, m1=1, m2=2, m3=2, and the partition direction is horizontal. That is, the arrangement candidate in FIG. 16 is [1,2,2: horizontal]. Since n=3, the display unit 13 is divided into three stages. m1=1 partial regions 52 (52_a) are arranged in the first stage, m2=2 partial regions 52 (52_b, 52_c) are arranged in the second stage, and m3=2 partial regions 52 (52_d, 52_e) are arranged in the third stage. In this embodiment, in the case of a horizontal arrangement candidate, when the screen of the display unit 13 is divided into n stages, the partial regions are equally divided so that the vertical length of each stage is the same. Also, in this embodiment, the partial regions 52 having larger areas are arranged from the top. However, the vertical length of each stage and the relationship between the area and arrangement of each partial region 52 are merely examples and do not need to be limited to those described above.

[0069] FIG. 17 is a diagram showing an example of the arrangement of a plurality of partial regions 52 displayed on the display unit 13 of the user terminal 10. In FIG. 17, n=3, m1=1, m2=2, m3=2, and the partition direction is vertical. That is, the arrangement candidate in FIG. 17 is [1,2,2: vertical]. Since n=3, the display unit 13 is divided into three columns. In the first column, m1=1 partial regions 52 (52_a) are arranged, in the second column, m2=2 partial regions 52 (52_b, 52_c) are arranged, and in the third column, m3=2 partial regions 52 (52_d, 52_e) are arranged. In this embodiment, in the case of a vertical arrangement candidate, when the screen of the display unit 13 is divided into n columns, the partial regions are equally divided so that the horizontal lengths of the columns are the same. In this embodiment, the partial regions 52 having larger areas are arranged from the left. However, the horizontal length of each row and the relationship between the area and arrangement of each partial region 52 are merely examples, and are not limited to those described above.

[0070] In Fig. 18, n=2, m1=2, m2=3, and the partition direction is horizontal. That is, the arrangement candidate in Fig. 18 is [2,3: horizontal]. Since n=2, the display unit 13 is divided into two rows. In the first row, m1=2 partial regions 52 (52_a, 52_b) are arranged, and in the second row, m2=3 partial regions 52 (52_c, 52_d, 52_e) are arranged.

[0071] In Fig. 19, n = 2, m1 = 1, m2 = 4, and the partition direction is horizontal. That is, the arrangement candidate in Fig. 19 is [1, 4: horizontal]. Since n = 2, the display unit 13 is divided into two rows. In the first row, m1 = 1 partial regions 52 (52_a) are arranged, and in the second row, m2 = 4 partial regions 52 (52_b, 52_c, 52_d, 52_e) are arranged.

[0072] In Fig. 20, n = 1, m1 = 5, and the partition direction is horizontal. That is, the arrangement candidate in Fig. 20 is [5: horizontal]. Since n = 1, the display unit 13 is configured in one row and is not divided. m1 = 5 partial regions 52 (52_a, 52_b, 52_c, 52_d, 52_e) are arranged in the first row. This arrangement candidate is substantially the same as [1, 1, 1, 1, 1: vertical].

[0073] When the number of partial regions 52 is five, various patterns of arrangement candidates may exist, such as [5: horizontal], [5: vertical], [1,2,1,1: horizontal], [1,1,2,1: vertical], and [1,1,1,2: horizontal]. However, if all patterns are considered as arrangement candidates, the amount of calculation for the process of selecting the optimal one from among them may increase, and the process may take time. Therefore, in this embodiment, a plurality of arrangement candidates are determined in advance for each number of partial regions 52. For example, when the number of partial regions 52 is five, only eight patterns, [1,4: horizontal], [1,4: vertical], [2,3: horizontal], [2,3: vertical], [1,2,2: horizontal], [1,2,2: vertical], [1,1,1,1,1: horizontal], and [1,1,1,1,1: vertical], may be determined in advance as arrangement candidates. Patterns in which the partition orientation is different between horizontal and vertical may be determined in advance as arrangement candidates.

[0074] Fig. 21 is a diagram showing a specific example of a pattern of the shape of a provided video. For example, when a provided video is provided by each of five users, the aspect ratio of each provided video may be different. In the example of Fig. 21, provided videos 60_1 to 60_5 provided by the first to fifth users, respectively, all have different aspect ratios. In Fig. 21, the aspect ratio (horizontal / vertical) values ​​are larger from the top.

[0075] The display control unit 181 may determine which arrangement candidate to adopt when determining the size and display position of the partial area 52. In this case, the display control unit 181 first obtains the aspect ratio of each provided moving image 60 to be displayed, and sorts them by the aspect ratio. Next, the display control unit 181 performs the following process for all arrangement candidates. First, the display control unit 181 selects an arrangement candidate to be processed (hereinafter referred to as a "processing target arrangement candidate"), and sorts each partial area 52 in the processing target arrangement candidate by the aspect ratio (W1 / H1 in FIG. 15) of the image display candidate area 55. The display control unit 181 allocates the provided moving images 60 with larger aspect ratios in the order of the partial areas 52 with the largest aspect ratios of the image display candidate area 55.

[0076] At this time, the image display area 56 is determined while maintaining the aspect ratio of the provided video 60 so that the area of ​​the image display area 56 in each image display candidate area 55 is maximized. Information representing the size and display position of the partial areas 52 at this time, the allocation of the provided video to each partial area 52, and the size and display position of the image display area 56 in each partial area 52 is called arrangement candidate information. The display control unit 181 calculates the sum of the areas of the image display areas 56 in each partial area 52 in the processing target arrangement candidate (hereinafter referred to as "total display area"). The display control unit 181 calculates the total display area for all arrangement candidates. The display control unit 181 selects the arrangement candidate with the maximum total display area, and performs display according to the arrangement candidate information of the selected arrangement candidate.

[0077] 22 to 24 are diagrams showing specific examples of the arrangement candidate information. FIG. 22 shows a specific example of the arrangement candidate information in which each provided image shown in FIG. 21 is arranged in the arrangement candidate shown in FIG. 16. The provided moving image 60_1 having the largest aspect ratio is allocated to the partial region 52_a having the image display candidate region 55 having the largest aspect ratio. The image display region 56_1 of the partial region 52_a is determined so as to have the largest area while maintaining the aspect ratio of the provided moving image 60_1. For example, in the example of FIG. 22, the image display region 56_1 may be determined so that the height of the provided moving image 60_1 matches the height of the image display candidate region 55 of the partial region 52_a. In the example of FIG. 22, the aspect ratios of the image display candidate regions 55 of the remaining four partial regions 52 are the same. In this case, the provided moving images 60 may be allocated to each partial region 52 in order from the largest aspect ratio of the provided moving image 60 based on the arrangement position, for example. For example, partial regions 52 may be allocated in order from the top, or from the left if they are in the same row. Therefore, in the example of FIG. 22, partial regions 52_b, 52_c, 52_d, and 52_e are allocated in order from the largest aspect ratio. The image display region 56 of each partial region 52 is determined so as to have the largest area while maintaining the aspect ratio of the allocated provided video 60. For example, in the example of FIG. 22, the image display region 56 may be determined so that the height of each of the remaining four provided video images 60 matches the height of each image display candidate region 55.

[0078] FIG. 23 shows a specific example of arrangement candidate information in which each provided image shown in FIG. 21 is arranged in the arrangement candidate shown in FIG. 17. The provided moving image 60_1 having the largest aspect ratio is assigned to the partial area 52 having the image display candidate area 55 having the largest aspect ratio. However, in the example of FIG. 23, the aspect ratios of the image display candidate areas 55 of the four partial areas 52 (52_b, 52_c, 52_d, 52_e) are the same. In this case, the provided moving images 60 may be assigned to each partial area 52 in descending order of aspect ratio of the provided moving images 60 based on the arrangement position, for example. For example, the partial areas 52 located in the right column may be assigned in descending order, and if they are in the same column, the partial areas 52 may be assigned in descending order of aspect ratio. Therefore, in the example of FIG. 23, the partial areas 52_e, 52_d, 52_c, and 52_b are assigned in descending order of aspect ratio. Then, the image display area 56 of each partial area 52 is determined so as to have the largest area while maintaining the aspect ratio of the assigned provided video 60. For example, in the example of FIG. 23, the image display area 56 may be determined so that the height of each provided video 60 matches the width of each image display candidate area 55.

[0079] FIG. 24 shows a specific example of the arrangement candidate information in which each provided image shown in FIG. 21 is arranged in the arrangement candidate shown in FIG. 18. The provided moving image 60_1 having the largest aspect ratio is assigned to the partial area 52 having the image display candidate area 55 having the largest aspect ratio. However, in the example of FIG. 24, the aspect ratios of the image display candidate areas 55 of the two partial areas 52 (52_a, 52_b) are the same. In this case, the provided moving images 60 may be assigned to each partial area 52 in descending order of aspect ratio of the provided moving images 60 based on the arrangement position, for example. For example, the partial areas 52 located in the upper row may be assigned in descending order, and if they are in the same row, the partial areas 52 located in the leftward order may be assigned in descending order. Therefore, in the example of FIG. 24, the partial areas 52_a, 52_b, 52_c, 52_b, and 52_e are assigned in descending order of aspect ratio. Then, the image display area 56 of each partial area 52 is determined so as to have the largest area while maintaining the aspect ratio of the assigned provided moving image 60. For example, in the example of FIG. 24, the image display area 56 may be determined so that the width of each provided moving image 60 matches the width of each image display candidate area 55.

[0080] The display control unit 181 also acquires arrangement candidate information regarding arrangement candidates other than those shown in Figs. 22 to 24. Then, the display control unit 181 selects the arrangement candidate with the largest total display area from among the arrangement candidates. For example, the arrangement candidate information of the arrangement candidate shown in Fig. 22 may be selected. Then, the display control unit 181 controls the display based on the selected arrangement candidate information.

[0081] The display control unit 181 may select the above-mentioned arrangement candidate information at a predetermined timing and control the display. For example, the above-mentioned display control may be executed at a predetermined cycle (for example, every second, every 10 seconds, or every minute). The display control unit 181 may execute the above-mentioned display control at a timing when a user of the device itself (user terminal 10) performs a predetermined operation. By executing the display control at such a timing, it is possible to prevent the arrangement candidate information on the screen from changing at a timing unintended by the user. In other words, it is possible to prevent the display position of the provided image or the size of the display area from changing at a timing unintended by the user. Specific examples of such a predetermined operation include, for example, an operation to change the number of displayed partial areas 52, an operation to change the display position or size of the partial areas 52, and an operation to change the display area (for example, a window) of an application (for example, a WEB browser) of the conversation control system 100.

[0082] The display control unit 181 may execute the above-mentioned display control at the timing when a user of another user terminal 10 performs a predetermined operation. By executing the display control at such a timing, it is possible to prevent the screen arrangement candidate information from changing unnecessarily frequently. In other words, it is possible to prevent the display position of the provided image and the size of the display area from changing unnecessarily frequently. Specific examples of such a predetermined operation include, for example, an operation to change the size of the provided video, an operation to start providing the provided video, and an operation to stop providing the provided video.

[0083] The display control of the reduced partial area 51 and partial area 52 shown in FIGS. 9 to 24 may be performed by the display control unit 181 of the user terminal 10, or may be performed by the display information generation unit 233 of the conversation control device 20.

[0084] In the conversation control system 100 configured in this manner, it is possible to realize a screen display that is more convenient for the user when conversing with one or more other users. Specifically, this is as follows. In the conversation control system 100, the size and position of the partial area 52 in which the video provided by each user etc. (provided video) is displayed are determined so that the area in which the provided video is displayed is made larger. This makes it possible to realize a screen display that is more convenient for the user.

[0085] Furthermore, in the conversation control system 100, when determining the size and position of the partial area 52 in which the video provided by each user (provided video) is displayed, the layout candidates to be considered are limited in advance. This makes it possible to determine the display more quickly with less calculation.

[0086] In addition, data of the provided video may be transmitted from the user terminal 10 that provides the video (e.g., the sharing source) to the user terminal 10 that receives the video via the conversation control device 20, or it may be transmitted from the user terminal 10 that provides the video to the user terminal 10 that receives the video without passing through the conversation control device 20.

[0087] The conversation control unit 183 or the conversation control unit 234 may perform control so that the voice of the user displayed in the partial area 52 is output louder than the voice of the user displayed in the reduced partial area 51.

[0088] Fig. 25 is a diagram showing a specific example of arrangement candidate information of a modified example. In Fig. 25, the width of the partial area 52 in the first row is shorter than the actual display width of the screen in the first row. In this way, the height and width of the partial area 52 do not necessarily have to be the maximum possible value in the screen or window area of ​​the display unit 13, and may be configured to be values ​​smaller than the maximum value. In addition, the sizes of multiple partial areas 52 arranged in the same row or column may be different from each other.

[0089] FIG. 26 is a diagram showing an outline of a hardware configuration example of an information processing device 900 applied to this embodiment. The information processing device 900 includes a processor 901, a main memory device 902, a communication interface 903, an auxiliary memory device 904, an input / output interface 905, and an internal bus 906. The processor 901, the main memory device 902, the communication interface 903, the auxiliary memory device 904, and the input / output interface 905 are communicably connected to each other via the internal bus 906. The information processing device 900 may be applied to, for example, a user terminal 10 and a conversation control device 20. In this case, for example, the communication unit 11 and the communication unit 21 may be configured using the communication interface 903. For example, the memory unit 17 and the memory unit 22 may be configured using the auxiliary memory device 904. In addition, the control unit 18 and the control unit 23 may be configured using the processor 901 and the main memory device 902.

[0090] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0091] 100...conversation control system, 10...user terminal, 20...conversation control device, 11...communication unit, 12...operation unit, 13...display unit, 14...audio input unit, 15...audio output unit, 16...imaging unit, 17...storage unit, 171...user information storage unit, 18...control unit, 181...display control unit, 182...conference control unit, 183...conversation control unit, 21...communication unit, 22...storage unit, 221...conference room information storage unit, 23...control unit, 231...user control unit, 232...conference room control unit, 233...display information generation unit, 234...conversation control unit, 30...conference room area image, 31...conversation area, 311...conference room name, 312...entry button, 313...detail button, 32...listening area, 41...icon image, 91...outdoor area

Claims

1. A communication unit that receives data of a plurality of moving images provided from another device; a control unit that displays the moving image received by the communication unit by allocating the moving image to a plurality of partial regions obtained by dividing a display screen into a plurality of partial regions; The control unit determines the allocation of the partial regions and the videos for each of a plurality of arrangement candidates indicating possible arrangements of the partial regions on the display screen so that the sum of the areas of the display areas of the multiple videos is largest while maintaining the aspect ratio of the multiple videos, and determines the allocation of the partial regions and the videos according to the arrangement candidate among the plurality of arrangement candidates that maximizes the sum of the areas of the display areas of the videos.

2. The control system according to claim 1 , wherein the control unit determines the allocation of the partial regions to the moving images based on an aspect ratio of each moving image and an aspect ratio of each partial region.

3. A communication unit that receives data of a plurality of moving images provided from another device; a control unit that displays the moving image received by the communication unit by allocating the moving image to a plurality of partial regions obtained by dividing a display screen into a plurality of partial regions; the control unit determines an allocation of the partial regions and the videos such that a sum of areas of display regions of the plurality of videos is larger while maintaining an aspect ratio of the plurality of videos; the control unit determines allocation of the partial regions to the video images based on an aspect ratio of each video image and an aspect ratio of each partial region; The control unit determines the allocation of the partial regions to the videos so that the order of the aspect ratios of the videos matches the order of the aspect ratios of the partial regions.

4. The computer receives data of a plurality of moving images provided from another device, The computer displays the received video by allocating the video to a plurality of partial regions obtained by dividing the display screen into a plurality of partial regions; A control method in which a computer determines an allocation of the partial regions and the videos for each of a plurality of arrangement candidates indicating possible arrangements of the partial regions on a display screen so that the sum of the areas of the display areas of the multiple videos is maximized while maintaining the aspect ratio of the multiple videos, and determines the allocation of the partial regions and the videos according to the arrangement candidate among the plurality of arrangement candidates that maximizes the sum of the areas of the display areas of the videos.

5. A communication unit that receives data of a plurality of moving images provided from another device; a control unit that displays the moving image received by the communication unit by allocating the moving image to a plurality of partial regions obtained by dividing a display screen into a plurality of partial regions; The control unit determines the allocation of the partial areas to the videos for each of a plurality of arrangement candidates indicating possible arrangements of the partial areas on a display screen so that the sum of the areas of the display areas of the videos is largest while maintaining the aspect ratio of the plurality of videos, and determines the allocation of the partial areas to the videos in accordance with the arrangement candidate among the plurality of arrangement candidates that maximizes the sum of the areas of the display areas of the videos.

6. A communication unit that receives data of a plurality of moving images provided from another device; a control unit that displays the moving image received by the communication unit by allocating the moving image to a plurality of partial regions obtained by dividing a display screen into a plurality of partial regions; the control unit determines an allocation of the partial regions and the videos such that a sum of areas of display regions of the plurality of videos is larger while maintaining an aspect ratio of the plurality of videos; the control unit determines allocation of the partial regions to the video images based on an aspect ratio of each video image and an aspect ratio of each partial region; A computer program for operating a computer as a control system, in which the control unit determines the allocation of the partial regions to the videos so that the order of the aspect ratios of each video matches the order of the aspect ratios of each partial region.

Images