Data processing device, data processing method and data processing system

Abstract

To enhance presence of a remote conference.SOLUTION: A data processing apparatus 1 in a system that holds a remote conference between a plurality of locations acquires a plurality of pieces of location image data corresponding to the plurality of locations showing a plurality of participants participating in the remote conference, for each of the plurality of locations, deforms the location image data corresponding to a plurality of other locations into shapes corresponding to the number of the plurality of locations, and thereby generates a plurality of pieces of deformed image data. The data processing apparatus 1 generates composite image data in which the plurality of pieces of deformed image data is arranged at positions corresponding to the number of the plurality of locations, and transmits the pieces of composite image data corresponding to the plurality of locations to one or more electronic apparatuses corresponding to the plurality of locations.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a data processing device, a data processing method, and a data processing system. [Background technology]

[0002] Conventionally, remote conference systems for holding a conference among multiple people in remote locations have been known (see, for example, Patent Document 1). In conventional remote conference systems, the sense of realism is enhanced by creating an image in which captured images of each of the multiple conference participants are arranged on a virtual conference table. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-54192 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional remote conference systems, individual captured images of multiple conference participants are arranged on a virtual conference table, but there are cases where multiple participants gather in conference rooms at multiple locations. In such cases, if a composite image in which images captured at each location are simply arranged around the virtual conference table is displayed in each conference room, the sense of realism is reduced.

[0005] One of the objects of the present invention is to improve the sense of realism of a remote conference. [Means for solving the problem]

[0006] A first aspect of the data processing device of the present invention is a data processing device in a system for conducting remote conferences between multiple locations, and includes an image data acquisition unit that acquires multiple location image data corresponding to the multiple locations, each indicating multiple participants participating in the remote conference; an image data generation unit that generates multiple transformed image data for each of the multiple locations by transforming the location image data corresponding to multiple other locations into shapes corresponding to the number of the multiple locations, and generates composite image data in which the multiple transformed image data are arranged at positions corresponding to the number of the multiple locations; and a data transmission unit that transmits the composite image data corresponding to each of the multiple locations to one or more electronic devices corresponding to the multiple locations.

[0007] The image data generation unit may generate the composite image data in which the plurality of transformed image data is arranged in a number obtained by subtracting one from the number of the plurality of bases.

[0008] The image data generation unit may generate the composite image data in which the plurality of transformed image data, the number of which is the number of the plurality of bases minus one, are arranged symmetrically.

[0009] The image data generation unit may generate the composite image data in which the multiple transformed image data are arranged based on the multiple base image data corresponding to multiple bases other than the base to which the composite image data is to be sent.

[0010] The image data generating unit may generate the plurality of transformed image data by performing different image processing on face image data and body image data included in the base image data.

[0011] The image data generation unit may generate the plurality of transformed image data by transforming the body image data based on an arrangement direction of desk image data corresponding to each of the plurality of locations.

[0012] The data processing device may further have a setting reception unit that receives settings of a base selected from the multiple bases from participants in the remote conference, and the image data generation unit may generate the composite image data by combining participant image data corresponding to the participant with the base image data corresponding to the base selected by the participant.

[0013] The data processing device may further have a setting reception unit that receives a setting for the number of the multiple locations, and the image data generation unit may generate the multiple transformed image data by transforming the location image data into a shape corresponding to the number of the multiple locations received by the setting reception unit.

[0014] The data processing device may further include an audio data acquisition unit that acquires multiple location audio data based on audio collected at the multiple locations, and an audio data generation unit that generates processed audio data by processing the multiple location audio data based on the positions of each of the multiple transformed image data in the synthetic image data, and the data transmission unit may transmit the multiple processed audio data corresponding to audio collected at multiple locations different from each of the multiple locations to one or more electronic devices corresponding to the multiple locations.

[0015] The audio data generation unit may generate the processed audio data including first position audio data to be output to a first position speaker installed at a first position of at least one of the plurality of bases, and second position audio data to be output to a second position speaker installed at a second position of at least one of the plurality of bases.

[0016] A data processing method of a second aspect of the present invention is executed by a computer and includes the steps of acquiring multiple base station image data corresponding to multiple base stations, the multiple base station image data indicating multiple participants participating in a remote conference between multiple base stations; generating multiple transformed image data for each of the multiple base stations by transforming the base station image data corresponding to multiple other base stations into shapes corresponding to the number of the multiple base stations; generating composite image data in which the multiple transformed image data are arranged at positions corresponding to the number of the multiple base stations; and transmitting the composite image data corresponding to each of the multiple base stations to one or more electronic devices corresponding to the multiple base stations.

[0017] A third aspect of the data processing system of the present invention is a data processing system for conducting remote conferences between multiple locations, and includes one or more electronic devices installed at each of the multiple locations; an image data acquisition unit that acquires multiple location image data corresponding to the multiple locations, indicating multiple participants participating in the remote conference; an image data generation unit that generates multiple transformed image data for each of the multiple locations by transforming the location image data corresponding to multiple other locations into shapes corresponding to the number of the multiple locations, and generates composite image data in which the multiple transformed image data are arranged at positions corresponding to the number of the multiple locations; and a data transmission unit that transmits the composite image data corresponding to each of the multiple locations to the one or more electronic devices corresponding to the multiple locations. [Effects of the Invention]

[0018] The present invention has the effect of improving the sense of realism of a remote conference. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a diagram illustrating an overview of a data processing system. [Figure 2] FIG. 10 is a schematic plan view showing participants being photographed by cameras at individual locations as viewed from above. [Figure 3]10 is a schematic plan view of a virtual layout of tables in individual base image data in accordance with the number of bases, as viewed from above. FIG. [Figure 4] FIG. 4 is a diagram schematically illustrating an example of a screen displayed at a certain location in the table layout illustrated in FIG. 3. [Figure 5] FIG. 1 is a diagram showing an outline of the flow of image and audio data between a plurality of locations. [Figure 6] FIG. 1 is a diagram illustrating a configuration of a data processing device. [Figure 7] FIG. 10 is a diagram for explaining a method by which a remote participant specifies a position at which image data is to be combined. [Figure 8] FIG. 10 is a schematic diagram for explaining candidates for positions where image data of remote participants are to be combined. [Figure 9] FIG. 10 is a diagram illustrating an example in which an empty seat area is created for a remote participant. [Figure 10] 1 is a flowchart showing the flow of processing in the data processing device 1. DETAILED DESCRIPTION OF THE INVENTION

[0020] [Data Processing System S Overview] 1 is a diagram illustrating an overview of a data processing system S. The data processing system S is a system for multiple people at multiple locations to hold a remote conference. In the following description, people who take part in the remote conference are called participants.

[0021] FIG. 1 shows multiple locations, namely, a first location, a second location, ..., an nth location (n is an integer equal to or greater than 3), which are connected by a network N. The network N includes, for example, the Internet or an intranet. The data processing device 1 and electronic devices at each location are connected to the network N. The network N may include at least one of wired and wireless communication. For example, the network N may be configured by wired or wireless communication, or may include both wired and wireless sections. The wireless communication may include, for example, at least one of cellular communication, wireless LAN (local area network) communication, and satellite communication.

[0022] In the following, the location numbers of the first to nth locations will be represented by "i" (where i is any one of 1 to n), and the number of participants located at the i-th location with location number i will be represented by "j", an integer greater than or equal to 1, and participants at each location (i-th location) will be referred to as "Ai-j". Furthermore, when individual participants (j) at a certain location (i) are not distinguished, they may be abbreviated as "participant Ai", and when both individual locations (i) and individual participants (j) are not distinguished, they may be abbreviated as "participant A".

[0023] At the i-th location, a router 21-i, a computer 22-i, a camera 23-i, a display 24-i, and a speaker 25-i (speakers 25L-i and 25R-i) are installed. At the i-th location, for example, one or more participants Ai-1, Ai-2, Ai-3, and Ai-4 are seated around a table. The router 21-i, the computer 22-i, the camera 23-i, the display 24-i, and the speaker 25-i are electronic devices (corresponding to the i-th electronic device) used by the participants Ai-1, Ai-2, Ai-3, and Ai-4 at the i-th location to watch the remote conference.

[0024] In the following, when there is no need to distinguish between electronic devices such as computer 22-i, camera 23-i, display 24-i, and speaker 25-i at individual locations, the location number i may be omitted, such as computer 22, camera 23, display 24, and speaker 25.

[0025] In this embodiment, an example is shown in which camera 23-i has a microphone and generates ith image data based on ith image captured within ith location and ith audio data based on ith audio within ith location, but these electronic devices may be integrated, or equivalent functions may be provided by even more electronic devices.

[0026] For example, at an i-th location, one or more participants Ai may be seated in front of a computer 22 equipped with a camera, microphone, and speaker, and the computer 22 may generate i-th image data based on an i-th image taken of the participant Ai and i-th audio data based on the i-th audio of the participant A.

[0027] In FIG. 1, the first to nth locations are depicted as having the same configuration or layout, but some or all of the first to nth locations may have different configurations or layouts. Furthermore, the number of participants A participating in the remote conference may be different at some or all of the first to nth locations. In other words, the number (j) of participants A at each location (i) may be the same or different. Furthermore, image data captured by camera 23-i at each i-th location may be referred to below as "i-th location image data" or simply as "location image data." Furthermore, "image data" may be abbreviated to "image" as appropriate.

[0028] At least one of the first to n-th locations is provided with a plurality of cameras 23 to capture images of participant A from a plurality of directions. Also, at least one of the first to n-th locations may have a microphone that is a pin microphone used by each participant A.

[0029] The data processing device 1 is a computer, such as a cloud server, that executes various processes for remote conferences between multiple locations. As will be described in detail later, the data processing device 1 transmits images and audio sent from one location to another location. This allows participants at each location to view images of participants at other locations while listening to the audio of participants at other locations.

[0030] For example, at the i-th location, the display 24-i displays an image of a participant Ak from multiple k-th locations (k is one or more of 1 to n satisfying k ≠ i) excluding the i-th location, and the voice of the participant Ak is output from the speaker 25-i.

[0031] 2A and 2B are schematic plan views of participants at individual locations as viewed from above, captured by a camera 23. FIG. 2A shows three participants Ai-1, Ai-2, and Ai-3 seated facing a table Ti (where i represents the location number described above) at a certain location, and the space including the table Ti and each participant Ai is captured by a camera 23 installed with the imaging direction facing each participant Ai. FIG. 2B shows an example of a location image captured by the camera 23 of the table Ti and each participant Ai, as shown in FIG. 2A.

[0032] Here, for example, if the site image data of each k-th site is displayed separately (for example, in different areas of a display area divided into multiple (n-1) parts) on the display 24-i of the i-th site as is, the sense of unity or realism of the remote conference will be reduced. Therefore, the data processing device 1 transforms and combines the multiple (n-1) site image data sent from each of the multiple k-th sites excluding the i-th site, for example.

[0033] As a non-limiting example, the data processing device 1 transforms and combines the image data of each location so that it appears to a participant Ai located at the ith location that a participant Ak located at another kth location is participating with each table (or desk) at the kth location arranged in a specific layout in a single conference room space.

[0034] Figure 3 is a schematic plan view, viewed from above, of the tables in the individual site image data virtually arranged in a layout corresponding to the number of sites n. Figure 3(a) shows an example in which tables T1 to T3 of three sites are arranged in a triangular layout when the number of sites n is 3. Figure 3(b) shows an example in which tables T1 to T4 of four sites are arranged in a rectangular layout when n is 4.

[0035] Figure 4 is a diagram schematically illustrating an example of a screen displayed on the display 24-1 at the first location in the table layout illustrated in Figure 3. Figure 4(a) illustrates the triangular table layout illustrated in Figure 3(a) in which participants A2-1, A2-2, and A2-3 at the second location and participants A3-1 and A3-2 at the third location are seated at table T2 at the second location and table T3 at the third location, respectively, and participating in a remote conference.

[0036] 4(b) shows a situation in which, in the rectangular table layout illustrated in FIG. 3(b), participants A2-1, A2-2, and A2-3 from the second location are seated at table T2 at the second location, participants A3-1 and A3-2 from the third location are seated at table T3 at the third location, and participants A4-1, A4-2, and A4-3 from the fourth location are seated at table T4 at the fourth location, and are participating in a remote conference. At other k-th locations excluding the first location, the appearance of participants A at each location excluding the k-th location is displayed on the display 24 in a similar display mode.

[0037] This display mode allows a participant Ai at the i-th location to feel as if they are meeting with participants Ak at multiple other locations k at a table in a single conference room, arranged in a layout according to the number n of locations. This improves the sense of unity and realism of each participant taking part in the remote conference.

[0038] Therefore, the data processing device 1 generates composite image data including a table layout such as that illustrated in Figure 4, for each of n locations, by transforming and synthesizing location image data corresponding to multiple other (n-1) locations into a shape corresponding to the number n of locations, and transmits the generated composite image data to the computer 22 at the i-th location to be transmitted.

[0039] The data processing device 1 may composite at least a portion of image data of a participant Ai at the i-th location to which the composite image data is to be transmitted with the location image data of another location k. For example, as shown by the dotted line in Fig. 4, on the display 24-i at the i-th location, at least a portion of image data of an image of a participant Ai participating in a remote conference at the i-th location (for example, a view of the back of the participant Ai seated around a table Ti) may be composited with transformed image data obtained by transforming the image data of the other location k.

[0040] At this time, at least a portion of the captured image data of the table Ti may be combined with the transformed image data corresponding to another location k together with the back view of the participant Ai. In order to capture an image of the back view of the participant Ai, a camera that captures an image of the participant Ai from behind may be installed at the i-th location separately from the camera 23 that captures an image of the participant Ai from the front.

[0041] As described above, the data processing device 1 generates transformed image data by transforming a plurality of base image data of a plurality of bases in accordance with the number n of bases, and generates composite image data by combining the transformed image data.

[0042] Note that the target of transforming or combining the base image data may be at least a portion of each base image data. For example, a partial area of ​​each base image data including participant A and table Ti may be the target of transforming or combining, or the entire base image data may be the target of transforming or combining. Furthermore, the display mode shown in FIG. 4 is merely an example, and the display mode may be determined based on various settings or conditions, as described below.

[0043] [Data flow between multiple locations] 5 is a diagram showing an outline of the flow of image and audio data between multiple locations. The router 21-i at the i-th location transmits the i-th location image data and i-th location audio data generated by the camera 23-i to the data processing device 1.

[0044] The data processing device 1 generates n-1 (or n) transformed image data by transforming at least a portion of each of the n received base station image data corresponding to the number n of base stations, for example, as illustrated in Figure 4, into a shape corresponding to the number n of base stations, and generates composite image data in which each transformed image data is placed at a position corresponding to the number n of base stations (for example, a position specified by the table layout illustrated in Figure 4).

[0045] The data processing device 1 also processes the i-th location voice data received from the i-th location to create k-th processed voice data to be transmitted to another k-th location. The k-th processed voice data is, for example, stereo voice data or three-dimensional voice data in which the voice of the participant Ai at the i-th location is processed so that the participant Ak at another k-th location can hear the voice of the participant Ai at the i-th location from the position where the transformed image data corresponding to the i-th location is synthesized (or placed) in the k-th composite image data to be transmitted to the k-th location. The data processing device 1 transmits the composite image data and the processed voice to each corresponding location.

[0046] [Configuration and Operation of Data Processing Device 1] 6 is a diagram showing the configuration of the data processing device 1. The data processing device 1 has a communication unit 11, a storage unit 12, and a control unit 13. The control unit 13 has an acquisition unit 131, an acquisition unit 132, a setting acceptance unit 134, an image data generation unit 135, an audio data generation unit 136, and a data transmission unit 137. The acquisition units 131 and 132 may be integrated into a single acquisition unit. Furthermore, n acquisition units 131 and 132 may be provided corresponding to the number n of base stations, or some of the n acquisition units 131 or some of the n acquisition units 132 may be integrated into a single acquisition unit.

[0047] The communication unit 11 has a communication interface for transmitting data between the router 21, the computer 22, or the router 21 via the network N. The communication unit 11 inputs the base station image data received from the router 21 to the acquisition unit 131, and inputs the base station audio data received from the router 21 to the acquisition unit 132.

[0048] The communication unit 11 inputs the setting data transmitted by the router 21, the computer 22, or another information terminal to the setting reception unit 134. The setting data is, for example, data indicating the number of locations n, the attributes of the remote conference, the attributes of the participants, or the format of the image to be displayed on the screen, and is data input by the organizer of the remote conference or any of the participants. The organizer of the remote conference may or may not be included in the multiple participants. Data determining the placement (or composition) position of the transformed image data corresponding to the number of locations n as exemplified in FIG. 4 is included, for example, in "data indicating the format of the image to be displayed on the screen."

[0049] Furthermore, the communication unit 11 transmits to each location various types of data input from the data transmission unit 137. Specifically, as shown in Fig. 5, the communication unit 11 transmits the composite image data, processed audio data, and the like to one or more electronic devices used by the participants at each location to watch the remote conference.

[0050] The storage unit 12 has storage media such as a ROM (Read Only Memory), a RAM (Random Access Memory), and an SSD (Solid State Drive). The storage unit 12 stores programs executed by the control unit 13. The storage unit 12 also stores various setting data received by the communication unit 11. Furthermore, the storage unit 12 temporarily stores base image data and base audio data transmitted from each base. The setting data described above may be stored in the storage unit 12.

[0051] The control unit 13 has, for example, a CPU (Central Processing Unit). The control unit 13 executes the programs stored in the storage unit 12, thereby functioning as an acquisition unit 131, an acquisition unit 132, a setting acceptance unit 134, an image data generation unit 135, an audio data generation unit 136, and a data transmission unit 137.

[0052] The acquisition unit 131 acquires, via the communication unit 11, a plurality of base station image data corresponding to a plurality of base stations, which indicates a plurality of participants participating in a remote conference from each of the plurality of base stations. The acquisition unit 132 acquires, via the communication unit 11, a plurality of base station audio data based on audio collected by a microphone at each of the plurality of base stations. The acquisition unit 131 inputs the base station image data to the image data generation unit 135, and the acquisition unit 132 inputs the base station audio data to the audio data generation unit 136.

[0053] The base station image data acquired by the acquisition unit 131 is not limited to captured image data generated by capturing an image of the participant, and may include, for example, avatar image data of the participant created in advance. That is, the acquisition unit 131 may acquire at least one of captured image data or avatar image data of the participant. The acquisition unit 131 acquires, for example, either the captured image data or the avatar image data requested by the image data generation unit 135.

[0054] The setting reception unit 134 receives various settings by acquiring setting data input by the organizer of the remote conference or any of the participants via the communication unit 11. The setting reception unit 134 may receive settings via a user interface such as a keyboard and mouse (not shown). The setting reception unit 134 receives settings such as the number of bases n. The setting reception unit 134 may also receive settings of a base selected from multiple bases from a participant of the remote conference. The setting reception unit 134 notifies the image data generation unit 135 of the received setting data.

[0055] The image data generation unit 135 generates image data to be displayed on the display 24-i installed at each location participating in the remote conference. The image data generation unit 135 generates multiple (e.g., n-1) transformed image data by transforming at least a portion of each of the multiple location image data sent from the other multiple locations (the kth location) into a shape corresponding to the number n of locations as image data to be displayed on the display 24-i at a certain location, and generates composite image data in which the individual transformed image data are arranged at positions corresponding to the number n of locations (positions defined by the table layout illustrated in FIG. 4).

[0056] In other words, the image data generation unit 135 generates composite image data in which multiple transformed image data based on multiple base image data corresponding to the number of bases (n) minus 1 (for example, multiple bases other than the bases to which the composite image data is to be sent) is arranged.

[0057] As an example, the image data generation unit 135 generates composite image data including a table layout as illustrated in Fig. 4 by modifying and combining the site image data of n-1 other sites k excluding the site i to which the composite image data is to be transmitted. The image data generation unit 135 inputs the composite image data generated as described above to the data transmission unit 137.

[0058] 4, the image data generation unit 135 generates composite image data in which multiple pieces of transformed image data, the number of which is the number of multiple locations (n) minus 1, are arranged symmetrically. This is expected to reduce the processing load on the data processing device 1 compared to when the transformed image data are arranged asymmetrically, and is also expected to improve the visibility of other participants displayed on the display 24 for participants viewing the composite image data. However, the arrangement of the multiple pieces of transformed image data in the composite image data is not limited to symmetry, and may be determined to be any other display mode, including an asymmetric arrangement, that can improve the sense of unity or realism of the remote conference.

[0059] The audio data generation unit 136 generates audio data to be output from speakers installed at each location participating in the remote conference. The audio data generation unit 136 generates processed audio data by, for example, processing the audio data of multiple locations based on the positions of each of the multiple transformed image data in the composite image data. The audio data generation unit 136 inputs the generated processed audio data to the data transmission unit 137.

[0060] The audio data generation unit 136 generates multi-channel processed audio data including multiple audio signals transmitted from multiple other locations (k-th location) as audio data to be output from a speaker 25-i at a certain location (i-th location). The multi-channel processed audio data includes first-position audio data output from a speaker 25R-i at a first location (e.g., the right side) and second-position audio data output from a speaker 25L-i at a second location (e.g., the left side). The multi-channel processed audio data may include three or more pieces of audio data corresponding to three or more locations.

[0061] The speakers 25R-i and 25L-i used for multi-channel audio output may be installed at all of the multiple locations, or may be installed at only some of the multiple locations. In the latter case, the audio data generation unit 136 selectively transmits non-multi-channel audio data to, for example, a location among the multiple locations that does not support multi-channel audio output. Data identifying which of the multiple locations does not support multi-channel audio output is set, for example, as the setting data described above.

[0062] As an example of multi-channel audio output, the audio data generation unit 136 generates processed audio data including first position audio data to be output to a first position speaker installed at a first position in the i-th location and second position audio data to be output to a second position speaker installed at a second position in the i-th location, based on the positions of each of the multiple transformed image data to be combined with the combined image data. Specifically, when the position of the transformed image data including one or more participants is closer to the first position than the second position, the audio data generation unit 136 generates processed audio data that has been processed so that the proportion of the participants' voices included in the first position audio data is greater than the proportion of the participants' voices included in the second position audio data.

[0063] Conversely, when the position of the transformed image data including one or more participants is closer to the second position than the first position, the audio data generation unit 136 generates processed audio data processed so that the proportion of the voice of the second participant included in the second position audio data is greater than the proportion of the voice of the second participant included in the first position audio data. Note that when the position of the transformed image data including one or more participants is at an equal distance from the first position and the second position, the audio data generation unit 136 may generate processed audio data processed so that the proportions of the voice of the participants included in the first position audio data and the second position audio data are equal.

[0064] The audio data generator 136 may generate processed audio data further based on the result of identifying the depth direction position at the location corresponding to each position of the transformed image data in the composite image data. As an example, in the example shown in Fig. 4(a), the position of the transformed image data including the participants A2-1 to A2-3 at the second location is further back than the first participants A2-1 to A2-3, so the audio data generator 136 may generate processed audio data that has been processed so that the audio of the participants A2-1 to A2-3 is quieter than the audio of the first participants A1 to A3.

[0065] Similarly, since the position of the transformed image data including participants A3-1 and A3-2 at the third location is further back than the first participants A2-1 to A2-3, the audio data generation unit 136 may generate processed audio data that has been processed so that the audio of participants A3-1 and A3-2 is quieter than the audio of the first participants A1 to A3.

[0066] Furthermore, the audio data generation unit 136 may generate processed audio data based on the results of identifying the depth direction positions at the locations corresponding to the positions of each participant included in one transformed image data. For example, in the example shown in FIG. 4(a), in the transformed image data including participants A3-1 and A3-3 at the third location, participant A3-2 is located further back than participant A3-1, so the audio data generation unit 136 may generate processed audio data processed so that the audio of participant A3-2 is quieter than the audio of participant A3-1. The audio processing described above may also be applied to participants included in transformed image data corresponding to other locations based on their positional relationships in the transformed image data.

[0067] By the audio data generating unit 136 generating the processed audio data as described above, the sense of realism felt by participants listening to the audio based on this data is enhanced.

[0068] Regardless of whether the position of the deformed image data or the position of the participant in the deformed image data is closer to the first position than the second position or closer to the second position than the first position, the audio data generation unit 136 may generate processed audio data that has been processed to increase the proportion of the participant's voice included in the first position audio data according to the attributes of each participant. The audio data generation unit 136 may control the proportion of audio data as described above based on attributes such as the volume, voice quality, and clarity of the participant's voice. Specifically, in the example shown in FIG. 4(a), even if participant A3-1 is located closer to participant A3-2, if the volume of participant A3-1's voice is below a threshold or if the condition that the voice quality is difficult to hear is met, the audio data generation unit 136 may increase the proportion of participant A3-1's voice.

[0069] The data transmission unit 137 transmits the image data input from the image data generation unit 135 and the audio data input from the audio data generation unit 136 to the locations to which the respective data should be transmitted, via the communication unit 11. For example, the data transmission unit 137 transmits composite image data corresponding to each of the multiple locations to one or more electronic devices corresponding to the multiple locations. Furthermore, the data transmission unit 137 transmits a plurality of pieces of processed audio data corresponding to audio collected at multiple locations different from each of the multiple locations to one or more electronic devices corresponding to the multiple locations.

[0070] The functions of the data processing device 1 described above (for example, various functions of the control unit 13) may be realized by a plurality of data processing devices 1. For example, various processes such as generation of image data and generation of audio data may be distributed and performed by a plurality of data processing devices 1 (for example, processors mounted on each of a plurality of servers).

[0071] [Method for generating synthetic image data] The process by which the image data generating unit 135 generates the composite image data will be described in detail below. The image data generation unit 135 generates composite image data to be displayed on the display 24-i of the location i, for example, by transforming and combining the location image data of n-1 other locations k excluding the location i to which the composite image data is to be sent. For example, as shown in FIG. 4, the image data generation unit 135 generates composite image data by combining at least a portion of each of the transformed image data at a position in each transformed image where one or more participants are not included.

[0072] Here, the image data generation unit 135 generates multiple transformed images by, for example, applying different image processing to the face image data and the body image data included in each piece of base image data. For example, the image data generation unit 135 identifies, as face image data corresponding to a face region, an area in the base image data whose similarity to standard face image data stored in advance in the storage unit 12 is equal to or greater than a threshold. Also, the image data generation unit 135 identifies, as body image data corresponding to a body region, an area in the base image data whose similarity to standard body image data stored in advance in the storage unit 12 is equal to or greater than a threshold.

[0073] Then, the image data generating unit 135 generates multiple transformed image data by transforming one or both of the face image data and the body image data based on, for example, the arrangement direction of the image data of the table corresponding to each of the multiple locations. For example, the image data generating unit 135 transforms the face image data of the participant in the location image data so that the line of sight is directed toward the display 24.

[0074] 4(a), the image data generation unit 135 transforms the facial image data so that the gaze directions of the participant A2 at the second location and the participant A3 at the third location are directed toward the front. Also, the image data generation unit 135 transforms the body image data of the participant A2 at the second location and the participant A3 at the third location so that the gaze directions are directed toward the tables T2 and T3, respectively, based on the orientation of the table T2 at the second location and the orientation of the table T3 at the third location.

[0075] As described above, the image data generating unit 135 generates transformed image data by separately (e.g., different) processing the face image and the body image included in the base image data, thereby reducing the unnatural gaze direction and orientation of participants when multiple transformed image data are combined without processing. Therefore, it is possible to prevent a decrease in the sense of unity or realism of the remote conference.

[0076] If there is another remote participant participating in the remote conference from a location other than the first to nth locations, the image data generation unit 135 may generate composite image data at a location specified by the host or remote participant of the remote conference by combining participant image data corresponding to the remote participant with location image data corresponding to the location selected by the remote participant. As an example, the data transmission unit 137 transmits the location image data corresponding to the location selected by the remote participant to one or more electronic devices at the location selected by the remote participant from among the multiple locations via the setting reception unit 134. The image data generation unit 135 generates composite image data by combining at least a portion of the participant image data at a location specified by the remote participant in the location image data displayed on the electronic device.

[0077] By operating the image data generating unit 135 in this manner, the remote participant can have his or her image displayed at a location at the location that the remote participant desires. Note that the location image data transmitted to the computer 22 by the data transmitting unit 137 may be image data in which portions other than the image of one or more remote participants are modified from the location image data transmitted from the corresponding location.

[0078] 7A and 7B are diagrams illustrating a method by which another remote participant participating in a remote conference from a location other than the first to n-th locations can specify a position at which to superimpose their own participant image data. Fig. 7A shows a screen for specifying a superimposition position that is displayed on computer 22 at the first location when the remote participant selects the second location from among the first to third locations. In the screen shown in Fig. 7A, the position of a star mark X is indicated as a position at which participant image data can be superimposed.

[0079] 7(b) shows a screen for specifying a composite position that is displayed on computer 22 at the first location specified by the remote participant when the remote participant selects the third location from among the first to third locations. In the screen shown in FIG. 7(b), the position of a star mark Y is indicated as a position where participant image data can be composited.

[0080] When the image data shown in Figures 7(a) and 7(b) are transformed and combined as shown in Figure 4(a), the positions at which the participant image data can be combined in the combined image are, for example, the positions indicated by star marks X and Y in Figure 7(c), and the participant image data of the remote participant will be combined at one of these positions. Note that in Figures 7(a) and 7(b), each piece of base image data shows one candidate position at which the participant image data can be combined, but two or more candidates may be shown for one piece of base image data, depending on, for example, the availability of seats around the table.

[0081] The image data generation unit 135, for example, searches for an area in the base image data where the similarity to reference image data representing a person is equal to or greater than a threshold, thereby identifying an area of ​​the participant's image in the base image data, and causes the computer 22 to display an image in an area other than the identified area, with a mark indicating a position where the participant image data can be superimposed. The image data generation unit 135 may also cause the computer 22 to display an image in an area other than the identified area, with a mark indicating a position where the participant image data can be superimposed, at the center of an area of ​​approximately the same size as the area of ​​the participant's image. In this way, the image data generation unit 135 presents the participant with a position where the participant image data can be superimposed, allowing the participant to appropriately specify the position.

[0082] The data transmission unit 137 may transmit one or more pieces of site image data to an information terminal used by a participant registered as the organizer of the remote conference, and the image data generation unit 135 may generate composite image data by synthesizing at least a portion of the image data at a specified position in an image based on the site image data displayed on the organizer's information terminal. By operating the image data generation unit 135 in this manner, the organizer of the remote conference can arrange the images of the participants at positions appropriate for the relationship between multiple sites, the relationship between multiple participants, the purpose of the remote conference, etc.

[0083] The data transmission unit 137 may transmit the i-th location image data to one or more electronic devices used by one or more participants at the i-th location to watch the remote conference, and the image data generation unit 135 may generate composite image data by combining at least a portion of the image data of the participant at the i-th location at a specified position in the i-th location image displayed on a display 24-i, which is a display device installed at the i-th location, based on the i-th location image data acquired via the one or more electronic devices (e.g., router 21-i or computer 22-i). By operating the image data generation unit 135 in this manner, the participant at the i-th location can move their seat while viewing the image based on the i-th location image data and place the image data of the participant at the k-th location in an appropriate position.

[0084] The data transmission unit 137 may transmit the i-th location image data to one or more electronic devices used by one or more participants at the k-th location to view the remote conference, and the image data generation unit 135 may generate composite image data by synthesizing at least a portion of the image data of the participant at the k-th location at a specified position in an image based on the i-th location image data displayed on a display 24-k, which is a display device installed at the k-th location, based on the i-th location image data acquired via the one or more electronic devices (e.g., router 21-k or computer 22-k). By operating the image data generation unit 135 in this manner, it is possible to arrange the image data of the participant at the k-th location at a position that is easy to view for the participant at the k-th location viewing the image based on the composite image data.

[0085] The image data generating unit 135 may place images of participants at a certain location in an area different from the area in which the multiple participants at another location are captured, based on the settings received by the setting receiving unit 134. The area in which the multiple participants are captured is, for example, the smallest oval or elliptical area that includes the images of the multiple participants.

[0086] As an example, in the composite image data displayed at the second location as shown in Figures 8(a) and 8(b), the image data generation unit 135 places the image data of participant B in an area far away from the area where participant A at the second or third location is displayed. Here, participant B may be a participant at any of multiple locations excluding the location where the composite image data is displayed, or may be a participant at yet another location. For convenience, "participant B" may be referred to as "remote participant B."

[0087] When the system is set to display participants at different locations in the same area, such as when participant B and participant A at the second or third location belong to the same department or company, the image data generation unit 135 places an image of participant B within an area that includes one or more participants A at the location selected by participant B, as illustrated in FIG. 7.

[0088] For example, if participant B belongs to the same department or company as participant A3-i at the third location, and participant B selects and specifies the third location (for example, the position of star mark Y in Figure 7(c)) as the placement location for his or her image data, participant B will be displayed as if seated around table T3 where participant A3-j at the third location is seated.

[0089] By displaying such an image at the first location, participant A1-j can get a sense of realism as if participant B were participating in the remote conference together with participant A3-j at the third location. In addition to the display of participant B, information indicating that participant B is not actually at the third location (for example, that participant B is remotely participating at the third location) may be displayed on the display 24 at the first location.

[0090] On the other hand, if the setting is such that participants at different locations are not displayed in the same area, such as when participant B and participant A at the second or third location belong to different departments or companies, the image data generation unit 135 places the image of participant B in an area different from the area in which one or more participants A at the second or third location are displayed (for example, at the top of the composite image data), as shown in Figure 8.

[0091] In Figure 8, the area indicated by the dotted frame corresponds to the candidate area where participant B will be placed, and the candidate area may be set according to the number of transformed image data to be synthesized (in other words, the number of locations minus 1).

[0092] For example, Figure 8(a) shows an example in which two areas, calculated by subtracting one from the number of locations (3), are set as candidate areas (or locations) for placing participant B, and one of these areas (see the solid-line frame on the right) is designated as the area for placing participant B's image data.

[0093] Figure 8(b) shows an example in which three areas, calculated by subtracting one from the number of locations (four), are set as candidate areas (or locations) for placing participant B, and one of these areas (see the solid-line frame on the far right) is designated as the area for placing participant B's image data.

[0094] 8, by displaying participant B at a position away from the table where participant A is seated, participants A1 to A3 at the first location can intuitively understand that participant B does not belong to the same organization as participant A at the other location. In addition to displaying participant B, information indicating that participant B belongs to an organization different from the organization to which participant A at the other location belongs may be displayed on the display 24 at the first location.

[0095] 8, the area in which the image data of participant B is placed may be determined based on the placement position of the transformed image data in the composite image data and the relationship between participants A and B included in the transformed image data. For example, in FIG. 8(a), participant B belongs to a different department or company from participant A3-i at the third location, but may belong to a department or company related to the department or company to which participant A3-i at the third location belongs.

[0096] In such a case, the image data generation unit 135 may determine to place the image data of participant B above the position of the deformed image data corresponding to the third location in the composite image data, as shown by the solid-line frame in the upper right of Fig. 8(a). If participant B belongs to a department or company related to the department or company to which participant A2-i at the second location belongs, the image data generation unit 135 may determine to place the image data of participant B above the position of the deformed image data corresponding to the second location in the composite image data, as shown by the dotted-line frame in the upper left of Fig. 8(a).

[0097] Similarly, in the example of Figure 8(b), the image data generation unit 135 may determine that the candidate area located above the placement position of the transformed image data corresponding to the location associated with participant B is the area in which to place participant B's image data, out of the three candidate areas corresponding to the three locations at the top of the composite image data.

[0098] By operating the image data generating unit 135 as described above, the composite image data is displayed in a manner appropriate to the affiliation of the participants at each location, further enhancing the sense of realism. Also, the visibility of the relationships between the participants at each location is improved, which is expected to contribute to the smooth progress of the conference, for example.

[0099] [If it is difficult to specify the position where the remote participant B's image data can be overlaid] In some cases, it may be difficult to specify a location as shown in Fig. 7, such as when the location image data includes a large number of participants A. For example, in the location image data corresponding to the third location as shown in Fig. 7(b), there may be no candidate location or area (in other words, an empty seat location or an empty seat area) where remote participant B can be located.

[0100] In such a case, the image data generation unit 135 may create an empty seat area for the remote participant B by processing or modifying at least a portion of the site image data corresponding to the third site. As a non-limiting example, the image data generation unit 135 may virtually create an empty seat area in which the image of the remote participant can be placed by changing the size or shape (e.g., stretching or expanding in the length direction) of the image data corresponding to the third site table.

[0101] 9A and 9B are schematic diagrams illustrating an example of an image in which an empty seat area has been created. In FIG. 9A, a virtual table V3 has been added to table T3 at the third location, and an empty seat area Y1 is displayed around table V3. When remote participant B specifies empty seat area Y1 on this screen, the image data generator 135 generates composite image data in which an image of remote participant B is composited into the specified area, as shown in FIG. 9B.

[0102] This processing by the image data generation unit 135 reduces the number of cases where the image of remote participant B cannot be composited with the image displayed at the first location, thereby increasing the probability of having the remote participant virtually appear at the third location without compromising the realism of the conference.

[0103] The image data generating unit 135 may create an empty seat area on the condition that there is no empty seat area in the base image data, or may create an empty seat area when there is an empty seat area. As an example, the image data generating unit 135 may create an empty seat area in response to receiving an instruction from any participant to increase the empty seat area.

[0104] Furthermore, the image data generation unit 135 may arrange a virtual table in the base image data regardless of whether there is an empty seat area in the base image data. For example, the image data generation unit 135 may arrange a virtual table in the composite image data displayed on the display 24-i of the i-th base so that the table Ti arranged at the first base appears visually connected to at least one of the tables Tk at each of the multiple k-th bases in the composite image data.

[0105] Alternatively or additionally, the image data generation unit 135 may place a virtual table in the composite image data to make it appear as if the table Ti installed at the i-th location is placed close to at least one of the tables Tk at each of the multiple k-th locations displayed on the display 24-i at the i-th location, for example.

[0106] By arranging virtual tables in this way, participants at location i can feel as if location i and multiple other locations k different from location i are holding a conference in a single conference room space, further enhancing the sense of unity and realism of the conference.

[0107] [Use of captured images and avatar images] The images displayed on the displays 24 installed at each base may display captured images of each participant, or may display avatar images of each participant. The image data generation unit 135 determines whether to generate composite image data including captured image data of each participant or composite image data including avatar image data of each participant, based on the contents of settings made by the organizer or participants of the remote conference accepted by the setting acceptance unit 134, for example.

[0108] The image data generation unit 135 may unify the images of the participants displayed on the displays 24 at each location into either captured images or avatar images. As an example, when a setting is made to use the image of the participant at the i-th location as the avatar image, the image data generation unit 135 generates composite image data by combining image data including avatar image data of the participant Ai at the i-th location with avatar image data of the participant Ak at the k-th location, even if a setting is not made to use the captured image or avatar image for the image of each of the other multiple k-th locations.

[0109] On the other hand, when it is set that the image of the participant Ai at the i-th location is to be the captured image, even if a setting as to whether the image of the other participant Ak at the k-th location is to be the captured image or the avatar image is not set, the image data generation unit 135 generates composite image data by combining image data including the captured image data of the participant at the i-th location with the captured image data of the participant Ak at the k-th location. In this way, by the image data generation unit 135 generating composite image data consisting of either the captured image data or the avatar image data, it is possible to prevent the discomfort caused by the different types of images of some participants.

[0110] The image data generating unit 135 may determine whether to use captured image data or avatar image data as image data to be combined with the base image data (or transformed image data) based on the attributes of the remote conference set via the setting receiving unit 134. As an example, the image data generating unit 135 uses avatar image data as image data to be combined when the attributes of the remote conference are an internal conference, and uses captured image data when the attributes of the remote conference are a conference with people outside the company. By operating the image data generating unit 135 in this manner, for example, the second participant does not need to dress up for a remote conference when it is sufficient to display an avatar image.

[0111] [Processing flow in data processing device 1] Fig. 10 is a flowchart showing the flow of processing in the data processing device 1. The flowchart shown in Fig. 10 starts when the host of the remote conference performs an operation to start the remote conference and the setting receiving unit 134 acquires a request to start the remote conference.

[0112] When the remote conference starts, the setting receiving unit 134 receives the setting of the number of bases n (S10). The setting of the number of bases n may be received, for example, together with the request to start the remote conference, or may be received in a message different from the request.

[0113] In response to the reception of the setting of the number of locations n, the acquisition unit 131 acquires location image data from each of the n locations, and the acquisition unit 132 acquires location voice data from each of the n locations (S11). Although not shown in Fig. 10, the acquisition units 131 and 132 continuously acquire location image data and location voice data, respectively.

[0114] The site image data is input to the image data generation unit 135, and the site audio data is input to the audio data generation unit 136. The image data generation unit 135 generates a plurality of transformed image data by transforming the site image data into a shape corresponding to the number of sites n accepted by the setting acceptance unit 134 (S12). For example, when the number of sites n=3, each site image data is transformed to correspond to a triangular table layout as shown in FIGS. 3(a) and 4(a), and when the number of sites n=4, each site image data is transformed to correspond to a rectangular table layout as shown in FIGS. 3(b) and 4(b).

[0115] During this time, the setting reception unit 134 monitors whether or not a request for specifying a position at which to superimpose the participant image data for any of the multiple transformed image data to be superimposed by the image data generation unit 135 has been received (S13). If the setting reception unit 134 determines that a request for specifying a superimposition position has been received (YES in S13), the image data generation unit 135 transmits a position specification screen based on the base image data to be superimposed with the participant image data to the electronic device that sent the request (S14). After transmitting the position specification screen, if the image data generation unit 135 obtains information indicating the position specified by the participant from the electronic device that sent the request, the image data generation unit 135 determines the position indicated by the information as the superimposition position for the participant image data (S15).

[0116] If no request specifying a compositing position is received within a predetermined period of time in S13 (NO in S13), the image data generation unit 135 determines the compositing positions of the multiple transformed image data (S15). As an example, when the number of locations n=3, the image data generation unit 135 determines that the transformed image data of the locations (n-1=2) excluding the location where the composite image data is to be displayed should be composited at positions corresponding to two sides of the triangular table layout. When the number of locations n=4, the image data generation unit 135 determines that the transformed image data of the locations (n-1=3) excluding the location where the composite image data is to be displayed should be composited at positions corresponding to three sides of the rectangular table layout.

[0117] Thereafter, the image data generation unit 135 generates composite image data by arranging n-1 pieces of transformed image data at the determined combining positions (S16). Furthermore, the audio data generation unit 136 generates processed audio data based on the combining positions of each piece of transformed image data (S17). The data transmission unit 137 transmits the composite image data generated by the image data generation unit 135 and the processed audio data generated by the audio data generation unit 136 to the locations corresponding to each data (S18). Note that the generation of the composite image data (S16) may be performed after the generation of the processed audio data (S17), or may be performed in parallel with the generation of the processed audio data (S17).

[0118] If the setting reception unit 134 has not received an instruction to end the remote conference from the organizer of the remote conference (NO in S19), the acquisition units 131 and 132 acquire the base image data and the base audio data, respectively (S20), and the image data generation unit 135, the audio data generation unit 136, and the data transmission unit 137 repeat the processes from S16 to S18. If the setting reception unit 134 has received an instruction to end the remote conference from the organizer of the remote conference (YES in S19), the control unit 13 ends the process.

[0119] [Variation 1] In the above description, it may be assumed that a mismatch occurs between the setting of the number of base locations n accepted by the setting accepting unit 134 and the number of base location image data acquired by the acquiring unit 131. When such a mismatch occurs, for example, the control unit 13 may transmit an error message to the electronic device that set the number of base locations n, via the communication unit 11, to prompt the user to confirm the setting of the number of base locations n. Additionally or alternatively, the control unit 13 may transmit a message to the electronic device of the relevant base location indicating that the number of base location image data corresponding to the number of base locations n has not been received by the data processing device 1.

[0120] By this processing, even if a discrepancy occurs between the setting of the number of bases n and the number of base image data acquired by the data processing device 1, it is possible to appropriately prompt the user to review the setting of the number of bases n or check for any malfunctions in the electronic devices at the relevant base, thereby increasing the probability that a remote conference with the desired number of bases n can be started successfully.

[0121] [Effects of Data Processing Device 1] As described above, the data processing device 1 acquires multiple site image data corresponding to multiple sites, and generates multiple transformed image data for each site by transforming the site image data corresponding to the multiple other sites into shapes corresponding to the number of the multiple sites. The data processing device 1 also generates composite image data in which the multiple transformed image data are arranged at positions corresponding to the number of the multiple sites, and transmits the composite image data corresponding to each of the multiple sites to one or more electronic devices corresponding to the multiple sites. Therefore, participants at each site can experience a sense of unity or realism, as if they were participating in a remote conference together with participants at other sites.

[0122] In the above-described embodiments, "image data" is not limited to two-dimensional data but may include three-dimensional data (e.g., hologram data that represents participants in a three-dimensional manner). For example, when participants are displayed (or projected) in a three-dimensional manner using three-dimensional holography technology, the image processing described in the above-described embodiments may be applied.

[0123] In the above-described embodiments, the term "site" may be replaced with a term meaning a location where people participating in a remote conference are located, such as "base," "space," "conference room," or "room." The term "organization" may be replaced with a term meaning a gathering (group) of people, such as "group," "group," "team," or "circle." Furthermore, the term "transmission" may be replaced with "output," and the term "combination (of image data)" may be replaced with "superimposition." The term "identification" may be replaced with "detection" or "identification," for example.

[0124] Furthermore, the term "part" used in the configurations exemplified in the above-described embodiments may be interchangeably read as other terms such as "means," "circuit," or "device."

[0125] Furthermore, in the above-described embodiments, terms such as "first," "second," and "third" are used for convenience to distinguish between two or more elements, and do not limit the physical quantity, superiority or inferiority, or order of the elements to which the terms refer.

[0126] Furthermore, this invention will make it possible to contribute to Goal 9 of the United Nations' Sustainable Development Goals (SDGs), which is "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote innovation and resilience."

[0127] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by functionally or physically distributing or integrating any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combination also have the effects of the original embodiments. [Explanation of symbols]

[0128] 1 Data Processing Device 11 Communications Department 12 Storage section 13 Control Unit 21-1 to 21-n routers 22-1~22-n Computer 23-1~23-n Camera 24-1~24-n display 25-1~25-n speakers 131 Acquisition Department 132 Acquisition Department 134 Settings Reception Section 135 Image data generation unit 136 Audio data generation unit 137 Data transmission unit

Claims

1. A data processing device in a system for conducting a remote conference between multiple locations, an image data acquisition unit that acquires a plurality of base image data corresponding to the plurality of bases, the image data indicating a plurality of participants participating in the remote conference; an image data generation unit that generates, for each of the plurality of bases, a plurality of transformed image data by transforming the base image data corresponding to a plurality of other bases into shapes corresponding to the number of the plurality of bases, and generates composite image data in which the plurality of transformed image data are arranged at positions corresponding to the number of the plurality of bases; a data transmission unit that transmits the composite image data corresponding to each of the plurality of locations to one or more electronic devices corresponding to the plurality of locations; a setting reception unit that receives a setting of a base selected from the plurality of bases from a participant of the remote conference; and the image data generation unit generates the composite image data by combining the participant image data corresponding to the participant with the location image data corresponding to the location selected by the participant. Data processing device.

2. the image data generation unit generates the composite image data in which the plurality of transformed image data pieces are arranged in a number obtained by subtracting 1 from the number of the plurality of bases.

2. The data processing device according to claim 1.

3. the image data generation unit generates the composite image data in which the plurality of transformed image data, the number of which is equal to the number of the plurality of bases minus one, are arranged symmetrically.

3. The data processing device according to claim 2.

4. the image data generation unit generates the composite image data in which the plurality of transformed image data based on the plurality of base image data corresponding to the plurality of bases other than the base to which the composite image data is to be transmitted is arranged.

2. The data processing device according to claim 1.

5. the image data generation unit generates the plurality of transformed image data by performing different image processing on the face image data and the body image data included in the base image data, respectively; 2. The data processing device according to claim 1.

6. the image data generation unit generates the plurality of transformed image data by transforming the body image data based on the arrangement direction of the desk image data corresponding to each of the plurality of bases; 6. A data processing device according to claim 5.

7. a setting receiving unit that receives a setting of the number of the plurality of bases; the image data generation unit generates the plurality of transformed image data by transforming the location image data into a shape corresponding to the number of the plurality of locations accepted by the setting acceptance unit.

7. A data processing device according to claim 1.

8. a voice data acquisition unit that acquires a plurality of base voice data based on voices collected at the plurality of bases; a voice data generation unit that generates processed voice data by processing the plurality of base voice data based on the positions of the plurality of transformed image data in the composite image data; and the data transmission unit transmits, to one or more electronic devices corresponding to the plurality of locations, a plurality of the processed voice data corresponding to voices collected at a plurality of locations different from each of the plurality of locations; 2. The data processing device according to claim 1.

9. the voice data generation unit generates the processed voice data including first position voice data to be output to a first position speaker installed at a first position of at least one of the plurality of bases, and second position voice data to be output to a second position speaker installed at a second position of at least one of the plurality of bases.

9. A data processing device according to claim 8.

10. The computer executes acquiring a plurality of base image data corresponding to a plurality of bases, the base image data indicating a plurality of participants participating in the remote conference in a system for holding a remote conference among a plurality of bases; generating a plurality of transformed image data for each of the plurality of bases by transforming the base image data corresponding to a plurality of other bases into shapes corresponding to the number of the plurality of bases; generating composite image data in which the plurality of transformed image data are arranged at positions corresponding to the number of the plurality of bases; transmitting the composite image data corresponding to each of the plurality of locations to one or more electronic devices corresponding to the plurality of locations; receiving a setting of a location selected from the plurality of locations from a participant of the remote conference; and In the step of generating the composite image data, the composite image data is generated by combining the participant image data corresponding to the participant with the location image data corresponding to the location selected by the participant. Data processing methods.

11. A data processing system for conducting a remote conference between multiple locations, one or more electronic devices installed at each of the plurality of locations; an image data acquisition unit that acquires a plurality of base image data corresponding to the plurality of bases, the image data indicating a plurality of participants participating in the remote conference; an image data generation unit that generates, for each of the plurality of bases, a plurality of transformed image data by transforming the base image data corresponding to a plurality of other bases into shapes corresponding to the number of the plurality of bases, and generates composite image data in which the plurality of transformed image data are arranged at positions corresponding to the number of the plurality of bases; a data transmission unit that transmits the composite image data corresponding to each of the plurality of locations to the one or more electronic devices corresponding to the plurality of locations; a setting reception unit that receives a setting of a base selected from the plurality of bases from a participant of the remote conference; and the image data generation unit generates the composite image data by combining the participant image data corresponding to the participant with the location image data corresponding to the location selected by the participant. Data processing system.

Images