Video conferencing terminal, video conferencing system processing method and program

Abstract

The goal is to enable smooth video conferencing by allowing the settings of multiple devices to be configured simultaneously. [Solution] The video conferencing terminal 1 includes a camera processing unit 26 that generates an image to be transmitted to the other party's video conferencing terminal from an image captured by a camera 25, an audio processing unit 29 that generates audio to be transmitted to the other party's video conferencing terminal from audio collected by a microphone 27, and an input unit 4 that accepts input of a conversation mode selected by the user. The conversation mode has at least a first conversation mode and a second conversation mode. In the first conversation mode, the angle at which the image to be transmitted is captured and the angle at which the audio to be transmitted is picked up are approximately the same. In the second conversation mode, the angle at which the audio to be transmitted is picked up is set regardless of the angle at which the image to be transmitted is captured.

Description

Technical Field

[0001] The present invention relates to a video conferencing terminal, a processing method of a video conferencing system, and a program.

Background Art

[0002] There is a widespread conference system that realizes a remote conference by transmitting and receiving audio obtained by a microphone and images captured by a camera via a communication network.

[0003] The following Patent Document 1 discloses a video conferencing system that combines a panoramic camera and a microphone array, captures the entire conference room with the panoramic camera, and automatically zooms in and displays the video of the speaker when the speaker is present.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, as video conferencing and the like increase, the importance of clearly delivering one's own voice to the other party is increasing. With only the known noise cancellation function, the voices of people nearby will be included, so multiple microphones are being installed, and a function to cut off the voices of others from directions other than one's own is being advanced by the beamforming function. On the other hand, in a conference room or the like, it is necessary to deliver voices from all directions and at various distances to the other party as much as possible, and it is essential to switch the operation mode according to the conference scene.

[0006] However, simply changing the microphone's operating mode is insufficient, and other settings are often changed as well, meaning users need to adjust the behavior of each device depending on the situation. For example, regarding camera effects, a wide-angle lens and background blur may be unnecessary in a conference room, but at home, where privacy is a concern, it is preferable to narrow the camera's field of view and set a virtual background that includes background blur.

[0007] Furthermore, recently, there is a need for speaker output to be tailored to each participant in meetings held at their desks, while in conference rooms, there is a need to control the audio playback so that participants can easily hear each other regardless of their position. In other words, speaker output also requires switching between operating modes depending on the meeting scenario.

[0008] The present invention aims to provide a video conferencing terminal, a video conferencing system processing method, and a program that enable smooth video conferencing by allowing the simultaneous configuration of multiple device settings simply by selecting a conversation mode according to the video conferencing scene. [Means for solving the problem]

[0009] [1] The video conferencing terminal in this application example is a video conferencing terminal that conducts a video conference by sending and receiving images and sound with the other party's video conferencing terminal, and comprises an image generation means that generates an image to be sent to the other party's video conferencing terminal from an image captured by an imaging means, a sound generation means that generates sound to be sent to the other party's video conferencing terminal from sound collected by a sound collection means, and an input unit that accepts input of a conversation mode selected by the user, wherein the conversation mode comprises at least a first conversation mode and a second conversation mode, wherein in the first conversation mode, the angle at which the image to be sent is captured and the angle at which the sound to be sent is picked up are substantially the same, and in the second conversation mode, the angle at which the sound to be sent is picked up is set regardless of the angle at which the image to be sent is captured.

[0010] [9] In a preferred embodiment, the video conferencing terminal of this application further comprises a playback sound generation means that generates sound to be played back by a sound playback means from sound received from the other party's video conferencing terminal, wherein in the first conversation mode, the angle at which the image to be transmitted is captured, the angle at which the sound to be transmitted is picked up, and the angle at which the played-back sound is easily audible are substantially the same, and in the second conversation mode, the angle at which the sound to be transmitted is picked up and the angle at which the played-back sound is easily audible are set regardless of the angle at which the image to be transmitted is captured.

[0011]

[15] The processing method for the video conferencing system of this application example is a processing method for a video conferencing system that conducts a video conference by sending and receiving images and sound with the other party's video conferencing terminal, comprising: an input unit which receives input of a conversation mode selected by the user; an image generation processing unit which generates an image to be transmitted from an image captured by an imaging means; and a sound generation processing unit which generates a sound to be transmitted from a sound collected by a sound collection means; the conversation mode comprises at least a first conversation mode and a second conversation mode, wherein in the first conversation mode the angle at which the image to be transmitted is captured and the angle at which the sound to be transmitted is picked up are substantially the same, and in the second conversation mode the angle at which the sound to be transmitted is picked up is set regardless of the angle at which the image to be transmitted is captured.

[0012]

[16] The processing method for the video conferencing system of this application example is a processing method for a video conferencing system that conducts a video conference by sending and receiving images and sound with the other party's video conferencing terminal, comprising: an input unit which receives input of a conversation mode selected by the user; an image generation processing unit which generates an image to be transmitted from an image captured by an imaging means; an audio generation processing unit which generates an audio to be transmitted from an audio collected by an audio collection means; and a playback audio generation processing unit which generates an audio to be played back by an audio playback processing unit from an audio received from the other party's video conferencing terminal, wherein the conversation mode comprises at least a first conversation mode and a second conversation mode, wherein the first conversation mode is characterized in that the angle at which the image to be transmitted is captured, the angle at which the audio to be transmitted is picked up and the angle at which the played-back audio is easily heard are substantially the same, and the second conversation mode is characterized in that the angle at which the audio to be transmitted is picked up and the angle at which the played-back audio is easily heard are set regardless of the angle at which the image to be transmitted is captured.

[0013]

[17] The program of this example is a program for operating a video conferencing system that conducts a video conference by sending and receiving images and audio with a video conferencing terminal on the other side, and causes a computer to perform the steps of receiving input of a conversation mode selected by the user, an image generation step of generating an image to be transmitted from an image captured by an imaging means, and an audio generation step of generating audio to be transmitted from audio collected by a sound collection means, wherein the conversation mode has at least a first conversation mode and a second conversation mode, wherein in the first conversation mode the angle at which the image to be transmitted is captured and the angle at which the audio to be transmitted is picked up are substantially the same, and in the second conversation mode the angle at which the audio to be transmitted is picked up is set regardless of the angle at which the image to be transmitted is captured.

[0014]

[18] The program of this application example is a program for operating a video conferencing system that conducts a video conference by sending and receiving images and audio with a video conferencing terminal of the other party, and causes a computer to perform the steps of receiving input of a conversation mode selected by the user, generating an image to be transmitted from an image captured by an imaging means, generating an audio to be transmitted from an audio collected by a sound collection means, and generating a playback audio to be played back by an audio playback means from an audio received from a video conferencing terminal of the other party, wherein the conversation mode comprises at least a first conversation mode and a second conversation mode, wherein the first conversation mode is characterized in that the angle at which the image to be transmitted is captured, the angle at which the audio to be transmitted is picked up and the angle at which the playback audio is easily heard are substantially the same, and the second conversation mode is characterized in that the angle at which the audio to be transmitted is picked up and the angle at which the playback audio is easily heard are set regardless of the angle at which the image to be transmitted is captured. [Effects of the Invention]

[0015] The present invention provides a video conferencing terminal, a video conferencing system processing method, and a program that enable smooth video conferencing by allowing the simultaneous configuration of multiple device settings simply by selecting a conversation mode according to the video conferencing scene. [Brief explanation of the drawing]

[0016] [Figure 1] This diagram shows the configuration of a video conferencing terminal according to the embodiment. [Figure 2] This is a block diagram showing the configuration of the control unit in the video conferencing terminal according to Embodiment 1. [Figure 3] This diagram illustrates the settings for the field of view and directional angle in a video conferencing terminal according to Embodiment 1. [Figure 4] This diagram illustrates an example of a setting operation in a video conferencing terminal according to Embodiment 1. [Figure 5]This is a diagram for explaining another example of setting operations in the video conferencing terminal according to Embodiment 1. [Figure 6] This is a block diagram showing the configuration of the control unit in the video conferencing terminal according to Embodiment 2. [Figure 7] This is a diagram for explaining setting examples of the viewing angle and the directional angle of sound collection in the standard mode, private mode, privacy mode, conference room mode (2), and default in the video conferencing terminal according to Embodiment 2. [Figure 8] This is a diagram for explaining an example of speaker output in the video conferencing terminal according to Embodiment 2. [Figure 9] This is a diagram for explaining setting examples of the viewing angle and the directional angle of sound collection in the standard mode, private mode, privacy mode, conference room mode (2), and default in the video conferencing terminal according to Embodiment 3.

Modes for Carrying Out the Invention

[0017] 1. Embodiment 1 Hereinafter, the configuration of the video conferencing terminal 1 according to Embodiment 1 will be described with reference to the drawings.

[0018] <Overall Configuration of Video Conferencing Terminal 1> As shown in FIG. 1, the video conferencing terminal 1 includes an input unit 4, a camera device 15, an audio device 17, a display unit 13, a control unit 11, and a memory 12. The input unit 4, the camera device 15, the audio device 17, the display unit 13, the control unit 11, and the memory 12 are connected via a bus 3. Note that examples of the video conferencing terminal 1 include a PC (personal computer), but it is not limited thereto as long as it has the above configuration.

[0019] The input unit 4 functions as a user interface for the user to perform input operations. Depending on the conversation mode selected by the user via the input unit 4, the control unit 11 executes camera processing, audio device processing, and display processing (e.g., screen transitions) for the display unit 13 in the video conference described later. The conversation mode has a first conversation mode and a second conversation mode (standard mode), and the first conversation mode has multiple sub-conversation modes. The sub-conversation modes include a first sub-conversation mode (private mode), a second sub-conversation mode (privacy mode), and a third sub-conversation mode (conference room mode (1)).

[0020] The camera device 15 comprises an imaging means 25 (hereinafter, the "imaging means 25" may be referred to as "camera 25") and a camera processing unit 26. The camera 25 is equipped with a lens and an imaging unit (CCD or CMOS), the lens forms an image of the subject light, and the imaging unit outputs the formed subject light as R, G, and B image signals. The camera processing unit 26 is equipped with an A / D converter, an image processing LSI, memory, etc., and performs drive timing and exposure control of the camera 25, as well as functioning as an image generation means that processes the RGB image signal captured by the camera 25 (A / D conversion, etc.). For example, it generates an image to be transmitted to the other party's video conferencing terminal (not shown) by extracting an image of a predetermined range from the image captured by the camera 25. The camera processing unit 26 outputs the generated image to the control unit 11. The camera processing unit 26 also generates a virtual background image, as described later, if necessary.

[0021] The audio device 17 comprises a plurality of sound collection means 27 (hereinafter, the "sound collection means 27" may be referred to as "microphones 27"), at least one sound playback means 28 (hereinafter, the "sound playback means 28" may be referred to as "speaker 28"), and a sound processing unit 29. The microphones 27 collect sound and output the sound data to the sound processing unit 29. The speaker 28 outputs sound corresponding to the sound data output from the sound processing unit 29. The sound processing unit 29 includes an A / D converter, a D / A converter, a sound processing LSI, memory, etc., and functions as a sound generation means that processes the sound data input from the microphones 27 (A / D conversion, etc.). The sound processing unit 29 generates sound to be transmitted to the other party's video conferencing terminal (not shown) by performing directional control signal processing based on the spatial information of the sound collected by the plurality of microphones 27. The audio processing unit 29 outputs the generated audio data to the control unit 11, or performs D / A conversion on the audio data (digital) input from the control unit 11 and outputs it to the speaker 28. The audio processing unit 29 can perform noise cancellation processing as needed in standard mode, private mode, privacy mode, and conference room mode (1). In addition, in conference room mode (1), the audio processing unit 29 can adjust the sensitivity of sound pickup according to the magnitude of the audio signal.

[0022] The display unit 13 includes a display panel (not shown) and a touch panel (not shown). The display panel is a liquid crystal panel or an organic EL (Electro-Luminescence) panel, and displays operation buttons, operation menus, etc. The touch panel functions as a user interface that allows information input, is mounted on top of the display panel, and can be a resistive or capacitive type.

[0023] Memory 12 stores the camera's field of view data and the microphone 27's directivity (directional angle) for each setting mode, as described later. Memory 12 also stores setting mode programs for video conferencing (programs for setting the field of view and directional angle according to the mode) and video conferencing programs (for example, video conferencing applications), as described later.

[0024] As shown in Figure 2, the control unit 11 includes a directional control unit 101, a field of view control unit 102, a virtual background control unit 103, and a display control unit 104. By executing a setting mode program stored in memory 12, the directional control unit 101 performs a process (directional angle adjustment process) to adjust (control) the directionality (angle at which the transmitted sound is picked up) of the microphone 27 corresponding to the setting mode. The field of view control unit 102 performs a process to adjust (control) the field of view corresponding to the setting mode. The virtual background control unit 103 performs a process to set the background (blurring the user's background or using a virtual background) corresponding to the setting mode. In the private mode and privacy mode described later, the directional control unit 101 adjusts the directional angle of sound pickup so that it approximately matches the corresponding field of view.

[0025] The video conferencing system processing in the control unit 11 is carried out in the following steps. (1) The input unit 4 accepts input from the conversation mode selected by the user (reception process). (2) The camera processing unit 26 generates an image to be transmitted to the other party's video conferencing terminal from the image captured by the imaging means (image generation process). (3) The microphone 27 generates audio to be transmitted to the other party's video conferencing terminal from the audio collected by the microphone (audio generation process). (4) The conversation mode has a first conversation mode and a second conversation mode. In the first conversation mode, processing is performed to make the angle at which the transmitted image is captured and the angle at which the transmitted audio is picked up roughly the same. In the second conversation mode, processing is performed to set the angle at which the transmitted audio is picked up, regardless of the angle at which the transmitted image is captured. Furthermore, the program for executing the above-mentioned processes is pre-stored in memory 12 as a setting mode program for video conferencing.

[0026] This setting mode can be switched between standard mode, private mode, privacy mode, and conference room mode (1) by user operation via the input unit 4. The display control unit 104 has a function to transition between images of selection icons and adjustment icons for user operation (for example, the icons in Figures 4 and 5) (for example, transitions of images that highlight the selected icon).

[0027] <Types of setting modes> The following describes the standard mode, private mode, privacy mode, and conference room mode (1) that can be selected by the user. Figure 3 is a table showing examples of settings such as field of view and directional angle in standard mode, private mode, privacy mode, conference room mode (1), and the default settings.

[0028] [Standard Mode] Standard mode is a mode in which the angle at which the transmitted audio is picked up (hereinafter referred to as the "directivity angle of sound pickup") is set, regardless of the angle at which the image captured by the camera 25 that makes up the video conferencing terminal 1 is taken up (hereinafter referred to as the "angle of view"). When the video conferencing program (video conferencing app) is first launched, the display unit 13 displays the settings screen for standard mode, as shown on the left side of Figure 4. The right side of Figure 4 shows the display screen after selecting privacy mode.

[0029] When the standard mode is selected from the four setting modes at the top of the screen (standard mode, private mode, privacy mode, and conference room mode (1)), the field of view of the camera 25 and the directional angle of the microphone 27 are set to the field of view and directional angle of the microphone 27 that correspond to the standard mode, which are pre-stored in memory 12. In addition, selection icons for optional settings such as brightness correction, backlight correction, and On / Off (on / off) of the microphone 27 and speaker 28 are displayed on the display unit 13 (see Figure 4).

[0030] In the standard mode, the sound pickup directivity angle is set to capture sound from all directions (360 degrees when viewed from the vertical; hereafter, the term "viewed from the vertical" will be omitted when referring to the sound pickup directivity angle), but it can be adjusted as needed between 30 and 360 degrees. In standard mode, the field of view is set to 90 degrees when viewed from the vertical (hereafter, the term "viewed from the vertical" will be omitted when referring to the field of view), but it can also be set to a field of view other than 90 degrees. In the example in Figure 3, the field of view is set to 90 degrees and directivity is off (sound pickup directivity angle: 360 degrees). Note that this standard mode is set the same as the default (initial mode).

[0031] The captured images and recorded audio are transmitted via a communication network (not shown) to a video conferencing terminal other than your own (the other party's) terminal. In standard mode, the virtual background setting is optional, but you may set it to turn on or off as needed.

[0032] [Private Mode] Private mode is a mode that approximately matches the field of view for capturing images taken by the camera 25 that makes up the video conferencing terminal 1 with the directional angle of the microphone 27 for capturing sound, and is a mode that emphasizes the privacy effect (the effect of matching the field of view and directional angle of the microphone to the image and sound of a specific individual, thereby minimizing the influence of the images and sounds of other people). When private mode is selected from the four setting modes (standard mode, private mode, privacy mode, and conference room mode (1)) at the top of the display unit 13 screen, the field of view of the camera 25 and the directional angle of the microphone 27 are set to the field of view and directional angle of the microphone 27 that correspond to private mode and are stored in memory 12 in advance. As mentioned above, private mode is the first conversation mode and the first sub-conversation mode.

[0033] For example, if the field of view is 85 degrees, the directivity angle of sound pickup will also be set to 85 degrees. While the field of view and directivity angle range can be freely set within a range of 90 degrees to 30 degrees to achieve a suitable privacy effect, the directivity angle of sound pickup and the field of view should be approximately identical. The reason for approximately matching the field of view and the directivity angle of sound pickup is to ensure that only the voice of the user (speaker) is picked up, avoiding ambient noise and other people's voices. In other words, if the field of view is reduced, the directivity angle of sound pickup must also be reduced; otherwise, sounds from around the subject being captured will be audible, making it difficult to achieve a privacy effect in a limited space. Note that "approximately identical" means that the field of view and the directivity angle of sound pickup are matched to achieve the objective of picking up only the user's voice, and does not mean that the field of view and the sound pickup angle are strictly identical.

[0034] Furthermore, to enhance privacy, the field of view and the directional angle of sound pickup can be further reduced. For example, in the example shown in Figure 3, the field of view is set to 90 degrees, while the directional angle of sound pickup is set to 80 degrees. The field of view setting range is 30 degrees to 90 degrees, and can be freely set according to the purpose of the meeting. This setting can be stored (registered) in memory 12. By preparing several private mode setting patterns in advance, a specific private mode can be quickly set according to the purpose of the meeting. In private mode, the virtual background can be set by selecting from the virtual backgrounds stored in memory 12 by sending an instruction to the virtual background control unit 103 via the input unit 4. If there is no need to set a virtual background, it may be turned off.

[0035] [Privacy Mode] Privacy mode is a mode that approximately matches the field of view for capturing images taken by the camera 25 that makes up the video conferencing terminal 1 with the directional angle of sound pickup for capturing audio collected by the microphone 27. It further narrows the field of view range and the directional angle range of sound pickup compared to private mode to achieve a privacy effect (an effect that matches the field of view and directional angle of sound pickup to the image and voice of a specific individual, and suppresses the influence of other people's images and voices as much as possible, and further suppresses the influence of other people's images and voices than the privacy effect described above). The meaning of "approximately match" is the same as in private mode described above. When privacy mode is selected from the four setting modes (standard mode, private mode, privacy mode, and conference room mode (1)) at the top of the display unit 13 screen, the field of view of the camera 25 and the directional angle of sound pickup of the microphone 27 are set to the field of view and directional angle of sound pickup corresponding to the privacy mode that is stored in memory 12 in advance. As described above, privacy mode is the first conversation mode and the second sub-conversation mode.

[0036] For example, if the field of view is 45 degrees, the directional angle of sound pickup is also set to 45 degrees, so that the field of view and the sound pickup angle match. It is preferable to set the field of view and the directional angle of sound pickup to approximately match within the range of 30 degrees to 45 degrees to obtain a suitable privacy effect, but they may also be set within the range of 30 degrees to 90 degrees, while still approximately matching the directional angle of sound pickup and the field of view. The setting patterns for the field of view and directional angle of sound pickup in privacy mode are stored (registered) in memory 12. In privacy mode, as shown in Figure 5, the camera processing unit 26 generates the image to be transmitted by instructing the virtual background control unit 103 via the input unit 4. The camera processing unit 26 extracts the user's image from the captured image, removes the background, and then combines it with an image with a blurred background, or by combining it with a virtual background pre-stored in memory 12. The virtual background can also be turned on or off as needed (see the settings editing screen in Figure 5).

[0037] [Meeting Room Mode (1)] In conference room mode (1), the field of view for capturing images taken by the camera 25 that constitutes the video conferencing terminal 1 and the directional angle for sound pickup collected by the microphone 27 are approximately the same. When conference room mode (1) is selected from the four setting modes (standard mode, private mode, privacy mode, and conference room mode (1)) at the top of the display unit 13 screen, the field of view of the camera 25 and the directional angle for sound pickup of the microphone 27 are set to the field of view and directional angle for sound pickup corresponding to conference room mode (1) that are pre-stored in memory 12. This conference room mode (1) is automatically set when conference room mode (1) is selected by the user via the input unit 4. As mentioned above, conference room mode (1) is the first conversation mode and the third sub-conversation mode.

[0038] In conference room mode (1), the audio is processed to be clearly audible by performing directional control signal processing (beamforming) based on the spatial information of the sound collected by multiple sound collection means. Furthermore, the voices of people farther away from microphone 27 are processed to increase their signal level before transmission.

[0039] For example, if the PC is placed in the center of the table, the field of view is set to 360 degrees, and the directional angle of sound pickup is also set to 360 degrees, so that the field of view and the sound pickup angle match. However, to ensure a smooth meeting, if the PC is placed at the edge of the table, for example, the field of view may be set to 90 degrees and the directional angle of sound pickup to 360 degrees (see Figure 3), so that the field of view and the sound pickup angle do not match. Also, if the PC (camera, microphone) is placed in the center of the table, for example, it is necessary to clearly pick up the voices of people far from the PC, so the sensitivity of sound pickup needs to be adjusted according to the strength of the audio signal.

[0040] <Effects> [Effects in Embodiment 1] The video conferencing terminal 1 according to Embodiment 1 is a video conferencing terminal that conducts a video conference by sending and receiving images and audio with the other party's video conferencing terminal, and comprises a camera processing unit 26 that generates an image to be sent to the other party's video conferencing terminal from an image captured by a camera 25, an audio processing unit 29 that generates audio to be sent to the other party's video conferencing terminal from audio collected by a microphone 27, and an input unit 4 that accepts input of a conversation mode selected by the user, and the conversation mode has at least a first conversation mode (private mode, privacy mode) and a second conversation mode (standard mode), in the first conversation mode (private mode, privacy mode), the angle at which the image to be sent is captured and the angle at which the audio to be sent is picked up are substantially the same, and in the second conversation mode, the angle at which the audio to be sent is picked up is set regardless of the angle at which the image to be sent is captured. Therefore, in private mode and privacy mode, the private and privacy effects in a limited space are further emphasized, and in standard mode, it is possible to capture sound from all directions, enabling smooth video conferencing.

[0041] According to the video conferencing terminal 1 of Embodiment 1, the camera processing unit 26 generates an image to be transmitted by extracting an image of a predetermined range from the image captured by the camera 25, and the audio processing unit 29 generates an audio to be transmitted by performing directional control signal processing based on spatial information of the audio collected by the multiple microphones 27. Therefore, it is possible to crop and enhance an image within a predetermined range, while also emphasizing the sound at the target.

[0042] According to the video conferencing terminal 1 of Embodiment 1, the first conversation mode has multiple sub-conversation modes (private mode, privacy mode), and each of these sub-conversation modes has a different angle for picking up the transmitted audio. Therefore, the degree of freedom in selecting the appropriate transmitted audio according to the conversation mode is improved.

[0043] According to the video conferencing terminal 1 of Embodiment 1, it has a first sub-conversation mode (private mode) as a sub-conversation mode, and in the first sub-conversation mode, the angle at which the image to be transmitted is captured and the angle at which the sound to be transmitted is picked up are in a range of 90 degrees or less when viewed from the vertical, centered on the position of the imaging means. Therefore, it is possible to transmit suitable sound according to the private mode, which can pick up and transmit only the sound from the speaker and the vicinity of the speaker.

[0044] According to the video conferencing terminal 1 of Embodiment 1, in the second conversation mode (standard mode), the angle at which the image to be transmitted is captured is within a range of 90 degrees or less when viewed from the vertical, centered on the position of the camera 25, and the angle at which the audio to be transmitted is captured is 360 degrees when viewed from the vertical. Therefore, suitable images and audio can be transmitted according to the standard mode, which is a general type of setting.

[0045] According to the video conferencing terminal 1 of Embodiment 1, a second sub-conversation mode (privacy mode that can pick up and transmit only the voice of the speaker and the vicinity of the speaker) is further included as a sub-conversation mode. In the first sub-conversation mode (private mode), the angle at which the image to be transmitted is captured and the angle at which the voice to be transmitted is picked up are within a range of 90 degrees or less when viewed from the vertical, centered on the position of the camera 25. In the second sub-conversation mode (privacy mode), the angle at which the image to be transmitted is captured and the angle at which the voice to be transmitted is within a range of 45 degrees or less when viewed from the vertical, centered on the position of the camera 25. Therefore, suitable images and voices can be transmitted according to the private mode, and suitable images and voices can be transmitted according to the privacy mode.

[0046] According to the video conferencing terminal 1 of Embodiment 1, in the first sub-conversation mode (private mode) and the second sub-conversation mode (privacy mode), the camera processing unit 26 extracts the user's image from the captured image, removes the background, and synthesizes an image with a blurred background or synthesizes a virtual background to generate an image to be transmitted. Therefore, depending on the private mode and privacy mode, the private effect and privacy effect are emphasized, and only the user's voice can be appropriately picked up without picking up ambient sounds or the voices of others.

[0047] According to the video conferencing terminal 1 of Embodiment 1, a third sub-conversation mode (conference room mode (1)) is provided as a sub-conversation mode. In the third sub-conversation mode (conference room mode (1)), the angle at which the image to be transmitted is captured and the angle at which the sound to be transmitted is picked up is 360 degrees when viewed from the vertical, centered on the position of the camera 25. In the third sub-conversation mode, the audio processing unit 29 adjusts the sensitivity of sound pickup according to the magnitude of the audio signal. Therefore, images suitable for conference room mode (1) and suitable sound can be transmitted.

[0048] 2. Embodiment 2 Next, the configuration of the video conferencing terminal 1A according to Embodiment 2 will be described with reference to Figures 1, 4, and 6 to 8. Figure 6 is a block diagram showing the configuration of the control unit 11 provided in the video conferencing terminal 1A.

[0049] The video conferencing terminal 1A according to Embodiment 2 differs from the video conferencing terminal 1 according to Embodiment 1 in that it includes a playback sound generation means that generates sound to be played back by the sound playback means from the sound received from the other party's video conferencing terminal, and has a conference room mode (2) instead of conference room mode (1). In other respects, it is the same as the video conferencing terminal 1 according to Embodiment 1. Therefore, in describing the video conferencing terminal 1A, we will explain the differences in configuration from video conferencing terminal 1, and will omit explanations of points that are common to the configuration of video conferencing terminal 1.

[0050] The audio device 17 of the video conferencing terminal 1A includes two or more speakers 28 (see Figure 1). Each of the two or more speakers 28 outputs sound according to the audio data output from the audio processing unit 29.

[0051] The audio processing unit 29 includes an A / D converter, a D / A converter, an audio processing LSI, memory, etc. The audio processing unit 29 functions as an audio generation means for processing audio data input from the microphone 27, and also functions as a playback audio generation means for generating audio to be played back by the speaker 28 from audio received from the other party's video conferencing terminal.

[0052] The audio processing unit 29, acting as a means for generating playback audio, performs acoustic control on the audio received from the other party's video conferencing terminal according to the conversation mode selected by the user (see Figure 4). Specifically, by performing acoustic control, the audio processing unit 29 can adjust the angle at which the audio played back by the speaker 28 is easily heard, from a small angle such as 30 degrees in front of the video conferencing terminal 1A to 360 degrees, which is the entire circumference of the video conferencing terminal 1A. This makes it possible to substantially match the directional angle of sound pickup with the directional angle of playback.

[0053] As shown in Figure 6, the control unit 11 includes a microphone directivity control unit 101A, a field of view control unit 102, a speaker directivity control unit 105A, a virtual background control unit 103, and a display control unit 104. The microphone directivity control unit 101A in the video conferencing terminal 1A corresponds to the directivity control unit 101 in the video conferencing terminal 1.

[0054] The microphone directivity control unit 101A performs processing to adjust the directivity (angle at which the transmitted sound is picked up) of the microphone 27 to correspond to the mode set by the user. The speaker directivity control unit 105A performs processing to adjust the angle at which the sound played back by the speaker 28 is easily heard in response to the sound received from the other party's video conferencing terminal so that the playback directivity angle corresponds to the mode set by the user.

[0055] The video conferencing terminal 1A has a first conversation mode and a second conversation mode. In the first conversation mode, the field of view, the directional angle for sound pickup, and the directional angle for playback are set to be approximately the same. On the other hand, in the second conversation mode, the directional angle for sound pickup and the directional angle for playback are set regardless of the field of view.

[0056] The first conversation mode comprises a first sub-conversation mode and a second sub-conversation mode. The second conversation mode also comprises a fourth sub-conversation mode and a fifth sub-conversation mode.

[0057] In the first sub-conversation mode, the field of view, the directivity angle of sound pickup, and the directivity angle of playback are within a range of 90 degrees or less, centered on the position of the imaging means, when viewed from the vertical. In the following description, the first sub-conversation mode of the first conference mode will be referred to as "private mode."

[0058] In the second sub-conversation mode, the field of view, the directivity angle of sound pickup, and the directivity angle of playback are within a range of 45 degrees or less, centered on the position of the imaging device, when viewed from the vertical. In the following description, the second sub-conversation mode of the first conversation mode will be referred to as "privacy mode."

[0059] In the fourth sub-conference mode, the field of view is within a range of 90 degrees or less when viewed from the vertical, centered on the position of the imaging device. The directional angle for sound pickup and playback is 360 degrees when viewed from the vertical. In the following explanation, the fourth sub-conversation mode of the second conference mode will be referred to as "conference room mode (2)".

[0060] In the fifth sub-conference mode, the field of view, when viewed from the vertical, is within a range of 90 degrees or less, centered on the position of the imaging device. The directivity angle for sound pickup is 360 degrees when viewed from the vertical, and the directivity angle for playback is within a range of 90 degrees or less. In the following explanation, the fifth sub-conversation mode of the second conference mode will be referred to as the "standard mode."

[0061] The video conferencing terminal 1A has the four setting modes described above (standard mode, private mode, privacy mode, and conference room mode (2)) that can be selected by the user. The three setting modes other than conference room mode (2) are the same as those of video conferencing terminal 1, except that the directional angle of playback in the playback sound generation means is set to the numerical range described above. For this reason, only conference room mode (2) will be described here.

[0062] [Conference Room Mode (2)] Conference Room Mode (2) is a mode in which the angle for capturing the image taken by the camera 25 that makes up the video conferencing terminal 1A is set, and the angle at which the sound to be transmitted is picked up (sound pickup directional angle) and the angle at which the sound played back by the speaker 28 is easily heard (playback directional angle) are set. In Conference Room Mode (2), the sound pickup directional angle and the playback directional angle are set to 360 degrees.

[0063] When Conference Room Mode (2) is selected from the four setting modes (Standard Mode, Private Mode, Privacy Mode, and Conference Room Mode (2)), the field of view of the camera 25, the directional angle of the microphone 27 for sound pickup, and the directional angle of the speaker 28 for playback are set to the field of view, sound pickup directional angle, and playback directional angle corresponding to Conference Room Mode (2) stored in memory 12. In Conference Mode 2 according to Embodiment 2, the field of view is set to 90 degrees, the directional angle of sound pickup to 360 degrees, and the directional angle of playback to 360 degrees (see Figure 7, however, Figure 7 does not show the numerical value of the playback directional angle).

[0064] In conference room mode (2), the audio processing unit 29 performs sound control on the audio received from the other party's video conferencing terminal. Specifically, the audio processing unit 29 adjusts the audio played by the two or more speakers 28 so that the audio played by the speakers 28 is easily audible even at two or more positions at different angles from the video conferencing terminal 1A. By adjusting the audio played by the two or more speakers 28, it is possible to provide audio that is easy to hear even for people participating in the meeting at two or more positions at different angles from the video conferencing terminal 1A.

[0065] In a preferred embodiment, the audio processing unit 29 adjusts the audio played by the two or more speakers 28 so that the audio played by the two or more speakers 28 is easily audible at an angle that is easily audible in all directions (360 degrees) around the video conferencing terminal 1A (see Figure 8, speaker output omnidirectional sound mode: On). This makes it possible to provide easily audible audio to people participating in the conference from any direction (position) relative to the video conferencing terminal 1A.

[0066] In conference room mode (2), it is preferable to set the field of view, i.e., the range in which the image to be transmitted is captured, to an arbitrary angle, depending on the type of camera 25 used and the circumstances under which the camera 25 is used in the meeting. When using a 360-degree camera capable of capturing images in all directions as the camera 25, the field of view can be set to a range of 360 degrees or less, centered on the position of the imaging means, when viewed from the vertical direction. By setting the field of view to a range of 360 degrees or less, centered on the position of the imaging means, all participants in the meeting can be captured by the camera 25 and transmitted to the other party's video conferencing terminal. In addition, the audio received from the other party's video conferencing terminal can be acoustically controlled to make it easy to hear for all participants in the meeting.

[0067] The field of view may be set to a range of 90 degrees or less, centered on the position of the imaging means, when viewed from the vertical direction. By setting the field of view to a range of 90 degrees or less, centered on the position of the imaging means, the conference room mode (2) can be applied even when using a camera 25 with an angle range that can capture approximately 90 degrees. In addition, even for people participating in the conference outside the shooting range of camera 25, the audio received from the other party's video conferencing terminal can be provided with sound control to make it easier to hear.

[0068] Furthermore, in conference room mode (2), similar to conference room mode (1) in Embodiment 1, it is preferable to process the audio so that it can be heard clearly by performing signal processing (beamforming) to control sound collection based on the spatial information of the sound collected by the multiple sound collection means (microphones 27).

[0069] Furthermore, it is preferable that the voice generation means of the voice processing unit 29 adjusts the sensitivity of sound pickup according to the magnitude of the collected voice signal. For example, since the voice of a person far from the sound pickup means (microphone 27) tends to have a lower signal level, it is preferable to perform a process to increase the signal level (Far-field Boost) before transmitting it to the other party's video conferencing terminal (see Figure 7, microphone / Far-field Boost: On).

[0070] [Effects in Embodiment 2] [9]According to the video conferencing terminal 1A of Embodiment 2, the terminal further comprises a playback sound generation means that generates sound to be played back by a sound playback means (speaker 28) from sound received from the other party's video conferencing terminal. In the first conversation mode, the field of view is set so that the transmission directivity angle and the playback directivity angle are substantially the same. In the second conversation mode, the sound pickup directivity angle and the playback directivity angle are set regardless of the field of view. The video conferencing terminal 1A is equipped with a playback audio generation means, and furthermore, in the second conversation mode, regardless of the field of view, the directional angle for sound pickup and the directional angle for playback are set, so that even people participating in the conference from positions other than directly in front of the video conferencing terminal 1A (outside the field of view) can receive audio from the other party's video conferencing terminal with sound control so that it is easy to hear.

[0071]

[10] In a preferred embodiment of the video conferencing terminal 1A, the field of view (angle from which the transmitted image is captured) in the second conversation mode is within a range of 360 degrees or less when viewed from the vertical, centered on the position of the imaging means. This allows all participants in the meeting to be photographed by the camera 25 and transmitted to the other party's video conferencing terminal. Furthermore, the audio received from the other party's video conferencing terminal can be provided to all participants in the meeting with sound control to ensure clear and easy listening.

[0072]

[11] In a preferred embodiment of the video conferencing terminal 1A, the field of view (angle from which the transmitted image is captured) in the second conversation mode is within a range of 90 degrees or less, centered on the position of the imaging means when viewed from the vertical. This allows the conference room mode (2) to be applied even when a camera 25 with a shooting angle range of about 90 degrees is used. Furthermore, the audio received from the other party's video conferencing terminal can be provided to people participating in the conference outside the shooting range of the camera 25 with sound control to make it easier to hear.

[0073]

[12] In a preferred embodiment of the video conferencing terminal 1A, there is a conference room mode (2) (fourth sub-conference mode), and the playback sound generation means performs sound control in conference room mode (2) so that the sound is easily audible at two or more positions that are at different angles from the video conferencing terminal when viewed from the vertical. This makes it possible to provide easily audible sound to people participating in the conference at two or more positions that are at different angles from the video conferencing terminal 1A.

[0074]

[13] In a preferred embodiment of the video conferencing terminal 1A, in conference room mode (2) (fourth sub-conversation mode), the sound generation means preferably adjusts the sensitivity of sound pickup according to the magnitude of the collected sound signal. This allows the sound picked up by the sound pickup means to be adjusted to a suitable volume and transmitted to the other party's video conferencing terminal in conference room mode (2).

[0075] 3. Embodiment 3 Next, the video conferencing terminal 1B according to Embodiment 3 will be described with reference to Figures 1 and 9. Figure 9 is a block diagram showing the configuration of the control unit 11 provided in the video conferencing terminal 1A.

[0076] The video conferencing terminal 1B according to Embodiment 2 differs from the video conferencing terminal 1A according to Embodiment 2 in that it has a means for controlling the "directivity angle of playback," but is otherwise the same as the video conferencing terminal 1A according to Embodiment 2. Therefore, in describing the video conferencing terminal 1B, we will explain the differences in configuration from that of video conferencing terminal 1A, and will omit explanations of aspects that are common to the configuration of video conferencing terminal 1B and that are common to that of video conferencing terminal 1A as appropriate.

[0077] The video conferencing terminal 1B is equipped with a directional speaker as speaker 28 (see Figure 1). A directional speaker is a speaker that can deliver sound only to a specific area. By using a directional speaker as speaker 28, clear sound can be delivered to the targeted area. Examples of directional speakers include line array speakers, plane wave speakers, ultrasonic speakers, etc.

[0078] The video conferencing terminal 1B uses a directional speaker in Privacy Mode and Private Mode, out of the four setting modes (Standard Mode, Private Mode, Privacy Mode, and Conference Room Mode (2)) (indicated as "On" in the "Directional Speaker" column of Figure 9).

[0079] By using a directional speaker, it becomes possible to control the directivity of the audio being played in the direction in front of the video conferencing terminal 1B. In other words, by limiting the angle range in which the speaker 28 delivers audio to a narrow angle in front of the video conferencing terminal 1B, the directivity angle of playback can be made to roughly match the angle of view and the directivity angle of sound pickup, thereby realizing a privacy mode or private mode for the speaker 28.

[0080] On the other hand, in Standard mode and Conference Room mode (2), the directional speaker is not used (indicated as "Off" in the "Directional Speaker" column of Figure 9). This allows the audio received from the other party's video conferencing terminal to be played back and provided by the speaker 28 in an easily audible format, even to people who are not directly in front of the video conferencing terminal 1B.

[0081] [Effects in Embodiment 3] According to the video conferencing terminal 1B of Embodiment 3, a directional speaker is further provided, and in the first conversation mode, the directional angle of sound pickup (the angle at which the transmitted sound is picked up) and the directional angle of sound in the directional speaker are substantially the same. By using a directional speaker, it becomes possible to control the directivity of the audio being played in the direction in front of the video conferencing terminal 1B. In other words, by limiting the angle range in which the speaker 28 plays audio to a narrow angle in front of the video conferencing terminal 1B, the directivity angle of playback can be made to roughly match the angle of view and the directivity angle of sound pickup, thereby realizing a privacy mode or private mode for the speaker 28.

[0082] 4. Embodiment 4 Next, the processing method (1) for the video conferencing system according to Embodiment 4 will be explained with reference to Figures 1 to 5. The processing method (1) for the video conferencing system is a processing method for a video conferencing system that can be suitably applied to a video conferencing system using a video conferencing terminal 1.

[0083] The video conferencing system processing method (1) is a video conferencing system processing method that performs video conferencing by sending and receiving images and audio with a video conferencing terminal on the other side (not shown). The video conferencing system processing method (1) includes an input receiving process for the conversation mode selected by the user, an image generation process for generating an image to be transmitted from an image captured by the imaging means, and an audio generation process for generating audio to be transmitted from audio collected by the sound collection means.

[0084] The conversation mode has a first conversation mode and a second conversation mode (standard mode). In the first conversation mode, the field of view (the angle at which the transmitted image is captured) and the sound pickup directional angle (the angle at which the transmitted sound is picked up) are set to approximately match. In the second conversation mode, the sound pickup directional angle is set regardless of the field of view.

[0085] The first conversation mode has multiple sub-conversation modes. These sub-conversation modes include the first sub-conversation mode (private mode), the second sub-conversation mode (privacy mode), and the third sub-conversation mode (conference room mode (1)).

[0086] Figure 5 shows examples of the field of view and sound pickup directional angle in four modes (Standard Mode, Private Mode, Privacy Mode, and Conference Room Mode (1)).

[0087] [Effects in Embodiment 4] According to the processing method (1) of the video conferencing system according to Embodiment 4, in the first conversation mode (private mode, privacy mode), the angle at which the transmitted image is captured and the angle at which the transmitted audio is picked up are approximately the same. This further enhances the private and privacy effects in a limited space. In the second conversation mode, the angle at which the transmitted audio is picked up is set regardless of the angle at which the transmitted image is captured. This makes it possible to pick up audio from all directions, enabling smooth video conferencing.

[0088] 4. Embodiment 5 Next, the processing method (2) for the video conferencing system according to Embodiment 5 will be described with reference to Figures 1 and 6 to 9. The processing method (2) for the video conferencing system is a processing method for a video conferencing system that can be suitably applied to a video conferencing system using a video conferencing terminal 1.

[0089] The processing method (2) for the video conferencing system according to Embodiment 5 differs from the processing method (1) for the video conferencing system according to Embodiment 4 in that the playback audio generation processing unit performs playback audio generation processing to generate audio to be played back by the audio playback processing unit from the audio received from the other party's video conferencing terminal, and that it has a conference room mode (2) instead of conference room mode (1). In other respects, it is the same as the processing method (1) for the video conferencing system according to Embodiment 4. Therefore, in describing the processing method (2) for the video conferencing system, we will explain the differences from the configuration of the processing method (1) for the video conferencing system, and will omit explanations of points that are common to the configuration of the processing method (1) for the video conferencing system.

[0090] The video conferencing system processing method (2) is a video conferencing system processing method for conducting a video conference by sending and receiving images and audio with a video conferencing terminal of the other party (not shown), and further includes reception processing, image generation processing, audio generation processing, and playback audio generation processing. The playback audio generation processing is the process of generating playback audio from audio received from the other party's video conferencing terminal.

[0091] The video conferencing system processing method (2) has a first conversation mode and a second conversation mode. In the first conversation mode, the field of view (the angle at which the transmitted image is captured), the sound pickup directionality angle (the angle at which the transmitted sound is picked up), and the playback directionality angle (the angle at which the played-back sound is easily heard) are set to be approximately the same. In the second conversation mode, the sound pickup directionality angle and the playback directionality angle are set independently of the field of view.

[0092] The first conversation mode has multiple sub-conversation modes. These sub-conversation modes include the first sub-conversation mode (private mode) and the second sub-conversation mode (privacy mode).

[0093] The second conversation mode has multiple sub-conversation modes. These sub-conversation modes include a fourth conversation mode (conference room mode (2)) and a fifth conversation mode (standard mode).

[0094] Figure 7 shows examples of the field of view and sound pickup directional angle for the four modes (Standard Mode, Private Mode, Privacy Mode, and Conference Room Mode (2)).

[0095] [Effects in Embodiment 5] According to the processing method (2) of the video conferencing system according to Embodiment 5, in the first conversation mode (private mode, privacy mode), the private effect and privacy effect can be further emphasized. On the other hand, in conference room mode (2), even for people participating in the conference from a position other than directly in front of the video conferencing terminal 1A, the audio received from the other party's video conferencing terminal can be controlled to make the audio easier to hear.

[0096] 6. Embodiment 6 Next, program (1) according to embodiment 6 will be described with reference to Figures 1 to 5. Program (1) is a program that can be suitably applied to a video conferencing system using video conferencing terminal 1A and / or video conferencing terminal 1B.

[0097] Program (1) is a program that causes a computer to operate a video conferencing system that conducts video conferencing by sending and receiving images and audio with the other party's video conferencing terminal, and causes the computer to perform the steps of receiving input of a conversation mode selected by the user, generating an image from an image captured by an imaging means to generate an image to be transmitted, and generating an audio from an audio collected by a sound collection means to generate an audio to be transmitted.

[0098] The conversation mode includes at least a first conversation mode and a second conversation mode (standard mode). In the first conversation mode, the field of view (the angle at which the transmitted image is captured) and the sound pickup directional angle (the angle at which the transmitted sound is picked up) are set to approximately coincide. In the second conversation mode, the sound pickup directional angle is set independently of the field of view.

[0099] The first conversation mode has multiple sub-conversation modes. These sub-conversation modes include the first sub-conversation mode (private mode), the second sub-conversation mode (privacy mode), and the third sub-conversation mode (conference room mode (1)).

[0100] Figure 5 shows examples of the field of view and sound pickup directional angle in four modes (Standard Mode, Private Mode, Privacy Mode, and Conference Room Mode (1)).

[0101] [Effects in Embodiment 6] According to the program (1) of Embodiment 6, in the first conversation mode (private mode, privacy mode), the angle at which the transmitted image is captured and the angle at which the transmitted audio is picked up are approximately the same. This further enhances the private and privacy effects in a limited space. In the second conversation mode, the angle at which the transmitted audio is picked up is set regardless of the angle at which the transmitted image is captured. This makes it possible to pick up audio from all directions, enabling smooth video conferencing.

[0102] 7. Embodiment 7 Next, program (2) according to Embodiment 7 will be described with reference to Figures 1 and 6 to 9. Program (2) is a program that can be suitably applied to a video conferencing terminal 1A and / or a video conferencing system using video conferencing terminal 1A.

[0103] Note that program (2) according to Embodiment 7 differs from program (1) according to Embodiment 6 in that it causes the computer to execute a playback sound generation step that generates sound to be played back by the sound playback means from the sound received from the other party's video conferencing terminal, and that the conversation mode has a conference room mode (2) instead of conference room mode (1). Otherwise, it is the same as program (1) according to Embodiment 6. Therefore, in describing program (2), we will explain the differences in configuration from program (1), and will omit explanations of points that are common to the configuration of program (1) as appropriate.

[0104] Program (2) is a program for operating a video conferencing system that conducts video conferencing by sending and receiving images and audio with a video conferencing terminal of the other party (not shown), and causes the computer to execute an input receiving process for receiving the conversation mode selected by the user, an image generation process that generates an image to be transmitted from an image captured by an imaging means, an audio generation process that generates an audio to be transmitted from an audio collected by a sound collection means, and a playback audio generation process that generates an audio to be played back from an audio received from the other party's video conferencing terminal.

[0105] Program (2) has a first conversation mode and a second conversation mode. In the first conversation mode, the field of view (the angle at which the transmitted image is captured), the sound pickup directionality angle (the angle at which the transmitted sound is picked up), and the playback directionality angle (the angle at which the played-back sound is easily heard) are set to be approximately the same. In the second conversation mode, the sound pickup directionality angle and the playback directionality angle are set independently of the field of view.

[0106] The first conversation mode has multiple sub-conversation modes. These sub-conversation modes include the first sub-conversation mode (private mode) and the second sub-conversation mode (privacy mode).

[0107] The second conversation mode also has multiple sub-conversation modes. These sub-conversation modes include a fourth conversation mode (conference room mode (2)) and a fifth conversation mode (standard mode).

[0108] Figure 7 shows examples of the field of view and sound pickup directional angle for the four modes (Standard Mode, Private Mode, Privacy Mode, and Conference Room Mode (2)).

[0109] [Effects in Embodiment 7] According to the program (2) of Embodiment 7, in the first conversation mode (private mode, privacy mode), the private effect and privacy effect are further emphasized, and in the conference room mode (2), the audio received from the other party's video conferencing terminal can be controlled to make the audio easier to hear for people participating in the conference from positions other than directly in front of the video conferencing terminal 1A.

[0110] It should be noted that the present invention is not limited to the embodiments described above, and various modifications can be made without departing from the spirit of the invention. [Explanation of Symbols]

[0111] 1,1A,1B…Video conferencing terminal, 3…Bus, 4…Input unit, 11…Control unit, 12…Memory, 13…Display unit, 15…Camera device, 17…Audio device, 25…Camera, 26…Camera processing unit, 27…Microphone, 28…Speaker, 29…Audio processing unit, 101…Directional control unit, 101A…Microphone directivity control unit, 102…Field of view control unit, 103…Virtual background control unit, 104…Display control unit, 105A…Speaker directivity control unit

Claims

1. A video conferencing terminal that conducts video conferencing by sending and receiving images and audio with the other party's video conferencing terminal, Image generation means that generates an image to be transmitted to the other party's video conferencing terminal from an image captured by the imaging means, A sound generation means generates audio to be transmitted to the other party's video conferencing terminal from the audio collected by the sound collection means, It includes an input section that accepts input for the conversation mode selected by the user, The aforementioned conversation mode comprises at least a first conversation mode and a second conversation mode. In the first conversation mode described above, the angle at which the image to be transmitted is captured and the angle at which the audio to be transmitted is picked up are approximately the same. A video conferencing terminal characterized in that, in the second conversation mode, the angle for capturing the transmitted audio is set regardless of the angle at which the transmitted image is captured.

2. A video conferencing terminal according to claim 1, The image generation means generates an image to be transmitted by extracting an image from a predetermined range from the image captured by the imaging means. The video conferencing terminal is characterized in that the voice generation means generates voice to be transmitted by performing directional control signal processing based on spatial information of voice collected by a plurality of the sound collection means.

3. A video conferencing terminal according to claim 1, The first conversation mode has a plurality of sub-conversation modes, A video conferencing terminal characterized in that each of the multiple sub-conversation modes has a different angle for picking up the audio to be transmitted.

4. A video conferencing terminal according to claim 3, The aforementioned sub-conversation mode has a first sub-conversation mode, A video conferencing terminal characterized in that, in the first sub-conversation mode, the angle at which the image to be transmitted is captured and the angle at which the sound to be transmitted is picked up are within a range of 90 degrees or less when viewed from the vertical, centered on the position of the imaging means.

5. A video conferencing terminal according to claim 1, A video conferencing terminal characterized in that, in the second conversation mode, the angle at which the transmitted image is captured is within a range of 90 degrees or less when viewed from the vertical, centered on the position of the imaging means, and the angle at which the transmitted audio is picked up is 360 degrees when viewed from the vertical.

6. A video conferencing terminal according to claim 4, The aforementioned sub-conversation mode further comprises a second sub-conversation mode, In the first sub-conversation mode, the angle at which the image to be transmitted is captured and the angle at which the audio to be transmitted is picked up are within a range of 90 degrees or less, centered on the position of the imaging means, when viewed from the vertical direction. A video conferencing terminal characterized in that, in the second sub-conversation mode, the angle at which the image to be transmitted is captured and the angle at which the sound to be transmitted is picked up are within a range of 45 degrees or less when viewed from the vertical, centered on the position of the imaging means.

7. A video conferencing terminal according to claim 6, A video conferencing terminal characterized in that, in the first sub-conversation mode and the second sub-conversation mode, the image generation means extracts an image of the user with the background removed from a captured image, and generates an image to be transmitted by combining the image with the background blurred or by combining a virtual background.

8. A video conferencing terminal according to claim 4, The aforementioned sub-conversation mode has a third sub-conversation mode, In the third sub-conversation mode described above, the angle at which the image to be transmitted is captured and the angle at which the audio to be transmitted is picked up is 360 degrees when viewed from the vertical, centered on the position of the imaging means. In the third sub-conversation mode, the video conferencing terminal is characterized in that the voice generation means adjusts the sensitivity of sound pickup according to the magnitude of the voice signal.

9. A video conferencing terminal according to claim 1, The system further includes a playback audio generation means that generates audio to be played back by an audio playback means from audio received from the other party's video conferencing terminal, In the first conversation mode described above, the angle at which the image to be transmitted is captured, the angle at which the audio to be transmitted is picked up, and the angle at which the played audio is easily audible are approximately the same. The video conferencing terminal is characterized in that, in the second conversation mode, regardless of the angle at which the transmitted image is captured, the angle for capturing the transmitted audio and the angle at which the played-back audio is easily audible are set.

10. A video conferencing terminal according to claim 9, A video conferencing terminal characterized in that, in the second conversation mode, the angle at which the transmitted image is captured is within a range of 360 degrees or less when viewed from the vertical, centered on the position of the imaging means, and the angle at which the transmitted audio is picked up and the angle at which the reproduced audio is easily audible is 360 degrees when viewed from the vertical.

11. A video conferencing terminal according to claim 10, A video conferencing terminal characterized in that, in the second conversation mode, the angle at which the transmitted image is captured is within a range of 90 degrees or less when viewed from the vertical, centered on the position of the imaging means, and the angle at which the transmitted audio is picked up and the angle at which the reproduced audio is easily audible is 360 degrees when viewed from the vertical.

12. A video conferencing terminal according to claim 11, The video conferencing terminal is characterized in that the playback sound generation means has a fourth sub-conversation mode that performs acoustic control so that the sound is easier to hear at two or more positions that are at different angles from the center of the video conferencing terminal when viewed from the vertical direction.

13. A video conferencing terminal according to claim 12, In the fourth sub-conversation mode, the video conferencing terminal is characterized in that the voice generation means adjusts the sensitivity of sound pickup according to the magnitude of the voice signal.

14. A video conferencing terminal according to claim 9, Equipped with directional speakers, The audio playback means is characterized in that, in the first conversation mode, the angle at which the transmitted audio is picked up and the angle of directivity of the audio in the directional speaker are substantially the same.

15. In a processing method for a video conferencing system that conducts a video conference by sending and receiving images and audio with the other party's video conferencing terminal, The input unit accepts input from the conversation mode selected by the user, The image generation processing unit performs an image generation process that generates an image to be transmitted from the image captured by the imaging means, The audio generation processing unit performs audio generation processing to generate audio to be transmitted from the audio collected by the sound collection means. The aforementioned conversation mode comprises at least a first conversation mode and a second conversation mode. In the first conversation mode described above, the angle at which the image to be transmitted is captured and the angle at which the audio to be transmitted is picked up are approximately the same. The second conversation mode is a processing method for a video conferencing system, characterized in that the angle for capturing the transmitted audio is set regardless of the angle at which the transmitted image is captured.

16. In a processing method for a video conferencing system that conducts a video conference by sending and receiving images and audio with the other party's video conferencing terminal, The input unit accepts input from the conversation mode selected by the user, The image generation processing unit performs an image generation process that generates an image to be transmitted from the image captured by the imaging means, The audio generation processing unit performs an audio generation process that generates audio to be transmitted from the audio collected by the sound collection means, The playback audio generation processing unit performs playback audio generation processing, which generates audio to be played back by the audio playback processing unit from the audio received from the other party's video conferencing terminal. The aforementioned conversation mode comprises at least a first conversation mode and a second conversation mode. In the first conversation mode described above, the angle at which the image to be transmitted is captured, the angle at which the audio to be transmitted is picked up, and the angle at which the played audio is easily audible are approximately the same. The second conversation mode is a processing method for a video conferencing system, characterized in that the angle for capturing the transmitted audio and the angle at which the played-back audio is easily heard are set, regardless of the angle at which the transmitted image is captured.

17. Video conferencing is conducted by sending and receiving images and audio between the other party's video conferencing terminal. A program for making a video conferencing system functional, On the computer, A step to accept input from the user regarding the selected conversation mode, Image generation step of generating an image to be transmitted from an image captured by the imaging means, The sound collection means performs a sound generation step that generates audio to be transmitted from the audio it has collected, The aforementioned conversation mode comprises at least a first conversation mode and a second conversation mode. In the first conversation mode described above, the angle at which the image to be transmitted is captured and the angle at which the audio to be transmitted is picked up are approximately the same. The second conversation mode is a program characterized in that the angle at which the transmitted audio is picked up is set regardless of the angle at which the transmitted image is captured.

18. Video conferencing is conducted by sending and receiving images and audio between the other party's video conferencing terminal. A program for making a video conferencing system functional, On the computer, A step to accept input from the user regarding the selected conversation mode, Image generation step of generating an image to be transmitted from an image captured by the imaging means, A sound generation step in which the sound collection means generates sound to be transmitted from the sound collected, The process involves executing a playback audio generation step, which generates audio to be played back by an audio playback means from audio received from the other party's video conferencing terminal, The aforementioned conversation mode comprises at least a first conversation mode and a second conversation mode. In the first conversation mode described above, the angle at which the image to be transmitted is captured, the angle at which the audio to be transmitted is picked up, and the angle at which the played audio is easily audible are approximately the same. The second conversation mode is a program characterized by setting an angle for capturing the transmitted audio and an angle at which the played-back audio is easily audible, regardless of the angle at which the transmitted image is captured.

Images