Conference system, display control method, and program

Abstract

To provide a conference system for closing-up a video image of a speaker at an own point, capable of suppressing a close-up of a person at the other point, who was displayed on a display on the basis of a result of an error recognition as the speaker at the own point.SOLUTION: A conference system is a conference system containing a first device having a camera and a microphone, and a second device having a display. The first device includes: a direction detection part that detects a direction of a speaker on the basis of a voice acquired from the microphone; an image extraction part that extracts an image of the speaker from an image photographed by the camera on the basis of a direction of the speaker; and a close-up part that outputs the image of the speaker by closing up. The close-up part abandons the close-up of the image of the speaker in the case where the image of the speaker is an image displayed onto the display.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to a conference system, a display control method, and a program. [Background Art]

[0002] Conference systems that realize remote conferences by transmitting and receiving audio acquired by a microphone and images captured by a camera via a communication network have become widespread.

[0003] Further, Patent Document 1 describes a technology in which a panoramic camera and a microphone array are combined, the entire conference room is captured by the panoramic camera, and when there is a speaker, an image of the speaker is automatically close-up displayed. [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] However, with conventional technologies, for example, when a web conference is held using a large display such as an IWB (Interactive White Board), a person at another site displayed on the display may be erroneously recognized as a speaker at the local site and be close-up displayed.

[0005] An embodiment of the present invention has been made in view of the above problem. In a conference system that close-ups an image of a speaker at a local site, the embodiment suppresses the occurrence of a situation where a person at another site displayed on a display is erroneously recognized as a speaker at the local site and is close-up displayed. [Means for Solving the Problem]

[0006] To solve the above problems, a conference system according to one embodiment of the present invention is a conference system including a first device equipped with a camera and a microphone, and a second device equipped with a display, wherein the first device includes a direction detection unit that detects the direction of a speaker based on the sound acquired by the microphone, an image extraction unit that extracts an image of the speaker from an image captured by the camera based on the direction of the speaker, and a close-up unit that outputs a close-up of the speaker's image. A communication unit that communicates with the second device, the communication unit receiving from the second device a display image displayed by the second device on the display or an image of a second speaker included in the display image, and a comparison unit that compares an image of a first speaker extracted by the image extraction unit with an image of the second speaker included in the display image, The close-up section has, The degree of agreement between the image of the first speaker and the image of the second speaker is greater than or equal to a predetermined value. If this is the case, discontinue the close-up of the speaker's image. [Effects of the Invention]

[0007] According to one embodiment of the present invention, in a conference system that zooms in on the video of a speaker at the local location, it is possible to suppress the misidentification and zooming in on a person at another location displayed on the screen, who is mistakenly identified as a speaker at the local location. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the system configuration of a communication system according to one embodiment. [Figure 2] This figure shows another example of the configuration of a conference system according to one embodiment. [Figure 3] Figure (1) illustrates the outline of the process according to one embodiment. [Figure 4] Figure (2) illustrates the outline of the process according to one embodiment. [Figure 5] This figure shows an example of the hardware configuration of an IWB according to one embodiment. [Figure 6] This figure shows an example of the hardware configuration of a web conferencing device according to one embodiment. [Figure 7] This figure shows an example of a computer hardware configuration according to one embodiment. [Figure 8] This figure shows an example of the functional configuration of a web conferencing device according to one embodiment. [Figure 9] This figure shows an example of the functional configuration of an IWB according to one embodiment. [Figure 10] This is a sequence diagram showing an example of processing by a web conferencing device according to the first embodiment. [Figure 11] This is a sequence diagram showing an example of IWB processing according to the first embodiment. [Figure 12] This is a sequence diagram showing an example of processing by a web conferencing device according to the second embodiment. [Figure 13] This is a sequence diagram showing an example of IWB processing according to the second embodiment. [Figure 14] This figure shows an example of the functional configuration of a web conferencing device according to the third embodiment. [Figure 15] This is a sequence diagram showing an example of processing by a web conferencing device according to the third embodiment. [Figure 16] This is a sequence diagram (1) showing an example of the connection information setting process according to the fourth embodiment. [Figure 17] This is a sequence diagram (2) showing an example of the connection information setting process according to the fourth embodiment. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described in detail below with reference to the drawings.

[0010] <System Configuration> FIG. 1 is a diagram illustrating an example of the system configuration of a communication system according to an embodiment. A communication system 1 is a system in which one or more users A, user B, ... at a base X use a conference system 100 to hold a web conference with a user C who uses another conference system 110 at another base Y. In the example of FIG. 1, the communication system 1 includes the conference system 100 provided at the base X, another conference system 110 provided at another base Y, and a conference server 10. Further, the conference system 100, the other conference system 110, and the conference server 10 are connected to a communication network 2 such as the Internet and a LAN (Local Area Network), for example.

[0011] For example, the users A, B, ... participate in a web conference provided by the conference server 10 using the conference system 100. Further, the user C participates in the same web conference provided by the conference server 10 using the other conference system 110. Accordingly, the conference system 100 and the other conference system 110 can transmit and receive conference video including conference audio and conference images (moving images or still images) to and from each other via the conference server 10 to hold a web conference.

[0012] Note that the web conference provided by the conference server 10 may be any web conference as long as it can transmit and receive conference video to and from each other. Further, the conference system 100 may hold a web conference directly with the other conference system 110 via the communication network 2 without going through the conference server 10.

[0013] In the example of FIG. 1, the conference system 100 includes a web conference device 101 and an IWB (Interactive White Board) 102 connected to the web conference device 101. The web conference device 101 is an example of a first device including a camera and a microphone. Further, the IWB 102 is an example of a second device including a display.

[0014] A web conference device (first apparatus) 101 includes, for example, a microphone (microphone array) configured by arranging a plurality of microphones, and has a function of detecting the direction of a speaker using the microphone array. The web conference device 101 also includes one or more cameras that capture images of a user A, a user B, ... who are present around the web conference device. For example, the web conference device 101 may include a panoramic camera capable of capturing a 360-degree panoramic image of the surroundings, and may capture an image of the entire conference room. Alternatively, the web conference device 101 may capture images of the user A, the user B, ... who are present around the web conference device by combining images captured by a plurality of cameras.

[0015] An IWB (second apparatus) 102 is a touch sensor-equipped display, which is also called an electronic whiteboard. The IWB 102 allows direct writing, for example, with a pen or a finger, on a screen displayed on the display, and can also save the content displayed on the display as data. The IWB 102 can also be used as a large-sized display similar to a projector.

[0016] Furthermore, the IWB 102 according to the present embodiment is connected to a communication network 2, and has a web conference function for participating in a web conference provided by a conference server 10. For example, the IWB 102 transmits conference video including images and audio acquired by the web conference device 101 to another conference system 110 via the conference server 10. The IWB 102 also displays conference video received from another conference system 110 via the conference server 10 on the display. The IWB 102 also includes a speaker, and can output conference audio included in the conference video. As another example, the web conference device 101 may include a speaker, the IWB 102 may output audio data included in the conference video to the web conference device 101, and the web conference device 101 may output the conference audio.

[0017] For example, the web conferencing device 101 and the IWB 102 are connected by a USB (Universal Serial Bus) cable 103. In this case, the web conferencing device 101 transmits conference images (videos) to the IWB 102 using, for example, the UVC (USB Video Class) protocol. The web conferencing device 101 also transmits conference audio to the IWB 102 using, for example, the UAC (USB Audio Class) protocol. Furthermore, the IWB 102 can connect to the web conferencing device 101 via Ethernet® over USB and transmit display images (videos or still images) displayed on the display to the web conferencing device 101 using the HTTP protocol.

[0018] Furthermore, if the web conferencing device 101 is equipped with a speaker, the IWB 102 may transmit the conference audio to the web conferencing device 101 via UAC and output it from the speaker provided by the web conferencing device 101. In addition, the web conferencing device 101 and the IWB 102 may be connected to each other via an interface other than USB.

[0019] Other conferencing systems 110 can have any configuration as long as they can participate in web conferences provided by the conferencing server 10 and send and receive conference video with conferencing system 100. For example, other conferencing systems 110 may be information processing devices such as PCs (Personal Computers), tablet terminals, or smartphones, or electronic devices equipped with web conferencing functions such as video conferencing or IWB. Alternatively, other conferencing systems 110 may have the same configuration as conferencing system 100.

[0020] Figure 2 shows another example of the configuration of a conference system according to one embodiment. In the example in Figure 2, an example of the configuration of the conference system 100 is shown when PC 201 connected to the communication network 2 has a web conferencing function. In this case, the web conferencing device 101 is connected to PC 201 by a USB cable 103 or the like, and transmits conference images and conference audio to PC 201. In addition, IWB 102 is connected to PC 201 by, for example, an HDMI® (High-Definition Multimedia Interface) cable 202 or the like, and displays the conference video output by PC 201.

[0021] In the example shown in Figure 2, the web conferencing device 101 and the IWB 102 are connected via LAN 203, and the IWB 102 can transmit the display image and other data displayed on its screen to the web conferencing device 101.

[0022] (Summary of the process) Figures 3 and 4 are diagrams illustrating the overview of the process. The web conferencing device 101 has the function of detecting the direction of the speaker based on the sound acquired by the microphone (microphone array), extracting the speaker's image from the image captured by the camera based on the speaker's direction, and outputting the extracted speaker's image in close-up. With this function, for example, when user A speaks during a web conference, user A's image 301 is displayed in close-up on the other conference system 110 at site Y. Subsequently, when user B speaks, user B's image 301 is displayed in close-up on the other conference system 110 at the other site Y.

[0023] However, with conventional technology, for example, an image 303 of user C at another location Y, displayed on a large display such as IWB102, may be mistakenly identified as a speaker at location X and displayed in close-up on another conference system 110 at location Y. This problem can occur, for example, as shown in Figure 3, when noise such as background noise occurs while user C at another location Y is displayed on IWB102, or when user C's voice is output from the IWB102's speaker.

[0024] Therefore, the web conferencing device 101 according to this embodiment has a function to stop zooming in on the speaker's image if the extracted speaker's image is an image displayed on the display.

[0025] For example, as shown in Figure 4, let's assume that the display image 401 shown on the IWB 102's display shows user C. Also, let's assume that the image of the speaker extracted by the web conferencing device 101 based on the speaker's direction is user A's image 301. In this case, since the extracted speaker image is not the image displayed on the screen, the web conferencing device 101 performs a close-up of user A's image 301. Similarly, if the image of the speaker extracted based on the speaker's direction is user B's image 302, the web conferencing device 101 performs a close-up of user B's image 302.

[0026] On the other hand, let's assume that the image of the speaker extracted based on the speaker's direction is user C's image 303. In this case, since the extracted speaker image is the image displayed on the screen, the web conferencing device 101 stops zooming in on user C's image 303.

[0027] For example, the web conferencing device 101 acquires the display image shown on the IWB 102 from the IWB 102. The web conferencing device 101 also calculates the degree of match (similarity) between the image of the first speaker extracted based on the speaker's direction and the image of the second speaker included in the display image. If the degree of match is greater than or equal to a predetermined value, the device determines that the image of the first speaker is the image displayed on the screen.

[0028] As another example, the web conferencing device 101 may pre-store images of users A, B, etc., from its own location (location X). The web conferencing device 101 may also calculate the degree of match (or similarity) between the extracted image of the first speaker and the image of the second speaker, which is an image of a speaker from its own location. If the degree of match is less than a predetermined value, it may determine that the image displayed on the screen is that of the first speaker.

[0029] Thus, the web conferencing device 101 stops zooming in on the speaker's image if the image of the speaker extracted based on the speaker's direction is the same as the image displayed on the screen. Therefore, according to this embodiment, in a conferencing system 100 that zooms in on the video of a speaker at its own location, it is possible to suppress the misidentification and zooming in on a person at another location displayed on the screen, who is mistakenly identified as the speaker at its own location.

[0030] <Hardware Configuration> Next, we will describe examples of the hardware configurations of each device according to this embodiment.

[0031] (IWB hardware configuration) Figure 5 shows an example of the hardware configuration of an IWB according to one embodiment. As shown in Figure 2, the IWB 102 includes, for example, a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, an SSD (Solid State Drive) 504, a network I / F (Interface) 505, and an external device connection I / F 506.

[0032] Of these components, the CPU 501 is an arithmetic unit that controls the overall operation of the IWB 102. The ROM 502 is a non-volatile memory that stores programs used to start the CPU 501, such as the IPL (Initial Program Loader). The RAM 503 is a volatile memory used, for example, as the work area of ​​the CPU 501. The SSD 504 is a large-capacity, non-volatile storage device that stores various data, such as programs for the IWB 102.

[0033] Network I / F 505 is a communication interface for connecting IWB 102 to communication network 2 and for communicating with other devices. External device connection I / F 506 is an interface for connecting external devices such as a web conferencing device 101 or a speaker 550. Note that IWB 102 may have a built-in speaker 550.

[0034] The IWB102 also includes a capture device 511, a GPU (Graphics Processing Unit) 512, a display controller 513, a contact sensor 514, a sensor controller 515, an electronic pen controller 516, a short-range communication circuit 519, and an antenna 519a for the short-range communication circuit 519, a power switch 517, and selector switches 518, etc.

[0035] Of these, the capture device 511 captures (acquires) the display image shown on the display 540 as a moving image or a still image. The GPU 512 is a semiconductor chip (processor) that specializes in graphics. The display controller 513 controls and manages the screen display in order to output the output image from the GPU 512 to the display 540, etc. The contact sensor 514 detects when the electronic pen 530 or the user's hand H, etc., comes into contact with the display 540. The sensor controller 515 controls the processing of the contact sensor 514.

[0036] The contact sensor 514 performs coordinate input and detection, for example, using an infrared blocking method. This method of coordinate input and detection involves two light-emitting / receiving devices installed at both ends of the upper side of the display 540 emitting multiple infrared rays parallel to the display 540, which are reflected by reflective members provided around the display 540, and the light receiving element receives the light that returns along the same optical path as the light emitted. The contact sensor 514 outputs the ID of the infrared rays emitted by the two light-emitting / receiving devices blocked by the object to the sensor controller 515, and the sensor controller 515 identifies the coordinate position, which is the contact position of the object. The electronic pen controller 516 communicates with the electronic pen 530 to determine whether the pen tip or the pen end has touched the display 540. The short-range communication circuit 519 is a communication circuit that performs short-range wireless communication with other devices via the antenna 519a.

[0037] The power switch 517 is a switch for turning the power of the IWB 102 on or off. The selection switches 518 are a group of switches for adjusting, for example, the brightness and color of the display 540.

[0038] Furthermore, the IWB102 includes a bus line 520. The bus line 520 includes an address bus, a data bus, and various control signals for electrically connecting the various components, such as the CPU 501 shown in Figure 5.

[0039] (Hardware configuration of web conferencing devices) Figure 6 shows an example of the hardware configuration of a web conferencing device according to one embodiment. The web conferencing device 101 includes, for example, a CPU 601, ROM 602, RAM 603, SSD 604, network I / F 605, external device connection I / F 606, operation unit 607, image processing unit 608, cameras 609a, 609b, ..., sound processing unit 610, microphone array 611, speaker 612, and bus 613.

[0040] The CPU 601 is a processing unit that controls various functions of the web conferencing device 101 by executing a predetermined program. The ROM 602 is a non-volatile memory that stores, for example, a program used to start the CPU 601. The RAM 603 is a volatile memory used, for example, as the work area of ​​the CPU 601. The SSD 604 is a non-volatile, high-capacity storage device that stores, for example, programs, data, or configuration information for the web conferencing device 101.

[0041] Network I / F 605 is a communication interface for connecting the web conferencing device 101 to, for example, LAN 203 or communication network 2. External device connection I / F 606 is an interface for connecting external devices such as IWB 102 or PC 201 to the web conferencing device 101. Operation unit 607 is an input device that accepts user input, such as operation buttons, switches, or a touch panel.

[0042] The image processing unit 608 is a device that acquires images of the area around the web conferencing device 101 from one or more cameras 609a, 609b, ... and performs image processing on the acquired images, such as image synthesis, image quality correction, or distortion correction. The cameras 609a, 609b, ... are imaging devices that capture images of the area around the web conferencing device 101.

[0043] The sound processing unit 610 is a device that performs various sound processing, such as beamforming to control directivity, using, for example, a microphone array 611. The sound processing unit 610 also performs sound processing to output various sounds, such as conference audio, using a speaker 612. The bus 613 is connected in common to all of the above components and transmits, for example, address signals, data signals, and various control signals.

[0044] (Computer hardware configuration) PC201 has a hardware configuration of computer 700, for example, as shown in Figure 7. The conference server 10 is also composed of, for example, one or more computers 700.

[0045] Figure 7 shows the hardware configuration of a computer according to one embodiment. The computer 700 includes, for example, a CPU 701, ROM 702, RAM 703, HD (Hard Disk) 704, HDD (Hard Disk Drive) controller 705, display 706, external device connection I / F 707, network I / F 708, keyboard 709, pointing device 710, DVD-RW (Digital Versatile Disk ReWritable) drive 712, media I / F 714, and bus line 715, etc.

[0046] Of these, the CPU 701 is the arithmetic unit that controls the overall operation of the computer 700. The ROM 702 is a non-volatile memory that stores programs used to drive the CPU 701, such as the IPL. The RAM 703 is a volatile memory used as the work area of ​​the CPU 701. The HD 704 is a large-capacity storage device that stores programs such as the OS (Operating System) and applications, as well as various types of data. The HDD controller 705 controls the reading or writing of various types of data to the HD 704 according to the control of the CPU 701.

[0047] The display 706 displays various information such as cursors, menus, windows, characters, or images. The external device connection interface 707 is an interface for connecting various external devices. The network interface 708 is a communication interface for data communication using a communication network. The keyboard 709 is a type of input means equipped with multiple keys for inputting characters, numbers, and various instructions. The pointing device 710 is a type of input means for selecting and executing various instructions, selecting processing targets, moving the cursor, etc.

[0048] The DVD-RW drive 712 controls the reading or writing of various data to the DVD-RW 711, which is an example of a removable recording medium. Note that the DVD-RW 711 is not limited to DVD-RW; it may be any other removable recording medium. The media interface 714 controls the reading or writing (storage) of data to the media 713, such as flash memory. The bus line 715 includes an address bus, a data bus, and various control signals for electrically connecting the various components such as the CPU 701 shown in Figure 7.

[0049] <Functional Configuration> Next, an example of the functional configuration of the web conferencing device 101 and IWB 102 according to this embodiment will be described.

[0050] (Functional configuration of web conferencing devices) Figure 8 shows an example of the functional configuration of a web conferencing device according to one embodiment. The web conferencing device (first device) 101 implements, for example, a communication unit 801, an audio acquisition unit 802, a direction detection unit 803, an image acquisition unit 804, an image extraction unit 805, a close-up unit 806, a comparison unit 807, an image code analysis unit 808, an audio code analysis unit 809, and a UI (User Interface) unit 810, etc., by the CPU 601 executing a predetermined program. At least a part of each of the above functional configurations may be implemented by hardware.

[0051] The communication unit 801 performs communication processing to communicate with an IWB 102 or PC 201 connected to the web conferencing device 101, for example, using an external device connection I / F 606. Preferably, the communication unit 801 can also perform communication processing to communicate with an IWB 102 or the like connected to the web conferencing device 101 using a network I / F 605 or the like.

[0052] The audio acquisition unit 802 performs audio acquisition processing to acquire sound from the vicinity of the web conferencing device 101, for example, using a microphone array 611 and a sound processing unit 610. The audio acquisition unit 802 also performs beamforming using the microphone array 611, sound quality adjustment of the acquired sound, or volume adjustment of the acquired sound. The microphone array 611 is an example of a microphone provided in the first device. The web conferencing device 101 is also an example of the first device.

[0053] The direction detection unit 803 performs direction detection processing to detect the direction of the speaker based on the sound acquired by the voice acquisition unit 802 using the microphone array (microphone) 611. For example, the direction detection unit 803 analyzes the voice data acquired by multiple microphones to estimate the direction of the sound source.

[0054] The image acquisition unit 804 performs image acquisition processing to acquire images (moving images or still images) of the area around the web conferencing device 101, for example, using one or more cameras 609a, 609b, ... and an image processing unit 608, etc. The image acquisition unit 804 also performs image quality correction or distortion correction of the acquired images, for example. Hereinafter, the explanation will be given assuming that the images acquired by the image acquisition unit 804 are moving images.

[0055] The image extraction unit 805 performs an image extraction process to extract the speaker's image (the first speaker's image) from the image acquired by the image acquisition unit 804, based on the speaker's direction detected by the direction detection unit 803. For example, the image extraction unit 805 may use a pre-trained machine learning model, which has been trained in advance to estimate the region in which a person is visible from the input image, to extract an image (moving image) of the region in which a person is visible. Alternatively, the image extraction unit 805 may use, for example, a known pattern recognition technique to extract the speaker's image from the image acquired by the image acquisition unit 804.

[0056] Here, machine learning is a technique for enabling computers to acquire human-like learning abilities. It refers to a technique in which a computer autonomously generates algorithms necessary for data identification and other judgments from pre-inputted training data, and then applies these algorithms to new data to make predictions. The learning method for machine learning can be supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, or deep learning, or a combination of these learning methods; the learning method for machine learning is not restricted.

[0057] The close-up unit 806 performs a close-up process to zoom in on the speaker image extracted by the image extraction unit 805 and output it. The close-up unit 806 also stops zooming in on the speaker image if the speaker image extracted by the image extraction unit 805 is the same image displayed on the display.

[0058] The comparison unit 807 performs a comparison process that compares the image of the first speaker with the image of the second speaker and outputs the comparison result. For example, the comparison unit 807 calculates the degree of agreement (or similarity) between the image of the first speaker extracted by the image extraction unit 805 and the image of the second speaker included in the display image acquired by the close-up unit 806, and outputs a comparison result indicating whether the degree of agreement is greater than or equal to a predetermined value. In this case, the comparison unit 807 may, for example, use the image extraction unit 805 to extract the image of the second speaker from the display image.

[0059] In another embodiment, the comparison unit 807 calculates the degree of agreement between the image of the first speaker extracted by the image extraction unit 805 and the image of the second speaker acquired by the close-up unit 806, and outputs a comparison result indicating whether the degree of agreement is greater than or equal to a predetermined value. In yet another embodiment, the comparison unit 807 calculates the degree of agreement between the image of the first speaker extracted by the image extraction unit 805 and the image of the second speaker, which is a pre-stored image of the speaker at the local location, and outputs a comparison result indicating whether the degree of agreement is less than a predetermined value.

[0060] The image code analysis unit 808 analyzes the image code captured by the camera in the image acquisition unit 804 and performs image code analysis processing to extract connection information represented by the image code. For example, the image code analysis unit 808 analyzes the image code displayed on the display by the IWB 102 and obtains connection information such as the IP address of the IWB 102 represented by the image code. The image code analysis unit 808 also sets the obtained connection information in the communication unit 801.

[0061] The voice code analysis unit 809 analyzes the voice code (or sound code) acquired by the voice acquisition unit 802 and performs voice code analysis processing to extract connection information represented by the voice code. For example, the voice code analysis unit 809 analyzes the voice code output by the IWB 102 and obtains connection information such as the IP address of the IWB 102 represented by the voice code. The voice code analysis unit 809 also sets the acquired connection information in the communication unit 801.

[0062] The UI unit 810 accepts various operations from the user to the web conferencing device 101, for example, using the operation unit 607. As another example, the UI unit 810 may also accept user operations on the UI screen displayed on the information terminal, such as a smartphone, via wireless communication from the information terminal.

[0063] (IWB Functional Configuration) Figure 9 shows an example of the functional configuration of an IWB according to one embodiment. The IWB (second device) 102, for example, the CPU 501, executes a predetermined program to realize the first communication unit 901, conference control unit 902, display control unit 903, audio output unit 904, UI unit 905, second communication unit 906, image transmission unit 907, person image detection unit 908, image code generation unit 909, and audio code generation unit 910, etc. At least some of the above functional configurations may be realized by hardware.

[0064] The first communication unit 901, for example, uses the network interface 505 to connect the IWB 102 to the communication network 2 and performs a first communication process to communicate with the conference server 10 or another conference system 110. In the conference system 100 shown in Figure 2, the first communication unit 901 can also use the network interface 505 to connect the IWB 102 to the LAN 203 and communicate with the web conferencing device 101.

[0065] The conference control unit 902 connects to the conference server 10, for example, using the first communication unit 901, and executes conference control processing to conduct a web conference with another conference system 110. During the web conference, the conference control unit 902 transmits, for example, conference audio and conference video (including video) transmitted from the web conferencing device 101 to the other conference system 110 via the conference server 10. The conference control unit 902 also receives conference video from the other conference system 110 via the conference server 10. The conference control unit 902 may utilize various existing web conferencing mechanisms.

[0066] The display control unit 903 performs display control processing to display conference images (moving images) included in the conference video received by the conference control unit 902 from another conference system 110 on the display 540 using the display controller 513 or the like. The display control unit 903 can also display image codes generated by the image code generation unit 909, or the UI screen of the UI unit 905, etc., on the display 540.

[0067] The audio output unit 904 performs audio output processing to output conference audio contained in conference video received by the conference control unit 902 from another conference system 110 using a speaker 550 or the like. The audio output unit 904 can also output audio codes generated by the audio code generation unit 910.

[0068] The UI unit 905 uses the display control unit 903 to display the UI screen on the display 417 or the like, and uses the sensor controller 515 and the contact sensor 514 or the like to receive user input on the UI screen.

[0069] The second communication unit 906 performs a second communication process, for example, by using an external device connection I / F 506 to communicate with a web conferencing device 101 or the like.

[0070] The image transmission unit 907 performs image transmission processing to send images requested by the web conferencing device 101 to the web conferencing device 101 via the second communication unit 906. For example, in response to a request from the web conferencing device 101, the image transmission unit 907 acquires a display image (snapshot) of the conference image displayed on the display 540 by the display control unit 903 and transmits it to the web conferencing device 101.

[0071] In another embodiment, the image transmission unit 907 transmits to the web conferencing device 101, in response to a request from the web conferencing device 101, a person image detection unit 908 has detected a person image (image of the second speaker) from the display image.

[0072] The person image detection unit 908 performs person image detection processing to detect a person image (an image of the second speaker) from the display image captured by the image transmission unit 907. For example, the person image detection unit 908 extracts an image of the second speaker from the display image using the same method as described above by the image extraction unit 805.

[0073] The image code generation unit 909 performs an image code generation process to generate an image code (such as a two-dimensional code or a barcode) that represents connection information for communicating with the IWB 102 via the network, such as the IP address of the first communication unit 901.

[0074] The voice code generation unit 910 performs a voice code generation process that generates a voice code (or sound code) that represents connection information for communicating with the IWB 102 via the network, such as the IP address of the first communication unit 901.

[0075] <Processing flow> Next, the processing flow of the display control method according to this embodiment will be described.

[0076] [First Embodiment] Figure 10 is a sequence diagram showing an example of processing by a web conferencing device according to the first embodiment. This processing shows an example of processing performed by a web conferencing device 101 in a conferencing system 100, such as the one shown in Figure 1. It is assumed that a web conference is taking place between the conferencing system 100 and another conferencing system 110 at the start of the processing shown in Figure 10.

[0077] In step S1001, the audio acquisition unit 802 outputs the microphone audio (conference audio) acquired by the microphone array 611 to the direction detection unit 803. Accordingly, in step S1002, the direction detection unit 803 detects the direction of the speaker from the microphone audio and notifies the close-up unit 806 of the detected direction of the speaker.

[0078] In step S1003, the image acquisition unit 804 outputs camera images (conference images) captured by one or more cameras 609a, 609b, ... to the close-up unit 806.

[0079] In step S1004, the close-up unit 806 inputs the camera image acquired from the image acquisition unit 804 and the speaker's direction notified by the direction detection unit 803 to the image extraction unit 805. Accordingly, in step S1005, the image extraction unit 805 extracts an image of the first speaker, which is the image of the speaker making the statement, from the camera image based on the input camera image and the speaker's direction, and outputs the extracted speaker image to the close-up unit 806.

[0080] In steps S1006 and S1007, the close-up unit 806 requests a display image from the IWB 102 via the communication unit 801. In response, in steps S1008 and S1009, the IWB 102 transmits a display image (snapshot) of the conference video displayed on the display 417 or the like to the web conferencing device 101.

[0081] In step S1010, the close-up unit 806 inputs the image of the first speaker obtained from the image extraction unit 805 and the display image obtained from the IWB 102 to the comparison unit 807. Accordingly, in step S1011, the comparison unit 807 compares the input image of the first speaker with the image of the second speaker included in the input display image and outputs the comparison result to the close-up unit 806. This comparison result includes, for example, information indicating whether the degree of agreement (or similarity) between the image of the first speaker and the image of the second speaker is greater than or equal to a predetermined value.

[0082] The web conferencing device 101 branches its processing based on the comparison result notified by the comparison unit 807. For example, if the degree of matching between the image of the first speaker and the image of the second speaker is less than a predetermined value, the web conferencing device 101 executes the processing in steps S1021 to S1023. In step S1021, the close-up unit 806 zooms in on the image of the first speaker to create a conference image (moving image). In steps S1022 and S1023, the close-up unit 806 transmits the conference video, including the created conference image and conference audio, to the IWB 102 via the communication unit 801.

[0083] On the other hand, if the degree of matching between the image of the first speaker and the image of the second speaker is greater than or equal to a predetermined value, the web conferencing device 101 executes the processes of steps S1031 and S1032. In steps S1031 and S1032, the close-up unit 806 stops the close-up of the first speaker and transmits a conference video, including a conference image taken of, for example, a conference room at its own location, and the conference audio, to the IWB 102 via the communication unit 801.

[0084] (IWB processing) Figure 11 is a sequence diagram showing an example of IWB processing according to the first embodiment. This processing shows an example of IWB 102 processing corresponding to the processing of web conferencing device 101 described in Figure 10. Note that steps S1007 and S1008 in Figure 11 correspond to steps S1007 and S1008 in Figure 10.

[0085] In step S1007, when the second communication unit 906 receives a request for a display image from the web conferencing device 101, in step S1101, the second communication unit 906 sends a request to the image transmission unit 907 to acquire the display image.

[0086] In step S1102, when the image transmission unit 907 receives a request to acquire a display image, it acquires a display image (snapshot) that captures the conference image displayed on the display 540 by the display control unit 903. Also in step S1103, the image transmission unit 907 transmits the acquired display image to the second communication unit 906.

[0087] In step S1008, the second communication unit 906 transmits the display image received from the image transmission unit 907 to the web conferencing device 101.

[0088] Furthermore, in step S1111, when the second communication unit 906 receives the conference video transmitted by the web conferencing device 101, it executes the process in step S1112.

[0089] In step S1112, the second communication unit 906 outputs the conference video received from the web conferencing device 101 to the conference control unit 902. In step S1113, the conference control unit 902 transmits the conference video to another conference system 110. For example, the conference control unit 902 transmits the conference video to the other conference system 110 by transmitting the conference video to the conference server 10 via the first communication unit 901.

[0090] The processing shown in Figures 10 and 11 prevents the conference system 100, which zooms in on the video of the speaker at its own location, from mistakenly recognizing and zooming in on a person at another location displayed on the screen as the speaker at its own location.

[0091] [Second Embodiment] In the second embodiment, another example of the processing of the conference system 100 will be described. Figure 12 is a sequence diagram showing an example of the processing of a web conferencing device according to the second embodiment. Of the processing shown in Figure 12, steps S1001 to S1006 are the same as the processing of the web conferencing device according to the first embodiment described in Figure 10, so their explanation is omitted here. Also, a detailed explanation of the processing content similar to that of the first embodiment is omitted here.

[0092] In steps S1201 and S1202, the close-up unit 806 requests the IWB 102, via the communication unit 801, to display the image of the second speaker included in the display image. At this point, the web conferencing device 101 branches its processing depending on whether or not it has received the image of the second speaker from the IWB 102 within a predetermined time.

[0093] For example, if the web conferencing device 101 receives an image of the second speaker from the IWB 102 within a predetermined time, it executes the processes in steps S1211 to S1214. In steps S1211 and S1212, the communication unit 801 outputs the image of the second speaker received from the IWB 102 to the close-up unit 806. In step S1213, the close-up unit 806 inputs the image of the first speaker obtained from the image extraction unit 805 and the image of the second speaker obtained from the IWB 102 to the comparison unit 807. Accordingly, in step S1214, the comparison unit 807 compares the input images of the first and second speakers and outputs the comparison result to the close-up unit 806.

[0094] On the other hand, if the IWB 102 cannot receive the image of the second speaker within a predetermined time, the web conferencing device 101 executes the process in step S1221. In step S1221, the close-up unit 806 determines that the image of the first speaker and the image of the second speaker do not match (the degree of match is less than a predetermined value).

[0095] In step S1231, the web conferencing device 101 performs a video transmission process based on the comparison result between the image of the first speaker and the image of the second speaker. For example, if the degree of matching between the image of the first speaker and the image of the second speaker is less than a predetermined value, the web conferencing device 101 performs the processes in steps S1021 to S1023 of Figure 10. On the other hand, if the degree of matching between the image of the first speaker and the image of the second speaker is greater than or equal to a predetermined value, the web conferencing device 101 performs the processes in steps S1031 and S1032 of Figure 10.

[0096] (IWB processing) Figure 13 is a sequence diagram showing an example of IWB processing according to the second embodiment. This processing shows an example of IWB 102 processing corresponding to the web conferencing device processing according to the second embodiment described in Figure 12. Note that the processing in step S1202 of Figure 13 is This corresponds to the process in step S1202 in Figure 12. Also, the process in step S1211 in Figure 13 corresponds to the process in step S1211 in Figure 12. Furthermore, the processes in steps S1111 to S1113 in Figure 13 are the same as the processes in steps S1111 to S1113 explained in Figure 11, so the explanation is omitted here.

[0097] In step S1202, when the second communication unit 906 receives a request for an image of the second speaker from the web conferencing device 101, in step S1301, the second communication unit 906 sends a request to the image transmission unit 907 to acquire an image of the second speaker.

[0098] In step S1302, when the image transmission unit 907 receives a request to acquire an image of the second speaker, it acquires a display image (snapshot) of the conference image displayed on the display 540 by the display control unit 903. Also in step S1303, the image transmission unit 907 inputs the acquired display image to the person image detection unit 908.

[0099] In step S1304, the person image detection unit 908 obtains an image of the region in which a person is visible (the image of the second speaker) from the input display image. For example, the person image detection unit 908 extracts an image of the region in which a person is visible (the image of the second speaker) from the display image using a pre-trained machine learning model that has been trained in advance to extract an image of the region in which a person is visible from the display image.

[0100] If the person image detection unit 908 is able to acquire an image of the second speaker from the display image, the IWB 102 executes the processes in steps S1311, S1312, and S1211. On the other hand, if the person image detection unit 908 is unable to acquire an image of the second speaker, the IWB 102 stops executing the processes in steps S1311, S1312, and S1211.

[0101] In step S1311, the person image detection unit 908 outputs the acquired image of the second speaker to the image transmission unit. Accordingly, in steps S1312 and S1211, the image transmission unit 907 transmits the image of the second speaker to the web conferencing device 101 via the second communication unit 906.

[0102] In the processing shown in Figures 12 and 13, the conference system 100, which zooms in on the video of the speaker at its own location, can suppress the misidentification and zoom-in of a person at another location displayed on the screen as a speaker at its own location. [Third Embodiment] In the third embodiment, an example of processing will be described in which the web conferencing device 101 has a function to register images of users (user A, user B, ...) who are participating in a web conference at its own location (location X) to the web conferencing device 101.

[0103] <Functional Configuration> Figure 14 shows an example of the functional configuration of a web conferencing device according to the third embodiment. As shown in Figure 14, the web conferencing device 101 according to the third embodiment has, in addition to the functional configuration of the web conferencing device 101 according to the first embodiment described in Figure 8, a user image registration unit 1401 and a user image storage unit 1402.

[0104] The user image registration unit 1401 is implemented, for example, by a program executed by the CPU 601, and performs user image registration processing to register images of users participating in web conferences at its own location. For example, when the UI unit 810 receives a user image registration operation, the user image registration unit 1401 stores the user image acquired by the image acquisition unit 804 in the user image storage unit 1402, etc.

[0105] Preferably, the user image registration unit 1401 stores images of all users A, B, ... participating in the web conference at its own location (for example, location X) in the user image storage unit 1402.

[0106] As another example, the user image registration unit 1401 may automatically acquire images of users A, B, ... who are around the web conferencing device 101 before the web conference starts, and store them in the user image storage unit 1402.

[0107] The user image storage unit 1402 is implemented, for example, by a program executed by the CPU 601 and the SSD 604, and stores the user's image acquired by the user image registration unit 1401.

[0108] <Processing flow> Figure 15 is a sequence diagram showing an example of processing for a web conferencing device according to the third embodiment. At the start of the processing shown in Figure 15, it is assumed that the images of users (e.g., user A, user B, ...) participating in the web conference from the local location (e.g., location X) are stored in the user image storage unit 1402. Furthermore, the processing in steps S1001 to S1005 of Figure 15 is the same as the processing for the web conferencing device according to the first embodiment described in Figure 10, so its explanation is omitted here.

[0109] In steps S1501 and S1502, the close-up unit 806 acquires an image of a second speaker, which is an image of a user at its own location that has been stored in advance. This second speaker image includes, for example, an image of user A, an image of user B, etc.

[0110] In step S1503, the close-up unit 806 inputs the image of the first speaker obtained from the image extraction unit 805 and the image of the second speaker obtained from the user image storage unit 1402 to the comparison unit 807.

[0111] In step S1504, the comparison unit 807 compares the input image of the first speaker with the image of the second speaker and outputs the comparison result to the close-up unit 806. For example, the comparison unit 807 calculates the degree of agreement (or similarity) between each of the images of the first speaker and the second speaker, and if there is an image with an agreement degree greater than or equal to a predetermined value among the calculated agreement degrees, it outputs a comparison result to the close-up unit 806 indicating that the agreement degree is greater than or equal to the predetermined value. On the other hand, if there is no image with an agreement degree greater than or equal to the predetermined value among the calculated agreement degrees, the comparison unit 807 outputs a comparison result to the close-up unit 806 indicating that the agreement degree is less than the predetermined value.

[0112] If the degree of matching is equal to or greater than a predetermined value, the web conferencing device 101 executes the processes in steps S1511 to S1513. In step S1511, the close-up unit 806 zooms in on the image of the first speaker to create a conference image (moving image). In steps S1512 and S1513, the close-up unit 806 transmits the conference video, including the created conference image and conference audio, to the IWB 102 or the like via the communication unit 801.

[0113] On the other hand, if the degree of match is less than a predetermined value, the web conferencing device 101 performs the processes in steps S1521 and S1522. In steps S1521 and S1522, the close-up unit 806 stops the close-up of the first speaker and transmits, for example, a conference image showing the conference room at the local site and a conference video including the conference audio to the IWB 102 or the like via the communication unit 801.

[0114] In the third embodiment, the image transmission unit 907 and the person image detection unit 908 of the IWB 102 are not used. Therefore, in the conference system 100 according to the third embodiment, conference video can be displayed using, for example, a large display, a projector, or an existing display device of an IWB that does not have the image transmission unit 907 and the person image detection unit 908, instead of the IWB 102.

[0115] Thus, according to the third embodiment, it is possible to use an existing display device to suppress the misidentification of a person at another location displayed on the screen as a speaker at the local location and zoom in on them.

[0116] [Fourth Embodiment] In the first to third embodiments, as shown in Figure 1, communication between the web conferencing device 101 and the IWB 102 was described as being performed using a USB cable 103 or the like. However, it is not limited to this, and as shown in Figure 2, communication between the web conferencing device 101 and the IWB 102 may be performed using a network such as a LAN 203.

[0117] In this case, the user can configure the web conferencing device 101 and the IWB 102 to communicate over the network by performing connection information configuration processing, for example, as shown in Figure 16 or Figure 17.

[0118] (Connection information setting process 1) Figure 16 is a sequence diagram (1) showing an example of the connection information setting process according to the fourth embodiment. This process shows an example of setting the IP address of the IWB102 on the web conferencing device 101 in a conference system 100 as shown in Figure 2. The IP address is an example of connection information for connecting to the IWB102.

[0119] In steps S1601 and S1602, when the UI unit 905 of the IWB102 receives an operation from the user to display an image code, it obtains an IP address from the first communication unit 901.

[0120] In step S1603, the UI unit 905 of the IWB102 inputs the acquired IP address to the image code generation unit 909. Accordingly, in step S1604, the image code generation unit 909 generates an image code representing the input IP address and outputs the generated image code to the UI unit 905.

[0121] In step S1605, the UI unit 905 of the IWB102 sends an image code display request to the display control unit 903, which requests the display of the generated image code. In response, in step S1606, the display control unit 903 displays an image code representing the IP address of the first communication unit 901 on the display 540 or the like.

[0122] From this state, in step S1611, the user performs an IP address setting operation on the UI unit 810 of the web conferencing device 101. In response, in step S1612, the UI unit 810 of the web conferencing device 101 instructs the image acquisition unit 804 to take a picture.

[0123] In steps S1613 and S1614, the image acquisition unit 804 captures the image code displayed by the IWB 102 and inputs the captured image code to the image code analysis unit 808. Since the image acquisition unit 804 can capture the area around the web conferencing device (the entire conference room) using one or more cameras 609a, 609b, ..., the user does not need to adjust the orientation of the web conferencing device 101.

[0124] In step S1615, the image code analysis unit 808 analyzes the input image code and obtains the IP address represented by the image code. Then, in step S1616, the image code analysis unit 808 sets the obtained IP address in the communication unit 801. As a result, the communication unit 801 can communicate with the IWB102, for example, using the network I / F 605.

[0125] (Connection information setting process 2) Figure 17 is a sequence diagram (2) showing an example of the connection information setting process according to the fourth embodiment. This process shows another example of the process of setting the IP address of the IWB 102 on the web conferencing device 101 in a conference system 100 as shown in Figure 2.

[0126] In steps S1701 and S1702, when the UI unit 905 of the IWB102 receives an operation from the user to display an image code, it obtains an IP address from the first communication unit 901.

[0127] In step S1703, the UI unit 905 of the IWB102 inputs the acquired IP address to the voice code generation unit 910. In response, in step S1704, the voice code generation unit 910 generates a voice code representing the input IP address and outputs the generated voice code to the UI unit 905.

[0128] In step S1705, the UI unit 905 of the IWB102 sends an audio code output request to the audio output unit 904, requesting the output of the generated audio code. In response, in step S1706, the audio output unit 904 outputs an audio code representing the IP address of the first communication unit 901 using a speaker 550 or the like.

[0129] From this state, in step S1711, the user performs an IP address setting operation on the UI unit 810 of the web conferencing device 101. In response, in step S1712, the UI unit 810 of the web conferencing device 101 instructs the audio acquisition unit 802 to acquire audio.

[0130] In steps S1713 and S1714, the audio acquisition unit 802 acquires the audio including the audio code output by the IWB102 and inputs the acquired audio (audio data) to the audio code analysis unit 809.

[0131] In step S1715, the voice code analysis unit 809 analyzes the input audio and obtains the IP address represented by the voice code. Then, in step S1716, the voice code analysis unit 809 sets the obtained IP address in the communication unit 801. This allows the communication unit 801 to communicate with the IWB102, for example, using the network I / F 605.

[0132] As described above, according to each embodiment of the present invention, in a conference system 100 that zooms in on the video of a speaker at its own location, it is possible to suppress the misidentification and zooming in on a person at another location displayed on the screen, who is mistakenly identified as a speaker at the own location.

[0133] <Supplement> Each function of the embodiments described above can be realized by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), and conventional circuit modules designed to execute each function described above.

[0134] <Note> This specification discloses the following conference systems, display control methods, and programs. (Section 1) A conferencing system comprising a first device equipped with a camera and a microphone, and a second device equipped with a display, The first apparatus described above, A direction detection unit detects the direction of the speaker based on the sound acquired by the aforementioned microphone, An image extraction unit extracts an image of the speaker from an image captured by the camera based on the speaker's direction, A close-up unit that outputs a close-up image of the aforementioned speaker, It has, The conference system includes a close-up section that, if the speaker's image is an image displayed on the display, stops the close-up of the speaker's image. (Section 2) The first device includes a comparison unit that compares the image of the first speaker extracted by the image extraction unit with the image of the second speaker included in the display image displayed on the display by the second device. The close-up unit stops zooming in on the first speaker's image if the degree of agreement between the first speaker's image and the second speaker's image is greater than or equal to a predetermined value. The conference system described in paragraph 1. (Section 3) The first device has a communication unit that communicates with the second device, The second device has an image transmission unit that transmits the display image to be displayed on the display to the first device. The conference system described in paragraph 1 or 2. (Section 4) The first device has a communication unit that communicates with the second device, The second device includes an image transmission unit that transmits an image of the second speaker, which is included in the display image displayed on the display, to the second device. The conference system described in paragraph 1 or 2. (Section 5) The conference system according to paragraph 4, wherein the image transmission unit stops the process of transmitting the image of the second speaker to the second device if no person is visible in the display image. (Section 6) The first device has a comparison unit that compares the image of the first speaker with the image of a second speaker, which is an image of a user at the local location that has been stored in advance. The close-up unit stops zooming in on the first speaker's image if the degree of similarity between the first speaker's image and the second speaker's image is less than a predetermined value. The conference system described in paragraph 1. (Section 7) The second device has a display control unit that displays an image code representing connection information for connecting to the second device on the display, The first device includes an image code analysis unit that analyzes the image code captured by the camera and sets the connection information represented by the image code in the communication unit. The conference system described in paragraph 3 or 4. (Section 8) The second device has an audio output unit that outputs an audio code representing connection information for connecting to the second device, The first device includes an audio code analysis unit that analyzes the audio code acquired by the microphone and sets the connection information represented by the audio code in the communication unit. A conference system as described in any of paragraphs 3, 4, and 7. (Section 9) In a conference system including a first device equipped with a camera and a microphone, and a second device equipped with a display, The first device, Based on the audio acquired by the aforementioned microphone, a detection process is performed to detect the direction of the speaker, Based on the direction of the speaker, an extraction process is performed to extract the speaker's image from the image captured by the camera, A close-up process that outputs a close-up of the speaker's image, Execute, The close-up process is a display control method that cancels the close-up of the speaker's image if the speaker's image is an image displayed on the display. (Section 10) In a conference system including a first device equipped with a camera and a microphone, and a second device equipped with a display, The first apparatus, Based on the audio acquired by the aforementioned microphone, a detection process is performed to detect the direction of the speaker, Based on the direction of the speaker, an extraction process is performed to extract the speaker's image from the image captured by the camera, A close-up process that zooms in on the image of the aforementioned speaker, Make it run, The close-up process is a program that, if the image of the speaker is an image displayed on the display, stops the close-up of the speaker's image.

[0135] Although embodiments of the present invention have been described above, the present invention is not limited to these specific embodiments, and various modifications and applications are possible within the scope of the gist of the present invention as described in the claims. [Explanation of Symbols]

[0136] 1. Communication System 100 Conference Systems 101 Web conferencing device (an example of the first device) 102 IWB (An example of a second device) 540 displays 609a, 609b cameras 611 Microphone Array (Microphone) 801 Communications Department 803 Direction detection unit 805 Image Extraction Unit 806 Close-up section 807 Comparison Section 808 Image Code Analysis Department 809 Voice Code Analysis Unit 903 Display Control Unit 904 Audio Output Section 907 Image transmission unit 1402 User Image Storage Unit [Prior art documents] [Patent Documents]

[0137] [Patent Document 1] Japanese Patent Publication No. 2017-34502

Claims

1. A conferencing system comprising a first device equipped with a camera and a microphone, and a second device equipped with a display, The first apparatus is A direction detection unit detects the direction of the speaker based on the sound acquired by the aforementioned microphone, An image extraction unit extracts an image of the speaker from an image captured by the camera based on the speaker's direction, A close-up unit that outputs a close-up image of the aforementioned speaker, A communication unit that communicates with the second device, the communication unit that receives from the second device a display image displayed on the display by the second device or an image of a second speaker included in the display image, A comparison unit compares the image of the first speaker extracted by the image extraction unit with the image of the second speaker included in the display image. It has, The close-up unit stops zooming in on the first speaker's image if the degree of agreement between the first speaker's image and the second speaker's image is greater than or equal to a predetermined value. Conference system.

2. The second device has an image transmission unit that transmits the display image displayed on the display to the first device, The conference system according to claim 1.

3. The second device has an image transmission unit that transmits an image of the second speaker included in the display image displayed on the display to the first device. The conference system according to claim 1.

4. The conference system according to claim 3, wherein the image transmission unit stops the process of transmitting the image of the second speaker to the first device if no person is visible in the display image.

5. The second device has a display control unit that displays an image code representing connection information for connecting to the second device on the display, The first device includes an image code analysis unit that analyzes the image code captured by the camera and sets the connection information represented by the image code in the communication unit. The conference system according to claim 2 or 3.

6. The second device has an audio output unit that outputs an audio code representing connection information for connecting to the second device, The first device includes an audio code analysis unit that analyzes the audio code acquired by the microphone and sets the connection information represented by the audio code in the communication unit. The conference system according to claim 2 or 3.

7. A conferencing system comprising a first device equipped with a camera and a microphone, and a second device equipped with a display, The first apparatus is A direction detection unit detects the direction of the speaker based on the sound acquired by the aforementioned microphone, An image extraction unit extracts an image of the speaker from an image captured by the camera based on the speaker's direction, A close-up unit that outputs a close-up image of the aforementioned speaker, A comparison unit compares the image of the first speaker extracted by the image extraction unit with the image of the second speaker, which is an image of a user at the local location that has been stored in advance. It has, The close-up unit stops zooming in on the first speaker's image if the degree of agreement between the first speaker's image and the second speaker's image is less than a predetermined value. Conference system.

8. In a conference system including a first device equipped with a camera and a microphone, and a second device equipped with a display, The first apparatus is, Based on the audio acquired by the aforementioned microphone, a detection process is performed to detect the direction of the speaker, Based on the direction of the speaker, an extraction process is performed to extract the speaker's image from the image captured by the camera, A close-up process that outputs a close-up of the speaker's image, The process of receiving from the second device a display image displayed on the display or an image of a second speaker included in the display image, A comparison process is performed to compare the image of the first speaker extracted in the extraction process with the image of the second speaker included in the display image. Execute, The close-up processing is a display control method that cancels the close-up of the first speaker's image if the degree of agreement between the image of the first speaker and the image of the second speaker is greater than or equal to a predetermined value.

9. In a conference system including a first device equipped with a camera and a microphone, and a second device equipped with a display, The first apparatus, Based on the audio acquired by the aforementioned microphone, a detection process is performed to detect the direction of the speaker, Based on the direction of the speaker, an extraction process is performed to extract the speaker's image from the image captured by the camera, A close-up process that zooms in on the image of the aforementioned speaker, The process of receiving from the second device a display image displayed on the display or an image of a second speaker included in the display image, A comparison process is performed to compare the image of the first speaker extracted in the extraction process with the image of the second speaker included in the display image. Make it run, The program for the close-up process cancels the close-up of the first speaker's image if the degree of similarity between the first speaker's image and the second speaker's image is greater than or equal to a predetermined value.

Images