Television conference apparatus, television conference method, and television conference program

Abstract

To provide a television conference apparatus, a television conference method, and a television conference program, in which an object can be displayed in an appropriate size.SOLUTION: A television conference apparatus 10 comprises: a display size acquisition unit 11; an image acquisition unit; an object detection unit 14; an object size acquisition unit 12; and a size determination unit 15. The display size acquisition unit 11 acquires display size information that indicates the size of a display area of an image. The image acquisition unit acquires image data that indicates an image. The object detection unit 14 detects an object from the image data acquired by the image acquisition unit. The object size acquisition unit 12 acquires object size information that indicates the size of an object corresponding to the display area. The size determination unit 15 determines the size in which the object is displayed, on the basis of the object size information acquired by the object size acquisition unit 12 and the object detected by the object detection unit 14.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a video conference device, a video conference method, and a video conference program for displaying an image. [Background technology]

[0002] In recent years, there are more and more occasions where multiple people hold conferences through displays at different locations without meeting face to face. Here, if a person at a remote location is displayed excessively large on the display of the user's location, the person at the user's location feels a sense of oppression. On the other hand, if a person at a remote location is displayed excessively small on the display of the user's location, the person at the user's location has difficulty visually recognizing the facial expression of the person at the remote location. For this reason, a technology has been developed that displays a person at a remote location at an appropriate size on the display of each location. For example, Patent Document 1 describes a video conference device used in a video conference system that transmits and receives video between the user's location and at least one remote location.

[0003] In the video conference device of Patent Document 1, a video receiving unit receives video from a remote location. In addition, zoom magnification setting information of the camera at the remote location is received by a zoom magnification setting receiving unit. Based on the zoom magnification setting information and screen size information of the display at the local location, a video magnification / reduction ratio derivation unit derives a magnification / reduction ratio for displaying each subject in the video captured by each camera at the remote location in life-size on the display at the local location. The received video is enlarged or reduced by the video magnification / reduction unit based on the derived magnification / reduction ratio. The enlarged or reduced video is displayed on the display at the local location. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2011-23886 A Summary of the Invention [Problem to be solved by the invention]

[0005] Patent Document 1 describes that a subject captured by each camera at a remote site can be displayed in life-size on each display at the local site, regardless of the screen size of the display at each site. However, subjects such as people are not always stationary. Therefore, the subject may move closer to or farther away from the camera. In this case, the validity of the zoom magnification of the camera set according to the subject is lost, and the subject cannot be displayed at an appropriate size.

[0006] An object of the present invention is to provide a video conference device, a video conference method, and a video conference program capable of displaying an object at an appropriate size. [Means for solving the problem]

[0007] A video conferencing device according to one aspect of the present invention includes a display size acquisition unit that acquires display size information indicating the size of a display area of ​​an image, an image acquisition unit that acquires image data indicating an image, an object detection unit that detects an object from the image data acquired by the image acquisition unit, an object size acquisition unit that acquires object size information indicating the size of the object corresponding to the display area, and a size determination unit that determines the size at which an object is to be displayed based on the object size information acquired by the object size acquisition unit and the object detected by the object detection unit.

[0008] A video conferencing method according to another aspect of the present invention includes obtaining display size information indicating a size of a display area of ​​an image, obtaining image data indicating the image, detecting an object from the obtained image data, obtaining object size information indicating a size of the object corresponding to the display area, and determining a size at which to display the object based on the obtained object size information and the detected object, and is executed by a computer.

[0009] A video conferencing program according to yet another aspect of the present invention causes a computer to execute the following processes: acquiring display size information indicating the size of a display area for an image; acquiring image data indicating the image; detecting an object from the acquired image data; acquiring object size information indicating the size of the object corresponding to the display area; and determining a size at which to display the object based on the acquired object size information and the detected object. Effect of the Invention

[0010] According to the present invention, an object can be displayed at an appropriate size. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of a video conference system including a video conference device according to an embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram for explaining the arrangement of a video conference system. [Diagram 3] FIG. 1 is a block diagram for explaining a configuration of a video conference device. [Figure 4] 11 is a table showing examples of multiple types of specified values ​​stored in a storage device. [Diagram 5] 4 is a flowchart showing an example of a video conference process performed by the video conference device of FIG. 3. [Figure 6] 4 is a flowchart showing an example of a video conference process performed by the video conference device of FIG. 3. [Figure 7] FIG. 11 is a conceptual diagram for explaining a video conference process. [Figure 8] FIG. 11 is a conceptual diagram for explaining a video conference process. [Figure 9] FIG. 11 is a conceptual diagram for explaining a video conference process. [Figure 10] FIG. 11 is a conceptual diagram for explaining a video conference process. [Figure 11] FIG. 11 is a conceptual diagram for explaining a video conference process. [Figure 12] FIG. 11 is a conceptual diagram for explaining a video conference process. [Figure 13] FIG. 11 is a conceptual diagram for explaining a video conference process. [Figure 14] FIG. 11 is a conceptual diagram for explaining a video conference process. [Figure 15] FIG. 11 is a conceptual diagram for explaining a video conference process. [Figure 16] FIG. 13 is a block diagram for explaining the configuration of a video conference system according to a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] 1. Video conference system configuration A video conference device, a video conference method, and a video conference program according to an embodiment of the present invention will be described in detail below with reference to the drawings. Fig. 1 is a block diagram showing the configuration of a video conference system including a video conference device according to an embodiment of the present invention. As shown in Fig. 1, the video conference system 100 includes a storage device 101, a CPU (Central Processing Unit) 102, a RAM (Random Access Memory) 103, a ROM (Read Only Memory) 104, a communication I / F (Interface) 105, a display device 106, an operation unit 107, an imaging device 108, a microphone 109, a speaker 110, a distance measuring sensor 111, and a bus 112.

[0013] The storage device 101, CPU 102, RAM 103, ROM 104, communication I / F 105, display device 106, operation unit 107, imaging device 108, microphone 109, and speaker 110 are connected to a bus 112. The storage device 101, CPU 102, RAM 103, ROM 104, communication I / F 105, and bus 112 configure a video conference device 10. The video conference device 10 may be realized by an information processing device such as a personal computer (PC).

[0014] The storage device 101 includes a storage medium such as a hard disk, an optical disk, a magnetic disk, or a memory card, and stores a video conference program in advance. The CPU 102 executes the video conference program stored in the storage device 101 to perform video conference processing. The video conference processing will be described in detail later. The RAM 103 is, for example, a volatile memory, and is used as a working area for the CPU 102 and temporarily stores various data.

[0015] ROM 104 is, for example, a non-volatile memory, and stores computer programs such as a system program. Instead of storage device 101, ROM 104 may store a video conference program. The video conference program may be provided in a form stored in a computer-readable storage medium 113 such as a CD (compact disc)-ROM, and may be installed in storage device 101 or ROM 104 by being read through communication I / F 105 or the like. In addition, when communication I / F 105 is connected to a communication network, a video conference program distributed from a server connected to the communication network may be installed in storage device 101 or ROM 104.

[0016] The display device 106 includes, for example, a liquid crystal display. The display device 106 is capable of displaying an image of a desired object, such as the face of a call partner in a video conference or a prototype. The operation unit 107 includes a keyboard or a pointing device. The pointing device includes a mouse, a joystick, or the like. The operation unit 107 is operated when a user makes a predetermined selection or gives an instruction to the video conference device 10.

[0017] The imaging device 108 captures an image of an object in the video conference system 100, thereby generating image data showing an image of the object. The object in the video conference system 100 may be a user of the video conference system 100, or may be an object such as a prototype that needs to be presented to a call partner. The generated image data is transmitted to a call partner in a video conference through the communication I / F 105. This makes it possible to present an image of the object in the video conference system 100 to the call partner. It is also possible to display an image of the object in the video conference system 100 on the display device 106 based on the generated image data.

[0018] The microphone 109 collects ambient sounds and generates sound data indicative of the sounds. The generated sound data is transmitted to the other party in the video conference through the communication I / F 105. This allows the other party to hear the sounds in the video conference system 100. The sounds in the video conference system 100 include the voice of the user. The speaker 110 reproduces the sounds output by the video conference device 10. The sounds output by the video conference device 10 include the voice of the other party. The distance sensor 111 measures the distance between itself and a specified object in front.

[0019] Fig. 2 is a diagram for explaining the arrangement of the video conference system 100. As shown in Fig. 2, the video conference system 100 is provided, for example, at a location P1 where a user resides. In the example of Fig. 2, the video conference device 10 is realized by a mobile PC. The display device 106 is connected to the video conference device 10 by an HDMI (registered trademark) cable. The operation unit 107 is connected to the video conference device 10 by a USB (Universal Serial Bus) cable.

[0020] The imaging device 108, the microphone 109, and the speaker 110 are built into the video conference device 10, which is a mobile PC. However, the embodiment is not limited to this. Some or all of the imaging device 108, the microphone 109, and the speaker 110 may be provided outside the video conference device 10 and connected to the video conference device 10 by a predetermined cable. Furthermore, the display device 106 may be a monitor provided in the video conference device 10.

[0021] The distance measurement sensor 111 is attached near the screen of the display device 106. In this case, the distance measurement sensor 111 measures the distance between the screen of the display device 106 and a predetermined object in front. In this example, the object is the user of the video conference system 100. The distance measurement sensor 111 may recognize the face of the user and measure the distance to the recognized face of the user as the distance between the screen of the display device 106 and the user. Alternatively, in the case where a predetermined marker is attached to the user, the distance measurement sensor 111 may recognize the marker and measure the distance to the recognized marker as the distance between the screen of the display device 106 and the user.

[0022] At a site P2 different from the site P1, an information processing device 200, a display device 210, an imaging device 220, a microphone 230, a speaker 240, and the like are provided. The site P2 is, for example, the residence of the other party. The information processing device 200 is, for example, a PC, and is connected to the video conference device 10 at the site P1 through a communication network. In addition, the display device 210, the imaging device 220, the microphone 230, and the speaker 240 are connected to the information processing device 200 by an HDMI cable, a USB cable, or the like.

[0023] The information processing device 200 acquires image data generated by the imaging device 108 from the video conference device 10 in real time, and outputs the image data to the display device 210. As a result, a moving image of an object in the video conference system 100 (the face of the user of the video conference system 100 in the example of FIG. 2) is displayed on the display device 210.

[0024] The imaging device 220 captures an image of an object at the site P2 to generate image data showing an image of the object. The object at the site P2 may be a call partner, or may be an object such as a prototype that needs to be presented to a user. The information processing device 200 transmits the image data generated by the imaging device 220 to the video conference device 10 in real time. As a result, a moving image of the object at the site P2 (the face of the call partner in the example of FIG. 2) is displayed on the display device 106. It is also possible to display an image of the object at the site P2 on the display device 210 based on the image data generated by the imaging device 220.

[0025] The microphone 230 collects surrounding sounds and generates sound data indicating the sounds. The information processing device 200 transmits the sound data generated by the microphone 230 to the video conference device 10. As a result, sounds at the location P2 (including the voice of the other party) are reproduced from the speaker 110. The information processing device 200 also acquires surrounding sounds collected by the microphone 109 from the video conference device 10 and outputs them to the speaker 240. As a result, sounds at the location P1 (including the voice of the user of the video conference system 100) are reproduced from the speaker 240.

[0026] 2. Configuration of video conferencing equipment Fig. 3 is a block diagram for explaining the configuration of the video conference device 10. As shown in Fig. 3, the video conference device 10 includes, as functional units, a display size acquisition unit 11, an object size acquisition unit 12, a communication unit 13, an object detection unit 14, a size determination unit 15, an output unit 16, a size switching unit 17, a distance acquisition unit 18, a frame switching unit 19, and an image processing unit 20. The functional units of the video conference device 10 are realized by the CPU 102 in Fig. 1 executing a video conference program stored in the storage device 101 or the like. Some or all of the functional units of the video conference device 10 may be realized by hardware such as an electronic circuit.

[0027] The display size acquisition unit 11 acquires screen size information indicating the size of the screen from the display device 106. The display size acquisition unit 11 also acquires resolution information indicating the image resolution from the display device 106. The screen size information and resolution information are acquired by receiving a signal in an EDID (Extended Display Identification Data) format from the display device 106. In this example, the screen size is a set of the vertical length and horizontal length of the screen, but the embodiment is not limited to this. The screen size may be other sizes such as the diagonal length of the screen or the area of ​​the screen.

[0028] Here, a remote conference device that performs signal protocol conversion may be provided between the display device 106 and the video conference device 10. For example, the display device 106 and the remote conference device may be connected by an HDMI cable, and the remote conference device and the video conference device 10 may be connected by a USB cable. In this case, the display size acquisition unit 11 acquires screen size information from the display device 106 via the remote conference device.

[0029] Furthermore, display size acquisition unit 11 acquires display size information indicating the size of the display area of ​​the image. A user can operate operation unit 107 to instruct display size acquisition unit 11 to use a part or the entire screen of display device 106 as the display area of ​​the image. When the entire screen of display device 106 is instructed as the display area of ​​the image, the size of the display area of ​​the image matches the size of the screen indicated by the screen size information.

[0030] The object size acquisition unit 12 acquires object size information indicating the size of an object corresponding to a display area. The object size information may indicate the size of an object corresponding to the size of the screen of the display device 106. In this example, the object is a human face. Therefore, a typical specified value indicating the size of a human face is stored in advance in the storage device 101 in FIG. 1. The object size acquisition unit 12 acquires the specified value from the storage device 101, and calculates the object size information based on the acquired specified value and the display size information acquired by the display size acquisition unit 11.

[0031] The communication unit 13 acquires image data generated by the imaging device 220 and sound data generated by the microphone 230 in Fig. 2 from the information processing device 200. The communication unit 13 also acquires image data generated by the imaging device 108 and sound data generated by the microphone 109, and transmits them to the information processing device 200 in Fig. 2. This makes it possible to display an image based on the image data generated by the imaging device 108 on the display device 210. Also, it is possible to play back sound based on the sound data generated by the microphone 109 from the speaker 240. In this way, the communication unit 13 functions as an image acquisition unit that acquires image data.

[0032] The object detection unit 14 detects an object from the image data acquired by the communication unit 13. As described above, in this example, the object is a person's face, more specifically, the face of the call partner at the location P2. Therefore, the target of object detection is image data generated by the imaging device 220 at the location P2. Here, the image may include multiple call partners. In this case, the face of any one of the call partners may be detected as the object, or the face of the call partner who is speaking may be detected as the object. The object detection unit 14 is also capable of calculating the size of the detected object.

[0033] Here, the result of object detection by object detection unit 14 may be used when object size acquisition unit 12 acquires a default value corresponding to the object from storage device 101. In this case, even if multiple types of default values ​​corresponding to multiple types of objects are stored in storage device 101, object size acquisition unit 12 can appropriately acquire the default value corresponding to the object.

[0034] FIG. 4 is a table showing an example of multiple types of specified values ​​stored in the storage device 101. As shown in FIG. 4, multiple types of objects and multiple types of specified values ​​are stored in the storage device 101 in a corresponding state. In the example of FIG. 4, the multiple types of objects are an adult male face, an adult female face, a Shiba Inu, an office desk, and a chair. In this example, the object size acquisition unit 12 selects a default value corresponding to an object detected by the object detection unit 14 from the multiple types of specified values ​​stored in the storage device 101. Note that the specified values ​​described in FIG. 4 are merely examples, and the specified values ​​are not limited to the example of FIG. 4.

[0035] The size determination unit 15 determines the size at which the object is to be displayed based on the object size information acquired by the object size acquisition unit 12 and the object detected by the object detection unit 14. The size determined at this time is called the standard size. The size determination unit 15 may calculate the proportion of the object detected by the object detection unit 14 in the image data acquired by the communication unit 13, and determine the standard size at which the object is to be displayed based on the calculation result.

[0036] For example, the display area of ​​an image indicated by the display size information acquired by display size acquisition unit 11 is 32 cm high x 57 cm wide. Object detection unit 14 notifies object size acquisition unit 12 of the type of object detected from the image acquired by communication unit 13. Object size acquisition unit 12 searches the table of Fig. 4 according to the acquired type of object, and obtains size information of the object (in this example, the face of an adult male) 23 cm high x 15 cm wide. Object detection unit 14 notifies size determination unit 15 of the ratio of the object detected from the image data in the received image data.

[0037] The size determination unit 15 calculates the size of the object when the received image is displayed without control (without changing the magnification) from the object ratio in the received image and the display area of ​​the image. For example, the size is 11.5 cm vertical x 7.5 cm horizontal. In this case, in order to display the object at the size indicated by the object size information, the image needs to be enlarged to four times its original size (twice vertical x twice horizontal). Therefore, the size determination unit 15 determines the four-times display magnification as the display size and notifies the output unit 16.

[0038] The output unit 16 outputs various sound data to the speaker 110 so that the sound can be reproduced. As a result, various sounds are reproduced from the speaker 110. The sound data includes sound data generated by the microphone 230 and acquired by the communication unit 13. Therefore, the sound reproduced from the speaker 110 includes the sound collected by the microphone 230 at the location P2.

[0039] Furthermore, output unit 16 outputs image data indicating an image of the object to display device 106 so that the object can be displayed at the size determined by size determination unit 15. As a result, the image of the object is displayed in the display area of ​​display device 106 based on the output image data.

[0040] In the above processing, the size of the image is determined by size determination unit 15 according to the display area, so an image showing an object of a fixed size can be displayed on display device 106 regardless of the size of the display area. Furthermore, since the processing is performed in real time, the size of the object in the image does not change even if the distance between the object and imaging device 220 changes. Therefore, the user can visually recognize an object of a fixed size on display device 106.

[0041] On the other hand, the preferred display size of an object may differ from user to user. Therefore, the size switching unit 17 can switch the standard size determined by the size determination unit 15 to another fixed size based on the user's selection. The user can select the display size of an object by operating the operation unit 107.

[0042] For example, the display size of an object may be selectable from three levels: "large", "standard", and "small". When "standard" is selected, the image of the object is displayed in the standard size. When "large" is selected, the image of the object is displayed in a size that is, for example, 10% larger than the standard size. When "small" is selected, the image of the object is displayed in a size that is, for example, 10% smaller than the standard size. Size switching unit 17 may switch the size in two levels instead of three levels, or in four or more levels.

[0043] Furthermore, the preferred display size of the object varies depending on the distance between the display surface of the image of the object and the user. Therefore, the distance acquisition unit 18 acquires the distance between the display surface of the image by the output unit 16 and the user from the distance measurement sensor 111. In this example, the display surface of the image is the screen of the display device 106. The size determination unit 15 determines the size at which the object is displayed based on the distance acquired by the distance acquisition unit 18. That is, the size at which the object is displayed is changed from the standard size to another size.

[0044] The standard size is a size at which an object is displayed when the distance between the display surface of the image and the user is a predetermined standard distance. When the distance between the display surface of the image and the user is greater than the standard distance, the size at which the object is displayed is determined to be larger than the standard size. Conversely, when the distance between the display surface of the image and the user is smaller than the standard distance, the size at which the object is displayed is determined to be smaller than the standard size.

[0045] The size at which the object is displayed may be determined by multiplying the standard size by a coefficient proportional to the distance. In this case, when the distance between the display surface of the image and the user is the standard distance, the coefficient is 1. When the distance is smaller than the standard distance, i.e., when the user is located close to the display surface of the image, the coefficient is smaller than 1. On the other hand, when the distance is larger than the standard distance, i.e., when the user is located far from the display surface of the image, the coefficient is larger than 1.

[0046] By performing the above processing in real time, the image of the object is prevented from appearing small even if the user moves away from the screen of display device 106. Conversely, the image of the object is prevented from appearing large even if the user moves closer to the screen of display device 106. Therefore, the user can visually recognize the image of the object with a constant sense of size, regardless of the distance from the display surface of the image.

[0047] When an image displayed on display device 106 includes a plurality of objects, frame switching unit 19 switches the image of the displayed object based on a selection by the user. In this example, the frame can be switched between a first frame in which images of all call partners are displayed, a second frame in which only the call partner currently speaking is displayed, and a third frame in which only the call partner currently speaking and the call partner next to him are displayed. The user can select one of the first to third frames by operating operation unit 107. Each time operation unit 107 is operated, the displayed image may be switched sequentially between the first to third frames.

[0048] The image processing unit 20 executes various processes on the image data output by the output unit 16. For example, the processes on the image data include background image processing or skin beautification. In particular, when the display size of the object is small, the background area of ​​the image becomes large. Therefore, it is useful to perform image processing on the background. Also, when the object is a person's face, when the display size of the object is large, the image becomes coarse. Therefore, it is useful to perform skin beautification. The user can individually instruct whether or not to execute each image processing by operating the operation unit 107.

[0049] 3. Video conference processing 5 and 6 are flow charts showing an example of a video conference process by the video conference device 10 of FIG. 3. FIGS. 7 to 15 are conceptual diagrams for explaining the video conference process. The video conference process of FIGS. 5 and 6 will be explained below using the video conference device 10 of FIG. 3 and the conceptual diagrams of FIGS. 7 to 15. Note that the numerical values ​​shown in FIGS. 7 to 15 are concrete examples for easy understanding, and these numerical values ​​are not limited to the examples of FIGS. 7 to 15. In addition, in the video conference process, audio data is transmitted and received between the video conference device 10 and the information processing device 200, and image data is transmitted from the video conference device 10 to the information processing device 200, but explanations of these are omitted.

[0050] First, as shown in FIG. 7, the display size acquisition unit 11 acquires screen size information from the display device 106 (step S1). In the example of FIG. 7, the vertical screen length of the display device 106 is 20 cm, and the horizontal screen length is 40 cm. Next, the communication unit 13 acquires image data by receiving the image data from the information processing device 200 of FIG. 2 (step S2). In this example, as shown in FIG. 8, the image data acquired in step S2 is image data generated by the imaging device 220 of FIG. 2, and shows an image of the other party. Next, the object detection unit 14 detects an object from the image data acquired in step S2 (step S3), as shown in FIG. 9.

[0051] Thereafter, as shown in Fig. 10, object size acquisition unit 12 acquires the specified value of the object stored in storage device 101 in Fig. 1, and acquires object size information based on the specified value and the screen size information acquired in step S1 (step S4). When acquiring the specified value of the object, the object detection result in step S3 is used. The object size information in this example indicates the actual display size of the object on display device 106, which corresponds to the screen size of display device 106. In the example of Fig. 10, the length of the object in the vertical direction is 6 cm, and the length of the object in the horizontal direction is 10 cm.

[0052] As shown in FIG. 11, the display size acquisition unit 11 acquires resolution information from the display device 106 (step S5). The resolution information is acquired by receiving a signal in EDID format from the display device 106, similar to the screen size information. The resolution information may be set in a PC constituting the video conference device 10. In this case, the display size acquisition unit 11 may acquire the resolution information from the PC, not from the display device 106. Step S5 may be executed before step S1, may be executed simultaneously with step S1, or may be executed between steps S1 to S4. In the example of FIG. 11, the vertical image resolution of the display device 106 is 2160 pixels, and the horizontal image resolution is 3840 pixels.

[0053] Next, as shown in Fig. 12, the object size acquisition unit 12 converts the size indicated by the object size information acquired in step S4 into the resolution of the object based on the resolution information acquired in step S5 (step S6). In the example of Fig. 12, the vertical resolution of the object is 648 pixels, and the horizontal resolution of the object is 960 pixels.

[0054] As shown in Fig. 13, the display size acquisition unit 11 acquires display size information based on a user's instruction (step S7). The display size information may be specified in units of length or in units of pixels. In this example, the display size information is specified in units of pixels. Here, when full-screen display is specified, the display size information matches the resolution information acquired in step S5. In the example of Fig. 13, the display area of ​​the image in the vertical direction is 1080 pixels, and the display area of ​​the image in the horizontal direction is 1200 pixels.

[0055] Next, as shown in Fig. 14, the object detection unit 14 calculates the size of the object detected in step S3, which corresponds to the display size acquired in step S7 (step S8). The size of the object may be calculated in units of length or in units of pixels. In this example, the size of the object is calculated in units of pixels. In the example of Fig. 14, the size of the object in the vertical direction is 300 pixels, and the size of the object in the horizontal direction is 300 pixels.

[0056] Thereafter, as shown in Fig. 15, the size determination unit 15 determines the display magnification of the image based on the resolution of the object converted in step S6 and the size of the object calculated in step S8 (step S9). In the example of Fig. 15, the horizontal display magnification is 3.20, and the vertical display magnification is 2.16. In this manner, the horizontal display magnification and the vertical display magnification may be different. Alternatively, when the horizontal display magnification and the vertical display magnification are different, the average value of the horizontal display magnification and the vertical display magnification may be determined as the display magnification.

[0057] Step S9 is a process for determining a display magnification for displaying an image at a standard size. Therefore, step S9 corresponds to determining a standard size for displaying an object based on the object size information acquired by the object size acquisition unit 12 and the object detected by the object detection unit 14.

[0058] The detection of an object in step S3 includes not only detecting an image of an object contained in the image, but also detecting the size (e.g., the number of pixels) of the image of the object in the entire image or the ratio of the image of the object in the entire image. Even if the size or ratio of the image of the object is detected, the display magnification of the image can be determined in step S9 using screen size information, resolution information, display size information, object size information, etc.

[0059] Here, the size determination unit 15 determines whether or not to change the display magnification (step S10). If a display size other than "standard" is selected by the user, or if the distance acquired by the distance acquisition unit 18 is not the standard distance, it is determined that the display magnification is to be changed. If it is determined that the display magnification is not to be changed, the process proceeds to step S12. On the other hand, if it is determined that the display magnification is to be changed, the size determination unit 15 determines the display magnification after the change based on the selected display size or the distance acquired by the distance acquisition unit 18 (step S11). Thereafter, the process proceeds to step S12.

[0060] In step S12, the image processing unit 20 determines whether or not an instruction for any type of image processing is ON (step S12). If it is determined that the instructions for all types of image processing are OFF, the process proceeds to step S14. On the other hand, if it is determined that an instruction for any type of image processing is ON, the image processing unit 20 executes the corresponding type of processing on the image data acquired in step S6 (step S13). Thereafter, the process proceeds to step S14.

[0061] In step S14, the output unit 16 causes the display device 106 to display an image at the display magnification determined in step S9 or step S11 based on the image data acquired in step S2 or the image data processed in step S13 (step S14). The image displayed in step S14 includes the object detected in step S3. When there are multiple detected objects, the displayed image is appropriately switched between the first to third frames according to the switching operation by the frame switching unit 19.

[0062] Thereafter, the process returns to step S2, and steps S2 to S14 are repeated. Therefore, even if the specified display area is changed, an image showing an object of a fixed size is displayed in real time as a moving image on display device 106, regardless of the size of the display area. Also, even if the selected display size is changed, if the user's position is changed, if the instruction as to whether or not to execute image processing is changed, or if the selected frame is changed, the displayed image can be updated in real time.

[0063] 4. Variations In the present embodiment, the image projection area is configured by the screen of one display device 106, but the embodiment is not limited to this. The image projection area may be configured by the screens of multiple display devices 106 arranged vertically and horizontally. In this case, the image projection area is enlarged. In addition, since the size of the object displayed in the image projection area is kept constant, discontinuity of the displayed image is prevented between the boundaries of the screens of adjacent display devices 106.

[0064] Alternatively, the projection area of ​​the image may be configured on a screen such as a wall. Fig. 16 is a block diagram for explaining the configuration of a video conference system 100 in a modified example. Hereinafter, the video conference system 100 in the modified example will be explained with respect to the differences from the video conference system 100 in Fig. 3.

[0065] As shown in FIG. 16, in this example, the video conference system 100 includes a projection device 114 instead of the display device 106. The projection device 114 includes, for example, a projector. The projection device 114 projects an image of an object onto a screen based on image data output by the output unit 16. Therefore, the output unit 16 displays the image of the object on the screen at the site P1 based on the output image data. In this example, the projection area of ​​the image on the screen becomes the display area. Therefore, the display area of ​​the image can be easily enlarged. The display size acquisition unit 11 acquires display size information from the projection device 114.

[0066] In each of the above-mentioned modified examples, the image display area is enlarged, so that it is possible to display a life-size image of the call partner as an object. Also, images of objects such as desks or tables around the call partner can be displayed in life-size. Therefore, the user can hold a conference with the call partner in the same space as if they were face-to-face.

[0067] 5. Effects of the embodiment As described above, the video conferencing device 10 of this embodiment includes a display size acquisition unit 11 that acquires display size information indicating the size of the display area of ​​an image, an object size acquisition unit 12 that acquires object size information indicating the size of an object corresponding to the display area, a communication unit 13 that acquires image data indicating an image, an object detection unit 14 that detects an object from the image data acquired by the communication unit 13, and a size determination unit 15 that determines the size at which the object is to be displayed based on the object size information acquired by the object size acquisition unit 12 and the object detected by the object detection unit 14.

[0068] In this video conference device 10, the size at which an object is displayed is determined based on the acquired object size information and the object detected from the image, so that the size at which the object is displayed can be determined uniformly in real time even if the object moves closer or farther away in the image, making it possible to display the object at an appropriate size.

[0069] The video conference device 10 may further include an output unit 16 that outputs image data indicating an image of the object so that the object can be displayed at the size determined by the size determination unit 15. In this case, the object can be easily displayed at the determined size.

[0070] Based on the output image data, the output unit 16 displays an image of the object on the display device 106 at the location P1 where the video conference device 10 is installed, and the display area may be a part or the whole of the screen of the display device 106. In this case, based on the display area of ​​the display device 106, display size information can be easily obtained.

[0071] Based on the output image data, the output unit 16 displays an image of the object on a screen at the location P1 where the video conference device 10 is installed, and the display area may be a projection area of ​​the image on the screen. With this configuration, even if the screen is large, the image of the object can be displayed at an appropriate size.

[0072] The video conference device 10 may further include a distance acquisition unit 18 that acquires the distance between the user and the display surface of the image by the output unit 16, and the size determination unit 15 may determine the size at which the object is to be displayed based further on the distance acquired by the distance acquisition unit 18. In this case, it becomes possible to display the object at a more appropriate size.

[0073] The size determination unit 15 may calculate the proportion of the object detected by the object detection unit 14 in the image data acquired by the communication unit 13, and determine the size at which the object is to be displayed based on the calculation result. In this case, the size at which the object is to be displayed can be easily determined based on the proportion of the image data that the object occupies.

[0074] The object size acquisition unit 12 acquires a typical specified value indicating the size of a human face, calculates object size information based on the acquired specified value, and the object detected by the object detection unit 14 may be a human face. In this case, it becomes possible to display a human face as an object at an appropriate size. In addition, since the object size information is acquired in a short time, it becomes easier to determine the size at which the object is displayed in real time.

[0075] 6. Other embodiments (1) In the above embodiment, the display area of ​​the image is the screen or the like of the display device 106 provided at the location P1, but the embodiment is not limited to this. The display area of ​​the image may be a part or the whole of the screen of the display device 210 provided at the location P2. In this case, the communication unit 13 receives information indicating the display area of ​​the image from the information processing device 200. The display size acquisition unit 11 acquires the display area of ​​the image based on the information received by the communication unit 13.

[0076] That is, at a location P2 different from the location where the video conference device 10 is installed, a display device 210 is installed to display an image based on image data acquired by the communication unit 13, and the display area may be a part or the whole of the screen of the display device 210. In this case, display size information can be easily acquired based on the display area of ​​the display device 210.

[0077] (2) In the above embodiment, the object is the face of the call partner at location P2, but the embodiment is not limited to this. The object may be the face of the user at location P1. In this case, an image showing the user's face of a certain size can be displayed on display device 106 in real time as a moving image, regardless of the size of the display area of ​​the image at location P1.

[0078] Moreover, image data showing the image may be transmitted to the information processing device 200 through the communication unit 13. In this case, regardless of the size of the display area of ​​the image at the location P1, an image showing the user's face of a certain size can be displayed in real time as a moving image on the display device 210. Therefore, the other party in the call can visually recognize the user's face of a certain size on the display device 210.

[0079] (3) In the above embodiment, the object to be detected by the object detection unit 14 is a human face, but the embodiment is not limited to this. The object to be detected by the object detection unit 14 may be an object such as a prototype that needs to be presented to a user or a call partner. In this case, a measuring device that measures the size of the object may be provided at the site P1 or the site P2. In this case, the object size acquisition unit 12 can acquire the size of the object from the measuring device and calculate object size information based on the acquired size.

[0080] That is, the object size acquisition unit 12 acquires the size of an object from a measuring device that measures the size of the object, calculates object size information based on the acquired size, and the object detected by the object detection unit 14 may be an object. In this case, it becomes possible to display any object as an object at an appropriate size. Also, since the object size information is acquired in a short time, it becomes easier to determine the size at which the object is displayed in real time.

[0081] (4) In the above embodiment, object size obtaining unit 12 obtains object size information indicating the size of an object corresponding to the screen size of display device 106, but the embodiment is not limited to this. Object size obtaining unit 12 may obtain object size information indicating the size of an object corresponding to the display area of ​​an image.

[0082] (5) In the above embodiment, the distance acquisition unit 18 acquires the distance between the display surface of the image and the user from the distance measurement sensor 111, but the embodiment is not limited to this. The distance acquisition unit 18 may acquire the distance between the display surface of the image and the user based on the sound collected by the microphone 109. In this case, the microphone 109 may be an array microphone. This allows the position of the user to be specified more accurately. As a result, the distance between the display surface of the image and the user can be acquired more accurately. Furthermore, when the distance between the display surface of the image and the user is acquired based on the sound collected by the microphone 109, the video conference system 100 does not need to include the distance measurement sensor 111.

[0083] (6) In the above embodiment, the video conference device 10 includes the output unit 16, the size switching unit 17, the distance acquisition unit 18, the frame switching unit 19, and the image processing unit 20, but the embodiment is not limited to this. The video conference device 10 does not need to include some or all of the output unit 16, the size switching unit 17, the distance acquisition unit 18, the frame switching unit 19, and the image processing unit 20.

[0084] For example, in the case where image data is transmitted to the information processing device 200 by the communication unit 13 without being output to the display device 106 or the projection device 114, the video conference device 10 may not include the output unit 16. In this case, the video conference system 100 may not include either the display device 106 or the projection device 114. In the case where it is not necessary to switch the display size or the frame of the displayed image, the video conference device 10 may not include the size switching unit 17 or the frame switching unit 19.

[0085] When the display magnification does not need to be changed according to the distance between the image display surface and the user, the video conference device 10 does not need to include the distance acquisition unit 18. In this case, the video conference system 100 does not need to include the distance measurement sensor 111. When the image data does not need to be processed, the video conference device 10 does not need to include the image processing unit 20.

[0086] 7. Summary of the embodiment (1) The video conference device according to paragraph 1 is a display size acquisition unit that acquires display size information indicating a size of a display area of ​​an image; an image acquisition unit that acquires image data representing an image; an object detection unit that detects an object from the image data acquired by the image acquisition unit; an object size acquisition unit that acquires object size information indicating a size of an object corresponding to the display area; The display device further includes a size determination unit that determines a size at which the object is to be displayed, based on the object size information acquired by the object size acquisition unit and the object detected by the object detection unit.

[0087] In this video conference device, the size at which an object is displayed is determined based on the acquired object size information and the object detected from the image, so that the size at which the object is displayed can be determined uniformly in real time even if the object moves closer or farther away in the image, making it possible to display the object at an appropriate size.

[0088] (2) The video conference device according to the first aspect of the present invention The display device may further include an output unit that outputs image data indicating an image of the object so that the object can be displayed at the size determined by the size determination unit.

[0089] In this case, the object can be easily displayed at the determined size.

[0090] (3) In the video conference device according to the second aspect, the output unit displays an image of the object on a first display device in a first location where the video conference device is installed based on the output image data; The display area may be a part or the whole of a screen of the first display device.

[0091] In this case, the display size information can be easily obtained based on the display area of ​​the first display device.

[0092] (4) In the video conference device according to the second aspect, The output unit displays an image of the object on a screen at a first location where the video conference device is installed based on the output image data; and The display area may be a projection area of ​​an image on the screen.

[0093] According to this configuration, even if the screen is large, the image of the object can be displayed at an appropriate size.

[0094] (5) The video conference device according to any one of the above items 2 to 4, a distance acquisition unit that acquires a distance between a display surface of an image displayed by the output unit and a user, The size determination unit may determine a size at which to display the object, further based on the distance acquired by the distance acquisition unit.

[0095] In this case, it becomes possible to display the object at a more appropriate size.

[0096] (6) In the video conference device according to the first or second aspect, a second display device is provided at a second location different from the location where the video conference device is provided, the second display device displaying an image based on the image data acquired by the image acquisition unit; The display area may be a part or the whole of a screen of the second display device.

[0097] In this case, the display size information can be easily obtained based on the display area of ​​the second display device.

[0098] (7) In the video conference device according to any one of paragraphs 1 to 6, The size determination unit may calculate a proportion of the image data acquired by the image acquisition unit that is occupied by the object detected by the object detection unit, and determine a size at which to display the object based on a result of the calculation.

[0099] In this case, the size at which the object is displayed can be easily determined based on the proportion of the image data that the object occupies.

[0100] (Item 8) In the video conference device according to any one of items 1 to 7, The object size acquisition unit acquires a typical standard value indicating a size of a human face, and calculates object size information based on the acquired standard value; The object detected by the object detection unit may be a human face.

[0101] In this case, it is possible to display a human face as an object at an appropriate size. Also, since the object size information is acquired in a short time, it becomes easier to determine the size at which the object is to be displayed in real time.

[0102] (Item 9) In the video conference device according to any one of items 1 to 8, the object size acquisition unit acquires a size of the object from a measuring device that measures the size of the object, and calculates object size information based on the acquired size; The object detected by the object detection unit may be the physical object.

[0103] In this case, it becomes possible to display any object at an appropriate size. Also, since object size information is obtained in a short time, it becomes easier to determine the size at which the object is to be displayed in real time.

[0104] (Clause 10) The video conference method according to Clause 10 is Obtaining display size information indicating a size of a display area of ​​the image; acquiring image data indicative of an image; Detecting objects from the acquired image data; obtaining object size information indicating a size of an object corresponding to the display area; determining a size at which to display the object based on the obtained object size information and the detected object; It is executed by a computer.

[0105] According to this video conference method, even if an object moves closer or further away in the image, the size at which the object is displayed can be determined uniformly in real time, thereby making it possible to display the object at an appropriate size.

[0106] (Article 11) The video conference program referred to in paragraph 11 shall be obtaining display size information indicating a size of a display area of ​​the image; obtaining image data indicative of an image; detecting an object from the acquired image data; obtaining object size information indicating a size of an object corresponding to the display area; determining a size at which to display the object based on the obtained object size information and the detected object; It is executed by a computer.

[0107] According to this video conference program, even if an object moves closer or farther in the image, the size at which the object is displayed can be determined uniformly in real time, thereby making it possible to display the object at an appropriate size. [Explanation of symbols]

[0108] 10... video conference device, 11... display size acquisition unit, 12... object size acquisition unit, 13... communication unit, 14... object detection unit, 15... size determination unit, 16... output unit, 17... size switching unit, 18... distance acquisition unit, 19... frame switching unit, 20... image processing unit, 100... video conference system, 101... storage device, 102... CPU, 103... RAM, 104... ROM, 105... communication I / F, 106, 210... display device, 107... operation unit, 108, 220... imaging device, 109, 230... microphone, 110, 240... speaker, 111... distance measurement sensor, 112... bus, 113... storage medium, 114... projection device, 200... information processing device, P1, P2... base

Claims

1. a display size acquisition unit that acquires display size information indicating a size of a display area of ​​an image; an image acquisition unit that acquires image data representing an image; an object detection unit that detects an object from the image data acquired by the image acquisition unit; an object size acquisition unit that acquires object size information indicating a size of the object corresponding to the display area; a size determination unit that determines a size at which an object is to be displayed, based on the object size information acquired by the object size acquisition unit and the object detected by the object detection unit.

2. The video conference device according to claim 1 , further comprising an output unit that outputs image data representing an image of the object so that the object can be displayed in the size determined by the size determination unit.

3. The output unit displays an image of the object on a first display device in a first location where the video conference device is installed based on the output image data; 3. The video conference device according to claim 2, wherein the display area is a part or the whole of a screen of the first display device.

4. The output unit displays an image of the object on a screen at a first location where the video conference device is installed based on the output image data; and 3. The video conference device according to claim 2, wherein the display area is a projection area of ​​an image on the screen.

5. a distance acquisition unit that acquires a distance between a display surface of an image displayed by the output unit and a user, 5. The video conference device according to claim 2, wherein the size determination unit determines a size at which to display the object, further based on the distance acquired by the distance acquisition unit.

6. a second display device is provided at a second location different from the location where the video conference device is provided, the second display device displaying an image based on the image data acquired by the image acquisition unit; 3. The video conference device according to claim 1, wherein the display area is a part or the whole of a screen of the second display device.

7. The video conferencing device according to any one of claims 1 to 4, wherein the size determination unit calculates a proportion of the image data acquired by the image acquisition unit that is occupied by the object detected by the object detection unit, and determines a size at which to display the object based on a calculation result.

8. The object size acquisition unit acquires a typical standard value indicating a size of a human face, and calculates object size information based on the acquired standard value; 5. The video conference device according to claim 1, wherein the object detected by the object detection unit is a human face.

9. the object size acquisition unit acquires a size of the object from a measuring device that measures the size of the object, and calculates object size information based on the acquired size; 5. The video conference device according to claim 1, wherein the object detected by the object detection unit is the physical object.

10. Obtaining display size information indicating a size of a display area of ​​the image; acquiring image data indicative of an image; Detecting objects from the acquired image data; obtaining object size information indicating a size of the object corresponding to the display area; determining a size at which to display the object based on the obtained object size information and the detected object; A computer-implemented method for videoconferencing.

11. obtaining display size information indicating a size of a display area of ​​the image; obtaining image data indicative of an image; detecting an object from the acquired image data; obtaining object size information indicating a size of the object corresponding to the display area; determining a size at which to display the object based on the obtained object size information and the detected object; A videoconferencing program executed by a computer.

Images