Display method and display system

Abstract

To provide a display method and a display system with which it is possible to given an impression to an observer that a person is display in life size, irrespective of the size of display region of a display device.SOLUTION: The display method includes: acquiring information that indicates the size of projection region SCA of a projector 20; acquiring an input image in which a person is included; generating information that indicates the size of face of the person included in the input image, on the basis of information that indicates the size of the projection region SCA; correcting the input image on the basis of information that indicates the size of face of the person, so that the size of face of the person included in the input image approaches the life size in the projection region SCA; generating projection image data that indicates a projection image G that includes the input image; and displaying the projection image G to the projection region SCA, on the basis of the projection image data.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a display method and a display system. [Background technology]

[0002] Patent Document 1 discloses a remote communication device that projects a person's entire body onto a screen with a diagonal size of 80 inches or more, giving the viewer the impression that the person is displayed life-size. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-167614 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, various display devices are expected to be used as remote communication devices. In this case, the size of the screen on which a person is displayed, i.e., the size of the display area, varies depending on the display device used. The remote communication device described in Patent Document 1 was unable to give the observer the impression that the person is displayed life-size on various display devices with different display area sizes. [Means for solving the problem]

[0005] The display method includes acquiring information indicating the size of a display area of ​​a display device, acquiring an input image including a person, generating information indicating the size of the person's face included in the input image based on the information indicating the size of the display area, correcting the input image based on the information indicating the size of the person's face so that the size of the person's face included in the input image on the display area approaches life-size, generating first image data indicating a first image including the input image, and displaying the first image in the display area based on the first image data.

[0006] The display system includes a processing device that generates first image data representing a first image, and a display device that displays the first image in a display area based on the first image data, wherein the processing device acquires information indicating the size of the display area, acquires an input image including a person, generates information indicating the size of the person's face included in the input image based on the information indicating the size of the display area, corrects the input image based on the information indicating the size of the person's face so that the size of the person's face included in the input image on the display area approaches life-size, and generates the first image data representing the first image including the input image. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a display system according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating an example configuration of a distant display system according to an embodiment of the present disclosure. [Figure 3] FIG. 1 is a diagram showing an example of the configuration of an image supply device according to an embodiment of the present disclosure. [Figure 4] FIG. 10 is a diagram showing an example of an input image. [Figure 5] FIG. 10 is a diagram showing an example of a projection image in which the size of a life-size person's face is not taken into consideration when the image is displayed in the projection area. [Figure 6]FIG. 10 is a diagram showing an example of a projection image that takes into account the size of a life-size person's face and is displayed in the projection area. [Figure 7] 10 is a flowchart showing the flow of a display method executed by a processing device of an image supply device according to a program. [Figure 8] 10 is a flowchart showing the flow of a display size acquisition process. [Figure 9] 10 is a flowchart showing the flow of a face size calculation process. [Figure 10] 10 is a flowchart showing the flow of an image generation process. DETAILED DESCRIPTION OF THE INVENTION

[0008] The embodiments described below are subject to various technically preferable limitations, but the embodiments of the present disclosure are not limited to the following embodiments.

[0009] 1. Embodiment Fig. 1 is a diagram illustrating an example configuration of a display system 1 according to an embodiment of the present disclosure. As shown in Fig. 1, the display system 1 includes an image supply device 10, a projector 20, and a camera 30. The image supply device 10 is connected to the projector 20 via wired communication. The image supply device 10 is also connected to the camera 30 via wired communication. The form of communication between the image supply device 10 and the projector 20 and the camera 30 is not limited to wired communication, and may be wireless communication.

[0010] A display system 1 according to an embodiment of the present disclosure is a system for conducting video conferencing, and communicates with a remote display system 2 (see FIG. 2 ) connected to the display system 1 via a network. More specifically, an image supply device 10 included in the display system 1 is connected via wireless network communication to an image supply device 11 included in the remote display system 2. The form of communication between the image supply device 10 and the image supply device 11 is not limited to wireless network communication, and may be wired network communication.

[0011] FIG. 2 is a diagram showing an example configuration of a distant display system 2. As shown in FIG. 2, the distant display system 2, like the display system 1, includes an image supply device 11, a projector 21, and a camera 31. The image supply device 11 has the same configuration as the image supply device 10. The projector 21 has the same configuration as the projector 20. The camera 31 has the same configuration as the camera 30. The connection between the image supply device 11 and the projector 21, and the connection between the image supply device 11 and the camera 31 are also the same as those in the display system 1.

[0012] 1 and 2, the image supply device 10, details of which will be described later, receives input image data indicating an input image I received from a distant display system 2. The image supply device 10 also supplies projection image data generated based on the received input image data to a projector 20. The image supply device 10 also transmits an image captured by a camera 30, which will be described later, to an image supply device 11 provided in the distant display system 2.

[0013] The projector 20 projects a projection image G based on the projection image data supplied from the image supply device 10 onto the screen SC. As shown in FIG. 1, the area on the screen SC onto which the projection image G is projected is a projection area SCA. In FIG. 1, the projection area SCA is the area indicated by a dotted line. The projector 20 is an example of a display device, and the projection area SCA is an example of a display area.

[0014] The display device is not limited to the projector 20 as long as it is a device that can display an image based on image data supplied from the image supply device 10. The display device may be, for example, a self-luminous display device such as a liquid crystal display device that displays an image on a liquid crystal display panel or an organic EL display device that displays an image on an organic EL panel. In this case, the display area may be an area on a display panel provided in the display device.

[0015] The camera 30 acquires a captured image of an area including the face of a user U using the display system 1, and transmits the captured image to the image supply device 10. The captured image captured by the camera 30 is transmitted by the image supply device 10 to the distant display system 2, and the distant display system 2 projects a projection image G2 generated based on the captured image including the face of the user U from the projector 21 onto the screen SC2, and displays the projection image G2 in a projection area SCA2 on the screen SC2.

[0016] Similarly to the display system 1, the distant display system 2 acquires a captured image of an area including the face of a user V using the distant display system 2, captured by a camera 31. The captured image of the area including the face of the user V is transmitted to the image supply device 10 as the above-mentioned input image I. Specifically, input image data representing the input image I is transmitted to the image supply device 10. In other words, the input image data is image data representing the user V, i.e., the input image I including a person.

[0017] Although details will be described later, the image supply device 10 enlarges or reduces, based on the size of the projection area SCA, an input image I based on input image data received from a distant display system 2. Furthermore, the image supply device 10 projects, from the projector 20, a projection image G based on projection image data including the enlarged or reduced input image I onto the projection area SCA, so that the face of the user V included in the input image I is displayed in a size close to life-size.

[0018] FIG. 3 is a diagram showing an example of the configuration of the image supply device 10. The image supply device 10 is a device that transmits generated projection image data to the projector 20, and may be, for example, a personal computer. As shown in FIG. 3, the image supply device 10 includes a processing device 110, a communication device 120, a storage device 130, and an input device 140. The input device 140 is, for example, a keyboard or a mouse. The processing device 110 is one or more processors, for example, a CPU (Central Processing Unit). The processing device 110 controls the image supply device 10 by operating in accordance with various programs stored in the storage device 130.

[0019] The communication device 120 is a device that performs wireless or wired communication with other devices, and includes, for example, an interface circuit. Specific examples of other devices that communicate with the communication device 120 include the projector 20, the camera 30, and the image supply device 11 of the remote display system 2 that supplies input image data to the image supply device 10. The communication device 120 receives input image data from the image supply device 11 included in the remote display system 2. The communication between the communication device 120 and other devices may be wired communication via a communication line such as a USB (Universal Serial Bus) cable or a LAN (Local Area Network) cable, or wireless communication via a wireless communication network such as a wireless LAN.

[0020] The storage device 130 is a recording medium readable by the processing device 110. The storage device 130 includes, for example, a nonvolatile memory and a volatile memory. The nonvolatile memory is, for example, a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read Only Memory). The volatile memory is, for example, a RAM (Random Access Memory). The nonvolatile memory of the storage device 130 stores various programs including a program P described below, and reference face size information B. The volatile memory of the storage device 130 is used by the processing device 110 as a work area when executing the various programs.

[0021] Examples of various programs stored in the nonvolatile memory include a kernel program and program P (not shown). The kernel program is a program for causing the processing device 110 to implement an OS (Operating System). When the image supply device 10 is powered on, the processing device 110 reads the kernel program from the nonvolatile memory to the volatile memory and starts executing the read kernel program. The processing device 110, operating in accordance with the kernel program, starts executing another program when, for example, a user U operates the input device 140 to instruct the processing device 110 to start executing the other program.

[0022] For example, when an instruction to start execution of program P is given by an operation on input device 140, processing device 110 reads program P from non-volatile memory to volatile memory and starts execution of program P read to volatile memory. Processing device 110 operating in accordance with program P functions as display size acquisition unit 111, image acquisition unit 112, face size calculation unit 113, image generation unit 114, and image output unit 115. In other words, each of display size acquisition unit 111, image acquisition unit 112, face size calculation unit 113, image generation unit 114, and image output unit 115 is a software module realized by operating processing device 110 in accordance with program P. The roles of each of display size acquisition unit 111, image acquisition unit 112, face size calculation unit 113, image generation unit 114, and image output unit 115 are as follows.

[0023] The display size acquisition unit 111 acquires the size of the projection area SCA onto which the projector 20 projects the projection image G on the screen SC, and acquires the PPI (Pixels Per Inch) of the projection image G on the screen SC based on the acquired size of the projection area SCA. The size of the projection area SCA is an example of the size of the display area, and the PPI of the projection image G on the screen SC is an example of information indicating the size of the display area.

[0024] The display size acquisition unit 111 acquires the size of the projection area SCA through an operation of the input device 140 by the user U. More specifically, the user U measures the length of a diagonal of the approximately rectangular projection area SCA on the screen SC, and inputs the measured diagonal length as the size of the projection area SCA to the image supply device 10 through an operation of the input device 140. The display size acquisition unit 111 acquires the diagonal length input by the user U as the size of the projection area SCA.

[0025] Furthermore, the display size acquisition unit 111 may be configured to have the user U input the lengths of at least two of the four sides that constitute the projection area SCA, which intersect with each other, as the size of the projection area SCA. In this case, if the projection area SCA is considered to be a rectangle, the diagonal of the projection area SCA can be considered to be a right triangle with the hypotenuse as the diagonal, one of the two input sides as the opposite side, and the other as the adjacent side. In this case, the well-known Pythagorean theorem can be used to calculate the length of the diagonal of the projection area SCA, which is the hypotenuse of the right triangle, from the lengths of the two input sides.

[0026] Furthermore, the display size acquisition unit 111 may use a distance measuring device (not shown) to measure the length of a diagonal line of the projection area SCA on the screen SC. In this case, the distance measuring device measures the three-dimensional coordinate values ​​of the positions on the screen SC of the start point and end point of a line segment that is a diagonal line of the projection area SCA, and calculates the length of the diagonal line of the projection area SCA based on these three-dimensional coordinate values.

[0027] The distance measuring device may be any device capable of determining the three-dimensional coordinate values ​​of the start and end points on the screen SC, and may be, for example, a stereo camera system consisting of two cameras or a depth camera. The depth camera may be, for example, a LiDAR (Light Detection and Ranging) sensor using a ToF (Time of Flight) method. The three-dimensional coordinate values ​​of the start and end points on the screen SC correspond to the three-dimensional coordinate values ​​of the display area.

[0028] Furthermore, the display size acquisition unit 111 may use a stereo camera system using the projector 20 and the camera 30 as a distance measurement device. In this case, the camera 30 is placed so that the screen SC falls within the imaging range. Furthermore, it is assumed that the positional relationship between the projector 20 and the camera 30 has already been measured and is known.

[0029] When using a stereo camera system using the projector 20 and the camera 30, the display size acquisition unit 111 first acquires a captured image from the projector 20, captured by the camera 30, of a known pattern image projected onto the screen SC. Then, by analyzing the captured image, the display size acquisition unit 111 acquires a correspondence relationship between a captured image coordinate system indicating positions on the captured image captured by the camera 30 and a projector coordinate system indicating positions on the projected image G of the projector 20. The pattern image may be any image having detectable features, and may be, for example, a gray code pattern image, a phase shift pattern image, or a checkered pattern image. The pattern image is an example of an image for measurement.

[0030] Next, by using this correspondence, display size acquisition unit 111 converts the coordinate values ​​of each pixel constituting the captured image captured by camera 30 into the projector coordinate system, and generates a captured image when captured from the position of the projector. Finally, by using the positional relationship between projector 20 and camera 30, the generated captured image, and the captured image captured by camera 30, display size acquisition unit 111 calculates three-dimensional coordinate values ​​of the positions on screen SC of the start point and end point, similar to a known stereo camera system. Furthermore, instead of camera 30, a camera (not shown) different from camera 30 may be used.

[0031] Furthermore, the display size acquisition unit 111 calculates the PPI of the projection image G on the screen SC based on the length of the diagonal of the projection area SCA determined from the calculated three-dimensional coordinate values. Here, if the number of pixels in the horizontal direction of the projection image G, i.e., in the x-axis direction, is n, the number of pixels in the vertical direction, i.e., in the y-axis direction, is m, and the length of the diagonal of the projection area SCA is s, the PPI can be calculated by the following formula (1).

[0032]

number

[0033] The display size acquisition unit 111 acquires the PPI of the projection image G on the screen SC by substituting the number of pixels in the horizontal and vertical directions of the projection image G and the length of the diagonal line of the projection area SCA into equation (1). Here, the unit of the diagonal line length s is inches. The number of pixels in the horizontal and vertical directions of the projection image G is set in advance by the user U operating the input device 140. The set number of pixels in the horizontal and vertical directions of the projection image G may be stored in the storage device 130.

[0034] The image acquisition unit 112 controls the communication device 120 to acquire input image data representing an input image I including a person. The input image I including a person is a captured image of a user V, who is a person, captured by a remote camera 31. Here, the person included in the input image I is not limited to an actual person captured by the camera 31. For example, an image including an avatar with a face, represented by two-dimensional or three-dimensional computer graphics, may be acquired as the input image I. The user V, who is a person included in the captured image, and the avatar with a face are examples of people.

[0035] The face size calculation unit 113 analyzes the input image data acquired by the image acquisition unit 112, and calculates the size of the face of a person included in the input image I on the projection area SCA when the input image I is projected onto the projection area SCA by the projector 20.

[0036] More specifically, first, the face size calculation unit 113 uses a known machine learning system to detect the positions of the left eye EL and the right eye ER in the face of a person included in the input image I. The position of the left eye EL is, for example, the position of the center of gravity of the iris included in the left eye EL. Similarly, the position of the right eye ER is, for example, the position of the center of gravity of the iris included in the right eye ER.

[0037] 4 is a diagram showing an example of input image I. Input image I is an image of user V captured by camera 31 included in distant display system 2. As shown in FIG. 4, input image I includes images showing user V and the left eye EL and right eye ER of user V.

[0038] Next, the face size calculation unit 113 calculates the distance between the positions of the left eye EL and the right eye ER detected in the input image I. Hereinafter, the distance between the positions of the left eye EL and the right eye ER will be referred to as the interpupillary distance. The interpupillary distance in the input image I is expressed as the number of pixels. Furthermore, the detected position of the left eye EL is an example of a first position, and the detected position of the right eye ER is an example of a second position.

[0039] Next, the face size calculation unit 113 converts the interpupillary distance in the input image I, which is expressed in pixels, into an interpupillary distance expressed in inches, using the PPI of the projection image G calculated by the display size acquisition unit 111. The interpupillary distance converted into inches is acquired as the size of the face of the person included in the input image I on the projection area SCA. The interpupillary distance converted into inches is information indicating the size of the face of the person included in the input image on the display area, and is an example of a first distance. Hereinafter, the interpupillary distance converted into inches will be referred to as the calculated value of the interpupillary distance.

[0040] Furthermore, the face size calculation unit 113 detects the positions of the left eye EL and the right eye ER in the face of the person included in the input image I using a known machine learning system, but this is not limiting. For example, the positions of the left eye EL and the right eye ER may be detected using a known template matching process. It is sufficient to at least detect the positions of the left eye EL and the right eye ER in the face of the person included in the input image I.

[0041] Furthermore, although the face size calculation unit 113 has been described as detecting the positions of the left eye EL and the right eye ER in the face of a person included in the input image I, this is not limiting. The face size calculation unit 113 may also detect the region of the person's face included in the input image I. In this case, the area of ​​the region detected as the region of the person's face may be acquired as information indicating the size of the person's face included in the input image I.

[0042] The image generation unit 114 enlarges or reduces the size of the input image I so that the size of the face of the person included in the input image I on the projection area SCA, calculated by the face size calculation unit 113, approaches the size of a life-size person's face. Furthermore, the image generation unit 114 generates projection image data that indicates a projection image G that includes the enlarged or reduced input image I. The process of enlarging or reducing the size of the input image I is an example of correcting the input image I.

[0043] Fig. 5 is a diagram showing an example of a case where a projection image G, which does not take into account the size of a life-size person's face, is displayed in a projection area SCA. In Fig. 5, a projector 20 projects an input image I, such as that shown in Fig. 4, received from a distant display system 2 onto a screen SC as a projection image G, and displays the projection image G in the projection area SCA. In Fig. 5, a life-size user V is shown by a dotted line for size comparison.

[0044] In the display system 1, the size of the projection area SCA varies depending on the type of display device used and the size and installation position of the screen SC. Therefore, as shown in Fig. 5, if an input image I based on input image data received from a distant display system 2 is projected as a projection image G from the projector 20, there is a risk that the user V included in the projection image G will be displayed at a size different from life-size.

[0045] On the other hand, Fig. 6 shows an example of a case where a projection image G, which takes into account the size of a life-size person's face, is displayed in the projection area SCA. In Fig. 6, the input image I shown in Fig. 4 received from the distant display system 2 is enlarged or reduced by the image generation unit 114 so that the size of the person's face in the projection area SCA approaches the size of a life-size person's face, and the projector 20 projects the projection image G including this input image I onto the screen SC, displaying the projection image G in the projection area SCA. As in Fig. 5, in Fig. 6, a life-size user V is shown by a dotted line for size comparison.

[0046] As shown in FIG. 6, the face of the user V included in the projection image G is displayed in the projection area SCA at a size close to life-size by the processing performed by the image generation unit 114.

[0047] The processing of the image generation unit 114 will be described in detail. First, the image generation unit 114 reads out the reference face size information B stored in the storage device 130. The reference face size information B includes information on the size of a life-size person's face. Specifically, the reference face size information B includes a reference value of the interpupillary distance for a life-size person. The unit of this reference value of the interpupillary distance is inches, and the value of the reference value of the interpupillary distance is, for example, 2.5 inches. Here, 2.5 inches is the average value of the interpupillary distances actually measured for Japanese people aged 18 to 34. The value of the reference value of the interpupillary distance is not limited to 2.5 inches, and another value may be used. The information on the size of a life-size person's face included in the reference face size information B, i.e., the reference value of the interpupillary distance, is an example of a second distance.

[0048] Next, the image generation unit 114 compares the calculated value of the interpupillary distance calculated by the face size calculation unit 113 with a first threshold value that is greater than the reference value of the interpupillary distance included in the reference face size information B. The first threshold value is, for example, 3.0 inches. If the calculated value of the interpupillary distance is greater than or equal to the first threshold value, the image generation unit 114 reduces the size of the input image I so that the calculated value of the interpupillary distance approaches the reference value of the interpupillary distance. For example, the size of the input image I may be reduced so that the calculated value of the interpupillary distance matches the reference value of the interpupillary distance.

[0049] If the calculated interpupillary distance is less than the first threshold, the image generator 114 compares the calculated interpupillary distance with a second threshold that is smaller than the reference interpupillary distance. The second threshold is, for example, 2.0 inches.

[0050] If the calculated value of the interpupillary distance is greater than the second threshold, the image generation unit 114 determines that the calculated value of the interpupillary distance is approximately the same as the reference value of the interpupillary distance, and does not perform processing to enlarge or reduce the input image I. In this case, the image generation unit 114 generates projection image data that indicates a projection image G that includes the input image I that has not been subjected to processing to enlarge or reduce it.

[0051] If the calculated interpupillary distance is equal to or less than the second threshold, the image generation unit 114 determines whether the projection area SCA has a size sufficient to display a life-size human face. More specifically, the image generation unit 114 compares a third threshold indicating the size of the projection area capable of displaying a life-size human face with the length of the diagonal of the projection area SCA, which indicates the size of the projection area SCA. The third threshold is, for example, 15.0 inches.

[0052] If the calculated interpupillary distance is equal to or less than the second threshold and the diagonal length of the projection area SCA is equal to or greater than the third threshold, the size of the input image I is enlarged so that the calculated interpupillary distance approaches the reference interpupillary distance value. If the diagonal length of the projection area SCA is equal to or greater than the third threshold, the size of the projection area SCA is large enough to display a life-size human face, and the size of the input image I can be enlarged so that the calculated interpupillary distance approaches the reference interpupillary distance value. For example, the size of the input image I may be enlarged so that the calculated interpupillary distance matches the reference interpupillary distance value.

[0053] On the other hand, if the calculated interpupillary distance is equal to or less than the second threshold and the diagonal length of the projection area SCA is less than the third threshold, the size of the input image I is enlarged so that the calculated interpupillary distance approaches a third distance that is shorter than the reference value of the interpupillary distance. The third distance is, for example, one-sixth the length of the diagonal length of the projection area SCA.

[0054] Here, if the length of the diagonal of the projection area SCA is less than a third threshold, it is determined that the size of the projection area SCA is not large enough to display a life-size person's face, and therefore the person's face cannot be displayed in life-size without extending beyond the projection area SCA. In other words, the size of the input image I cannot be enlarged so that the calculated value of the interpupillary distance approaches the reference value of the interpupillary distance. Therefore, the size of the input image I is enlarged so that the calculated value of the interpupillary distance approaches a third distance that is shorter than the reference value of the interpupillary distance. For example, the size of the input image I may be enlarged so that the calculated value of the interpupillary distance matches the third distance. Alternatively, if the calculated value of the interpupillary distance is equal to or less than the second threshold and the length of the diagonal of the projection area SCA is less than the third threshold, correction based on the calculated value of the interpupillary distance may not be performed on the input image I.

[0055] When enlarging or reducing the size of the input image I, the difference between the calculated interpupillary distance value and the reference interpupillary distance value may be converted into the number of pixels based on the PPI of the projection image G calculated by the display size acquisition unit 111, and the input image I may be enlarged or reduced based on the ratio between the interpupillary distance in the input image I and the difference value converted into the number of pixels.

[0056] Next, the image generation unit 114 generates projection image data representing a projection image G including the enlarged or reduced input image I. The projection image G only needs to include at least a person included in the enlarged or reduced input image I. For example, the projection image G may include an image of the enlarged or reduced input image I that has been trimmed so that an area including the person remains. When trimming the enlarged or reduced input image I, for example, the trimming may be performed so that the aspect ratio of the input image I and the aspect ratio of the projection area SCA are close to each other. Furthermore, the projection image G only needs to include at least a person included in the input image I, and may be a composite image including other images.

[0057] The image output unit 115 controls the communication device 120 to output the projection image data generated by the image generation unit 114 to the projector 20. The projection image G based on the projection image data is an example of a first image. The projection image data is also an example of first image data.

[0058] 7 is a flowchart showing the flow of a display method executed by processing device 110 operating according to program P. As shown in FIG. 7, this display method includes the processes of steps SA100 to SA500.

[0059] In step SA100, the processing device 110 functions as a display size obtaining unit 111. In step SA100, the processing device 110 executes a display size obtaining process. Fig. 8 is a flowchart showing the flow of the display size obtaining process. As shown in Fig. 8, the display size obtaining process includes the processes of steps SA110 to SA120.

[0060] In step SA110, the processing device 110 acquires the length of the diagonal line of the projection area SCA as the size of the projection area SCA through an operation by the user U on the input device 140.

[0061] In step SA120 following step SA110, the PPI of the projection image G is calculated using the diagonal length of the projection area SCA obtained by the operation received from the user U and the number of horizontal and vertical pixels of the projection image G that has been set in advance by the user U.

[0062] When the processing of step SA120 is completed, the display size acquisition processing ends and the processing proceeds to step SA200. In step SA200, the processing device 110 functions as the image acquisition unit 112. In step SA200, the processing device 110 controls the communication device 120 to receive input image data and acquire the input image I.

[0063] In step SA300 following step SA200, the processing device 110 functions as the face size calculation unit 113. In step SA300, the processing device 110 executes a face size calculation process. Fig. 9 is a flowchart showing the flow of the face size calculation process. As shown in Fig. 9, the face size calculation process includes the processes of steps SA310 to SA330.

[0064] In step SA310, the processing device 110 detects the positions of the left eye EL and the right eye ER in the face of the person included in the input image I.

[0065] In step SA320 following step SA310, the processing device 110 calculates the interpupillary distance on the input image I based on the detected positions of the left eye EL and the right eye ER.

[0066] In step SA330 following step SA320, the processing device 110 obtains a calculated value of the interpupillary distance by converting the calculated interpupillary distance on the input image I to inches based on the PPI of the projection image G calculated in step SA120.

[0067] When the processing of step SA330 is completed, the face size calculation processing ends, and the processing proceeds to step SA400. In step SA400, the processing device 110 functions as the image generation unit 114. In step SA400, the processing device 110 executes image generation processing. FIG. 10 is a flowchart showing the flow of the image generation processing. As shown in FIG. 10, the image generation processing includes the processing of steps SA410 to SA480.

[0068] In step SA410, the processing device 110 reads out the reference face size information B from the storage device 130, and obtains the reference value of the interpupillary distance for a life-size person.

[0069] In step SA420, the processing device 110 determines whether the calculated value of the interpupillary distance acquired in step SA330 is equal to or greater than the first threshold value, and if the calculated value of the interpupillary distance is equal to or greater than the first threshold value (step SA420 / YES), the processing device 110 proceeds to step SA430.

[0070] In step SA430, processing device 110 reduces the size of input image I so that the calculated value of the interpupillary distance approaches the reference value of the interpupillary distance, and then proceeds to step SA480.

[0071] If the calculated interpupillary distance is less than the first threshold (step SA420 / NO), the process proceeds to step SA440.

[0072] In step SA440, the processing device 110 determines whether the calculated value of the interpupillary distance is less than or equal to the second threshold, and if the calculated value of the interpupillary distance is greater than the second threshold (step SA440 / NO), proceeds to step SA480 without enlarging or reducing the input image I.

[0073] If the calculated interpupillary distance is equal to or less than the second threshold value (YES in step SA440), the process proceeds to step SA450.

[0074] In step SA450, the processing device 110 determines whether the length of the diagonal of the projection area SCA acquired in step SA110 is greater than or equal to a third threshold, and if the length of the diagonal of the projection area SCA is greater than or equal to the third threshold (step SA450 / YES), proceeds to step SA460.

[0075] In step SA460, processing device 110 enlarges the size of input image I so that the calculated value of the interpupillary distance approaches the reference value of the interpupillary distance, and then proceeds to step SA480.

[0076] If the length of the diagonal line of the projection area SCA is less than the third threshold value (step SA450 / NO), the process proceeds to step SA470.

[0077] In step SA470, processing device 110 enlarges the size of input image I so that the calculated interpupillary distance approaches a third distance that is shorter than the reference interpupillary distance value, and then proceeds to step SA480.

[0078] In step SA480, the processing device 110 generates projection image data representing a projection image G including the input image I, and ends the image generation process.

[0079] When the image generation process is completed, the process proceeds to step SA500. In step SA500, the processing device 110 functions as the image output unit 115. In step SA500, the processing device 110 controls the communication device 120 to supply the projection image data generated in step SA480 to the projector 20. The projector 20 projects a projection image G based on the projection image data supplied from the image supply device 10 onto the screen SC. As a result, the projection image G is displayed in the projection area SCA of the screen SC.

[0080] As described above, according to the display system 1 of this embodiment, the input image I received from the distant display system 2 can be enlarged or reduced in consideration of the size of the projection area SCA, and the projection image G including the enlarged or reduced input image I can be displayed in the projection area SCA. Therefore, regardless of the size of the projection area SCA, the impression can be given to the observer user U that the face of the user V, a person included in the input image I, is displayed life-size in the projection area SCA.

[0081] 2. Variations The above embodiments can be modified as follows. (1) In the above embodiment, the program P that causes the processing device 110 of the image supply device 10 to execute a display method that prominently exhibits the features of the present disclosure is pre-stored in the storage device 130 of the image supply device 10. However, the program P may be distributed in a state in which it is written to a computer-readable recording medium such as a flash ROM, or may be distributed by downloading via a telecommunications line such as the Internet. By operating a general computer in accordance with the program P distributed in these ways, the computer can be made to execute the display method of the present disclosure.

[0082] (2) In the above embodiment, the display size acquisition unit 111, the image acquisition unit 112, the face size calculation unit 113, the image generation unit 114, and the image output unit 115 are all software modules. However, at least one of the display size acquisition unit 111, the image acquisition unit 112, the face size calculation unit 113, the image generation unit 114, and the image output unit 115 may be a hardware module such as an electronic circuit.

[0083] 3. Aspects grasped from at least one of the embodiments and modifications The present disclosure is not limited to the above-described embodiments and modifications, and can be realized in various forms without departing from the spirit thereof. For example, the present disclosure can also be realized in the following forms. The technical features in the above embodiments corresponding to the technical features in each form described below can be replaced or combined as appropriate to solve some or all of the problems of the present disclosure or to achieve some or all of the effects of the present disclosure. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate.

[0084] A display method according to one embodiment of the present disclosure includes acquiring a PPI of a projection image G on a screen SC of a projector 20, acquiring an input image I including a person, generating information indicating the size of the face of the person included in the input image I based on the PPI of the projection image G on the screen SC, correcting the input image I based on the information indicating the size of the face of the person included in the input image I so that the size of the face of the person included in the input image I approaches life-size on a projection area SCA, generating projection image data indicating a projection image G including the input image I, and displaying the projection image G in the projection area SCA based on the projection image data.

[0085] According to the display method of this aspect, the input image I is corrected so that the size of the face of user V, a person included in the input image I, approaches life-size on the projection area SCA, based on the PPI of the projection image G on the screen SC and information indicating the size of the face of the person included in the input image I. Therefore, regardless of the size of the projection area SCA of the projector 20, the impression can be given to the observer, user U, that the person's face is displayed life-size.

[0086] In a more preferred embodiment of the display method, generating information indicating the size of a person's face includes detecting the position of the person's left eye EL and the position of the person's right eye ER from the input image I, and generating a calculated value of the interpupillary distance indicating the distance on the projection area SCA between the positions of the left eye EL and the right eye ER as information indicating the size of the person's face.

[0087] According to the display method of this aspect, information indicating the size of a person's face can be generated based on the interpupillary distance, which is the distance between the person's left eye EL and right eye ER. Therefore, if the input image I includes at least the person's left eye EL and right eye ER, the input image I can be corrected according to the size of the projection area SCA of the projector 20.

[0088] In another preferred embodiment of the display method, correcting the input image I includes reducing the input image I so that the calculated value of the interpupillary distance approaches a reference value of the interpupillary distance when the calculated value of the interpupillary distance is equal to or greater than a first threshold, and enlarging the input image I so that the calculated value of the interpupillary distance approaches a reference value of the interpupillary distance when the calculated value of the interpupillary distance is equal to or less than a second threshold.

[0089] According to the display method of this aspect, the calculated value of the interpupillary distance, which is the distance between the left eye EL and right eye ER of a person included in the input image I, is corrected so as to approach the reference value of the interpupillary distance, which indicates the interpupillary distance of a life-size person. As a result, the impression that the face of the person displayed in the projection area SCA is displayed life-size can be given to the observer, user U.

[0090] In another preferred embodiment of the display method, correcting the input image I includes enlarging the input image I so that the calculated interpupillary distance approaches a reference value for the interpupillary distance when the size of the projection area SCA is equal to or greater than a third threshold indicating a size at which a person's face can be displayed life-size and the calculated value of the interpupillary distance is equal to or less than the second threshold, and enlarging the input image I so that the calculated value of the interpupillary distance approaches a third distance that is shorter than the reference value for the interpupillary distance when the size of the projection area SCA is less than the third threshold and the calculated value of the interpupillary distance is equal to or less than the second threshold.

[0091] According to the display method of this aspect, if the projection area SCA of the projector 20 is not large enough to display a person's face life-size, the calculated value of the interpupillary distance of the person included in the input image I is corrected so as to approach a third distance that is shorter than the reference value of the interpupillary distance. This makes it possible to prevent the display of a projection image G that is excessively enlarged relative to the size of the projection area SCA.

[0092] In another preferred embodiment of the display method, correcting the input image I includes enlarging the input image I so that the calculated interpupillary distance approaches a reference value for interpupillary distance when the size of the projection area SCA is equal to or greater than a third threshold indicating a size at which a person's face can be displayed life-size and the calculated value of the interpupillary distance is equal to or less than the second threshold, and when the size of the projection area SCA is less than the third threshold and the calculated value of the interpupillary distance is equal to or less than the second threshold, no correction is made to the input image I based on information indicating the size of the person's face.

[0093] According to the display method of this embodiment, if the projection area SCA of the projector 20 is not large enough to display a person's face life-size, no correction is performed on the input image I. This makes it possible to prevent the display of a projection image G that is excessively enlarged relative to the size of the projection area SCA.

[0094] In another preferred embodiment of the display method, the projector 20 projects the projection image G onto the projection area SCA, and acquiring the PPI of the projection image G on the screen SC includes acquiring an image of the pattern image projected by the projector 20 taken by a camera 30 whose positional relationship with the projector 20 is known, calculating three-dimensional coordinate values ​​where the projection area SCA is located based on the positional relationship and the captured image, and acquiring the PPI of the projection image G on the screen SC, which is information indicating the size of the projection area SCA, based on the three-dimensional coordinate values ​​where the projection area SCA is located.

[0095] According to the display method of this aspect, when the projector 20 is used as the display device, the PPI of the projected image G on the screen SC can be acquired by using the projector 20 and the camera 30. Therefore, regardless of the size of the projection area SCA of the projector 20, the impression that a person's face is displayed life-size can be given to the observer, the user U.

[0096] The display system 1 of the present disclosure includes a processing device 110 that generates projection image data representing a projection image G, and a projector 20 that displays the projection image G in a projection area SCA based on the projection image data, and the processing device 110 is characterized by acquiring the PPI of the projection image G on the screen SC, acquiring an input image I that includes a person, generating information indicating the size of the person's face included in the input image I based on the PPI of the projection image G on the screen SC, correcting the input image I based on the information indicating the size of the person's face so that the size of the person's face included in the input image I on the projection area SCA approaches life-size size, and generating projection image data representing the projection image G that includes the input image I.

[0097] According to the display system 1 of this embodiment, the input image I is corrected so that the size of the face of user V, a person included in the input image I, approaches life-size on the projection area SCA, based on the PPI of the projection image G on the screen SC and information indicating the size of the face of the person included in the input image I. Therefore, regardless of the size of the projection area SCA of the projector 20, the impression can be given to the observer, user U, that the person's face is displayed life-size. [Explanation of symbols]

[0098] 1...display system, 10...image supply device, 20...projector, 30...camera, 110...processing device, 111...display size acquisition unit, 112...image acquisition unit, 113...face size calculation unit, 114...image generation unit, 115...image output unit, 120...communication device, 130...storage device, 140...input device, SC...screen, SCA...projection area, G...projected image, I...input image, EL...left eye, ER...right eye.

Claims

1. obtaining information indicating a size of a display area of ​​the display device; obtaining an input image including a person; generating information indicating a size of the face of the person included in the input image based on information indicating a size of the display area; correcting the input image based on information indicating a size of the face of the person so that the size of the face of the person included in the input image approaches life-size on the display area; generating first image data representing a first image including the input image; displaying the first image in the display area based on the first image data; Including, the display device is a projector that projects the first image onto the display area, Obtaining information indicating a size of the display area includes: acquiring a captured image of the measurement image projected by the projector using a camera whose positional relationship with the projector is known; calculating three-dimensional coordinate values ​​of the display area based on the positional relationship and the captured image; acquiring information indicating a size of the display area based on three-dimensional coordinate values ​​at which the display area is located; Display methods including.

2. Generating information indicating a size of the person's face includes: Detecting a first position indicating a position of the person's left eye and a second position indicating a position of the person's right eye from the input image; generating a first distance indicating a distance between the first position and the second position on the display area as information indicating a size of the person's face; The display method of claim 1 , comprising:

3. Correcting the input image includes: When the first distance is equal to or greater than a first threshold, reducing the input image so that the first distance approaches a second distance indicating a reference value; When the first distance is equal to or smaller than a second threshold, enlarging the input image so that the first distance approaches the second distance; The display method according to claim 2 , comprising:

4. Correcting the input image includes: enlarging the input image so that the first distance approaches a second distance indicating a reference value when the size of the display area is equal to or larger than a third threshold value indicating a size at which the face of the person can be displayed in life-size and the first distance is equal to or smaller than a second threshold value; When the size of the display area is less than the third threshold and the first distance is equal to or less than the second threshold, enlarging the input image so that the first distance approaches a third distance that is shorter than the second distance; The display method according to claim 2 , comprising:

5. Correcting the input image includes: enlarging the input image so that the first distance approaches a second distance indicating a reference value when the size of the display area is equal to or larger than a third threshold indicating a size at which the face of the person can be displayed life-size and the first distance is equal to or smaller than a second threshold; when the size of the display area is less than the third threshold and the first distance is equal to or less than the second threshold, no correction is performed on the input image based on information indicating the size of the person's face; 3. The display method according to claim 2, wherein:

6. Obtaining information indicating the size of a display area of ​​a display device; obtaining an input image including a person; generating information indicating a size of the face of the person included in the input image based on information indicating a size of the display area; correcting the input image based on information indicating a size of the face of the person so that the size of the face of the person included in the input image approaches life-size on the display area; generating first image data representing a first image including the input image; displaying the first image in the display area based on the first image data; Generating information indicating a size of the person's face includes: Detecting a first position indicating a position of the person's left eye and a second position indicating a position of the person's right eye from the input image; generating a first distance indicating a distance between the first position and the second position on the display area as information indicating a size of the person's face; Including, Correcting the input image includes: enlarging the input image so that the first distance approaches a second distance indicating a reference value when the size of the display area is equal to or larger than a third threshold value indicating a size at which the face of the person can be displayed in life-size and the first distance is equal to or smaller than a second threshold value; When the size of the display area is less than the third threshold and the first distance is equal to or less than the second threshold, enlarging the input image so that the first distance approaches a third distance that is shorter than the second distance; Display methods including.

7. Obtaining information indicating a size of a display area of ​​a display device; obtaining an input image including a person; generating information indicating a size of the face of the person included in the input image based on information indicating a size of the display area; correcting the input image based on information indicating a size of the face of the person so that the size of the face of the person included in the input image approaches life-size on the display area; generating first image data representing a first image including the input image; displaying the first image in the display area based on the first image data; Including, Generating information indicating a size of the person's face includes: Detecting a first position indicating a position of the person's left eye and a second position indicating a position of the person's right eye from the input image; generating a first distance indicating a distance between the first position and the second position on the display area as information indicating a size of the person's face; Including, Correcting the input image includes: enlarging the input image so that the first distance approaches a second distance indicating a reference value when the size of the display area is equal to or larger than a third threshold indicating a size at which the face of the person can be displayed life-size and the first distance is equal to or smaller than a second threshold; when the size of the display area is less than the third threshold and the first distance is equal to or less than the second threshold, no correction is performed on the input image based on information indicating the size of the person's face; A display method characterized by:

8. a processing device that generates first image data representing a first image; a display device that displays the first image in a display area based on the first image data; Equipped with The processing device includes: obtaining information indicating a size of the display area; obtaining an input image including a person; generating information indicating a size of the face of the person included in the input image based on information indicating a size of the display area; correcting the input image based on information indicating a size of the face of the person so that the size of the face of the person included in the input image approaches life-size on the display area; generating the first image data representing the first image including the input image; the display device is a projector that projects the first image onto the display area, Obtaining information indicating a size of the display area includes: acquiring a captured image of the measurement image projected by the projector using a camera whose positional relationship with the projector is known; calculating three-dimensional coordinate values ​​of the display area based on the positional relationship and the captured image; acquiring information indicating a size of the display area based on three-dimensional coordinate values ​​at which the display area is located; Including A display system comprising:

9. A processing device that generates first image data representing a first image; a display device that displays the first image in a display area based on the first image data; Equipped with The processing device includes: obtaining information indicating a size of the display area; obtaining an input image including a person; generating information indicating a size of the face of the person included in the input image based on information indicating a size of the display area; correcting the input image based on information indicating a size of the face of the person so that the size of the face of the person included in the input image approaches life-size on the display area; generating the first image data representing the first image including the input image; Run Generating information indicating a size of the person's face includes: Detecting a first position indicating a position of the person's left eye and a second position indicating a position of the person's right eye from the input image; generating a first distance indicating a distance between the first position and the second position on the display area as information indicating a size of the person's face; Including, Correcting the input image includes: enlarging the input image so that the first distance approaches a second distance indicating a reference value when the size of the display area is equal to or larger than a third threshold value indicating a size at which the face of the person can be displayed in life-size and the first distance is equal to or smaller than a second threshold value; When the size of the display area is less than the third threshold and the first distance is equal to or less than the second threshold, enlarging the input image so that the first distance approaches a third distance that is shorter than the second distance; A display system comprising:

10. A processing device that generates first image data representing a first image; a display device that displays the first image in a display area based on the first image data; Equipped with The processing device includes: obtaining information indicating a size of the display area; obtaining an input image including a person; generating information indicating a size of the face of the person included in the input image based on information indicating a size of the display area; correcting the input image based on information indicating a size of the face of the person so that the size of the face of the person included in the input image approaches life-size on the display area; generating the first image data representing the first image including the input image; Run Generating information indicating a size of the person's face includes: Detecting a first position indicating a position of the person's left eye and a second position indicating a position of the person's right eye from the input image; generating a first distance indicating a distance between the first position and the second position on the display area as information indicating a size of the person's face; Including, Correcting the input image includes: enlarging the input image so that the first distance approaches a second distance indicating a reference value when the size of the display area is equal to or larger than a third threshold indicating a size at which the face of the person can be displayed life-size and the first distance is equal to or smaller than a second threshold; when the size of the display area is less than the third threshold and the first distance is equal to or less than the second threshold, no correction is performed on the input image based on information indicating the size of the person's face; A display system comprising:

Images