Display system

Abstract

To inhibit occurrence of an interference-fringe in an imaged picture.SOLUTION: A display system according to an embodiment comprises a display device, a camera, a shutter arranged between the display device and the camera, and a controller for controlling the display device and the shutter. The display device operates in accordance with one frame period including a first sub-frame period for displaying a picture in a display region and a second sub-frame period for making the display region be in a transparent state. The controller cyclically controls operation of the display device by taking a plurality of one frame periods as one cycle and controls the operation of the display device so that the plurality of one frame periods included in the one cycle start their second sub-frame periods at mutually different timings from starts of the one frame periods. The controller controls operation of the shutter so as to block light toward the camera in the first sub-frame period and so as to transmit light toward the camera in the second sub-frame period.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] Embodiments of the present invention relate to a display system.

Background Art

[0002] In recent years, Web conferences and video calls have become widespread. In such Web conferences and video calls, there are cases where the line of sight between a person looking at an image (video) displayed on a screen and the person displayed on the screen does not match.

[0003] For this reason, as a system capable of matching the line of sight between a person looking at an image displayed on a screen and the person displayed in the image, a system has been developed in which a camera is arranged behind a transparent display and a subject can be photographed through the screen. In such a system, imaging of a subject is performed when no image is displayed on the transparent display (that is, when the transparent display is in a transparent state).

[0004] By the way, for Web conferences and video calls, an inexpensive rolling shutter type camera (for example, a Web camera) is often used as a camera for imaging a subject. However, there is a problem that interference fringes are likely to occur in an image captured by a rolling shutter type camera.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] The problem that this invention aims to solve is to provide a display system that can suppress the generation of interference fringes in captured images. [Means for solving the problem]

[0007] A display system according to one embodiment includes a display device having a display area that displays an image and transmits ambient light, and a light-emitting element; a rolling shutter type camera disposed behind the display device; a shutter disposed between the display device and the camera; and a control device that controls the operation of the display device and the operation of the shutter. The display device operates according to a frame period that includes a first subframe period in which the light-emitting element is lit and an image is displayed in the display area, and a second subframe period in which the light-emitting element is not lit and the display area is made transparent. The control device periodically controls the operation of the display device with a plurality of frame periods as one cycle, and controls the operation of the display device such that the plurality of frame periods included in the cycle start the second subframe period at different timings from the start of the frame period. The control device controls the operation of the shutter to block light directed toward the camera during the first subframe period and to transmit light directed toward the camera during the second subframe period.

[0008] A display system according to one embodiment includes a display device having a display area that displays an image and transmits ambient light, and light-emitting elements; a rolling shutter type camera disposed behind the display device; a shutter disposed between the display device and the camera; and a control device that controls the operation of the display device and the operation of the shutter. The display device operates according to a frame period that includes a first subframe period in which the light-emitting elements are lit and an image is displayed in the display area, and a second subframe period in which the light-emitting elements are not lit and the display area is made transparent. The first subframe period includes a red subframe period in which a red light-emitting element is lit and a red image is displayed in the display area, a green subframe period in which a green light-emitting element is lit and a green image is displayed in the display area, and a blue subframe period in which a blue light-emitting element is lit and a blue image is displayed in the display area. The display device operates according to the 1 frame period which includes the red subframe period, the green subframe period, the blue subframe period, and the second subframe period, wherein the red subframe period, the green subframe period, and the blue subframe period are set to be equal in length, and the second subframe period is set to be half the length of the red subframe period, the green subframe period, and the blue subframe period. The control device periodically controls the operation of the display device with 2 frame periods as one cycle, and controls the operation of the display device so that in the first 1 frame period included in the cycle, each subframe period starts in the order of the first subframe period, then the second subframe period, and controls the operation of the display device so that in the next 1 frame period included in the cycle, each subframe period starts in the order of the second subframe period, then the first subframe period.The control device controls the operation of the shutter to transmit light toward the camera during half the time of the red subframe period, the green subframe period, and the blue subframe period included in the first subframe period, and during the second subframe period included in the first frame period, and to block light toward the camera during the other half of the time of the red subframe period, the green subframe period, and the blue subframe period included in the first subframe period, and during the second subframe period included in the next frame period. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a diagram showing a schematic configuration example of a display system according to one embodiment. [Figure 2] Figure 2 is a schematic plan view of the display device according to the same embodiment. [Figure 3] Figure 3 is a schematic cross-sectional view of the display device according to the same embodiment. [Figure 4] Figure 4 is a schematic cross-sectional view of the shutter according to the same embodiment. [Figure 5] Figure 5 is a block diagram showing an example of the functional configuration of the control device according to the same embodiment. [Figure 6] Figure 6 is a diagram illustrating an example of the operation of the display system according to the same embodiment. [Figure 7] Figure 7 is a diagram illustrating an example of the operation of the display system according to the same embodiment. [Figure 8] Figure 8 is a diagram illustrating an example of the operation of the display system according to the same embodiment. [Figure 9] Figure 9 illustrates an example of a method for displaying an image captured by the operation shown in Figure 6 on the display device of the person on the other end of the call. [Figure 10] Figure 10 is a diagram illustrating an example of the operation of the display system related to the first comparative example. [Figure 11] Figure 11 is a diagram illustrating an example of the operation of the display system related to the second comparative example. [Figure 12] FIG. 12 is a diagram for explaining an operation example of the display system according to the first modification of the embodiment. [Figure 13] FIG. 13 is a diagram for explaining another operation example of the table system according to the first modification of the embodiment. [Figure 14] FIG. 14 is a diagram for explaining an operation example of the display system according to the second modification of the embodiment. [Figure 15] FIG. 15 is a diagram for explaining an operation example of the display system according to the third modification of the embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION

[0010] Hereinafter, embodiments will be described with reference to the drawings. Note that the disclosure is merely an example, and the invention is not limited to the contents described in the following embodiments. Modifications that can be easily conceived by those skilled in the art are naturally included in the scope of the disclosure. For the sake of clarity, in the drawings, the sizes, shapes, etc. of each part may be changed from the actual implementation mode and schematically represented. In a plurality of drawings, the same reference numerals may be assigned to corresponding elements, and detailed descriptions may be omitted.

[0011] FIG. 1 is a diagram showing a schematic configuration example of a display system 1 according to an embodiment. The display system 1 is a system used when conducting a Web conference or a video call. As shown in FIG. 1, the display system 1 includes a display device 2, a camera 3, a shutter 4, a control device 5, and a computer 6.

[0012] The display device 2 is a so-called transparent display having a display area for displaying an image and transmitting external light. The appearance (face) of the call partner U2 is displayed on the display device 2.

[0013] The camera 3 is disposed behind the display device 2. As described above, since the display device 2 is a transparent display, the camera 3 can photograph the user U1 (subject) through the display device 2. Although details will be described later, the camera 3 according to the present embodiment is a rolling shutter type camera (for example, a web camera).

[0014] The shutter 4 is an external shutter disposed between the display device 2 and the camera 3. The shutter 4 controls the exposure time of an image sensor (not shown) constituting the camera 3 by performing an opening / closing operation. The shutter 4 transmits light directed toward the camera 3 in the open state and blocks light directed toward the camera 3 in the closed state. In FIG. 1, for the sake of convenience, the camera 3 and the shutter 4 are depicted as being separated, but the camera 3 and the shutter 4 may be in close contact with each other.

[0015] The control device 5 is connected to the display device 2 and the shutter 4. The control device 5 performs display control for displaying an image on the display device 2, lighting control for switching on / off a light-emitting element described later, opening / closing control for switching the shutter 4 between the open state and the closed state, and the like.

[0016] The computer 6 is communicably connected to the computer 6' on the call partner side via the network NW. The user U1 uses a predetermined application or a web browser pre-installed on the computer 6 to conduct a web conference or a video call with the call partner U2. The computer 6 transmits an image (video data) captured by the camera 3 to the computer 6' on the call partner side. Further, the computer 6 receives an image (video data) sent from the computer 6' on the call partner side and outputs the image to the control device 5.

[0017] According to the display system 1 shown in Figure 1, user U1 can see the face of the person they are talking to, displayed on the display device 2, while also being able to see the camera 3 positioned behind the display device 2. As a result, camera 3 can capture images of user U1 looking directly at the camera. Consequently, user U1 looking directly at the camera is displayed on the display device (not shown) on the other party's side, allowing the other party U2 to converse while making eye contact with user U1 displayed on their display device. By introducing the display system 1 shown in Figure 1 on both the user's and the other party's sides, both parties can converse while making eye contact, creating a natural conversation environment.

[0018] Figure 2 is a plan view showing a schematic configuration example of the display device 2 shown in Figure 1. As shown in Figure 2, the first direction X and the second direction Y intersect each other, and the third direction Z intersects the first direction X and the second direction Y. The first direction X corresponds to the row direction, and the second direction Y corresponds to the column direction. In one example, the first direction X, the second direction Y, and the third direction Z are orthogonal to each other, but they may intersect each other at angles other than 90 degrees. In this specification, the direction toward the tip of the arrow indicating the third direction Z is referred to as upward (or simply up) or forward, and the direction opposite to the tip of the arrow is referred to as downward (or simply down) or backward.

[0019] The display device 2 comprises a display panel PNL, wiring boards F1, F2, F3, etc. The display panel PNL has a display area DA for displaying images and a frame-shaped peripheral area SA surrounding the display area DA. The display area DA is provided with n gate lines G (G1 to Gn) and m source lines S (S1 to Sm). Note that n and m are both positive integers, and n may be equal to m, or n may be different from m. The multiple gate lines G each extend in a first direction X and are arranged at intervals in a second direction Y. In other words, the multiple gate lines G extend in the row direction. The multiple source lines S each extend in a second direction Y and are arranged at intervals in the first direction X. In other words, the multiple source lines S extend in the column direction. The display panel PNL has ends E1 and E2 along the first direction X and ends E3 and E4 along the second direction Y.

[0020] The wiring board F1 includes a gate driver GD. Multiple gate lines G are connected to the gate driver GD. The wiring board F2 includes a source driver SD. Multiple source lines S are connected to the source driver SD. Wiring boards F1 and F2 are connected to the display panel PNL and wiring board F3, respectively. Wiring board F3 is electrically connected to wiring boards F1 and F2, and the control device 5 shown in Figure 1, etc. The gate driver GD and source driver SD operate according to the control signals from the control device 5 shown in Figure 1. Note that wiring boards F1 and F2 may be provided as a single wiring board. Alternatively, wiring boards F1, F2, and F3 may be provided as a single wiring board.

[0021] Figure 3 is a cross-sectional view of the display device 2 shown in Figure 2. Here, only the main parts of the cross-section of the display device 2 in the YZ plane defined by the second direction Y and the third direction Z will be described. As shown in Figure 3, the display panel PNL comprises a first substrate SUB11, a second substrate SUB21, a liquid crystal layer LC1 as a display function layer, and the like.

[0022] The first substrate SUB11 comprises a transparent substrate 10, a plurality of pixel electrodes 11, an alignment film 12, etc. The second substrate SUB21 comprises a transparent substrate 20, a common electrode 21, an alignment film 22, etc. The plurality of pixel electrodes 11 and the common electrode 21 are formed from a transparent conductive material such as ITO or IZO and are located in the display area DA. The alignment film 12 and the alignment film 22 are in contact with the liquid crystal layer LC1, respectively.

[0023] The liquid crystal layer LC1 is located at least in the display area DA. The liquid crystal layer LC1 contains polymer-dispersed liquid crystal and is held between the first substrate SUB11 and the second substrate SUB21. In this embodiment, it is assumed that the liquid crystal layer LC1 contains a normal type polymer-dispersed liquid crystal. Therefore, the liquid crystal layer LC1 is in a scattering state when no electric field is acting on it, and in a transparent state when an electric field is acting on it. The scattering state is a state in which light incident on the liquid crystal layer LC1 is scattered within the liquid crystal layer LC1. The transparent state is a state in which light incident on the liquid crystal layer LC1 is transmitted through the liquid crystal layer LC1 with almost no scattering. The display panel PNL can display an image in the display area DA when the liquid crystal layer LC1 is in a scattering state, and can transmit ambient light when the liquid crystal layer LC1 is in a transparent state.

[0024] The first substrate SUB11 and the second substrate SUB21 are bonded together by a sealing material SE1. The first substrate SUB11 has an extension Ex that extends in the second direction Y beyond the edge E5 of the transparent substrate 20. The wiring boards F1 and F2 are connected to the extension Ex of the first board SUB11.

[0025] The light source unit LU is located in the peripheral region SA outside the display area DA. The light source unit LU includes a light-emitting element LS and a wiring board F4, etc. The light-emitting element LS is connected to the wiring board F4 and is located on the extension Ex. The light-emitting element LS has a light-emitting part (light-emitting surface) EM facing the end E5. Illumination light emitted from the light-emitting part EM enters the transparent substrate 20 from the end E5, propagates within the transparent substrate 20, and enters the liquid crystal layer LC1. The wiring board F4 is electrically connected to the control device 5 shown in Figure 1, etc. The light-emitting element LS operates according to the control signal from the control device 5 shown in Figure 1, and its illumination / de-illumination is controlled.

[0026] The light source unit LU according to this embodiment includes multiple colored light-emitting elements LS. Specifically, the light source unit LU includes a light-emitting element LSR that emits red light, a light-emitting element LSG that emits green light, and a light-emitting element LSB that emits blue light. As will be described in detail later, the display device 2 according to this embodiment is driven by a field sequential method in which one frame period includes multiple subframe periods. Therefore, in each subframe, at least one of the above-mentioned light-emitting elements LSR, LSG, and LSB is lit, and its color is switched for each subframe.

[0027] Figure 4 is a cross-sectional view showing a schematic configuration example of the shutter 4 shown in Figure 1. The shutter 4 according to this embodiment is a so-called liquid crystal shutter, and as shown in Figure 4, it comprises a first substrate SUB12, a second substrate SUB22, a liquid crystal layer LC2 that functions as a shutter, a first polarizing plate PL1, a second polarizing plate PL2, and the like. The first substrate SUB12 and the second substrate SUB22 are bonded together by a sealing material SE2.

[0028] The first substrate SUB12 comprises a transparent substrate 30, a first electrode 31, an alignment film 32, etc. The second substrate SUB22 comprises a transparent substrate 40, a second electrode 41, an alignment film 42, etc. The first electrode 31 and the second electrode 41 are formed from a transparent conductive material such as ITO or IZO, and are positioned to overlap with the liquid crystal layer LC2. A voltage equal to the same potential as the reference potential is applied to the second electrode 41, and a voltage equal to or higher than the reference potential, or lower than the reference potential, is applied to the first electrode 31. The alignment film 32 and the alignment film 42 are in contact with the liquid crystal layer LC2, respectively.

[0029] In the liquid crystal layer LC2, TN (Twisted Nematic) liquid crystal is used, and the initial orientation direction of the liquid crystal molecules on the first substrate SUB12 side is 90 degrees from the initial orientation direction of the liquid crystal molecules on the second substrate SUB22 side.

[0030] The shutter 4 operates in a normally white manner, where the orientation of the liquid crystal molecules changes when an electric field acts on the liquid crystal layer LC2, resulting in a closed state (black display), and the orientation of the liquid crystal molecules does not change when no electric field acts on the liquid crystal layer LC2, resulting in an open state (white display). However, the shutter 4 may also operate in a normally black manner, where the orientation of the liquid crystal molecules changes when an electric field acts on the liquid crystal layer LC2, resulting in an open state (white display), and the orientation of the liquid crystal molecules does not change when no electric field acts on the liquid crystal layer LC2, resulting in a closed state (black display).

[0031] The first polarizing plate PL1 is positioned on the lower surface of the first substrate SUB12. The second polarizing plate PL2 is positioned on the upper surface of the second substrate SUB22. The polarization axis of the first polarizing plate PL1 and the polarization axis of the second polarizing plate PL2 are, for example, in a crossed nicol relationship, i.e., 90 degrees. With this configuration, the shutter 4 can transmit light incident from the first substrate SUB12 side when it is open, and block light incident from the first substrate SUB12 side when it is closed. Note that if the light that passes through the display device 2 and heads toward the shutter 4 is polarized light with a polarization axis in a certain direction, the shutter 4 may have a configuration that includes only the second polarizing plate PL2.

[0032] The shutter 4, having the configuration shown in Figure 4, functions as a liquid crystal shutter and controls the exposure time of the image sensor that constitutes the camera 3. In this embodiment, it is assumed that the shutter 4 is a liquid crystal shutter, but it is not limited to this, and the shutter 4 may be a physical shutter. However, considering the sound of operation when the shutter opens and closes, it is preferable for the shutter 4 to be a liquid crystal shutter.

[0033] Figure 5 is a block diagram showing an example of the functional configuration of the control device 5 shown in Figure 1. As shown in Figure 5, the control device 5 includes a control unit 50, a display control unit 51, a lighting control unit 52, a shutter control unit 53, and the like.

[0034] The control unit 50 controls the operation of each of the parts 51 to 53. When the control unit 50 receives video data output from the computer 6 (i.e., video data showing the appearance of the caller U2 as shown in Figure 1), it outputs the video data to the display control unit 51.

[0035] When the display control unit 51 receives video data output from the control unit 50, it outputs to the display device 2 the red (R), green (G), and blue (B) video data contained in the video data, as well as a control signal for controlling the timing of displaying the image based on this video data. The lighting control unit 52 outputs to the display device 2 a control signal for controlling (switching) the lighting / non-lighting of the light-emitting element LS provided on the display device 2. The shutter control unit 53 outputs to the shutter 4 a control signal for controlling (switching) the open / closed state of the shutter 4, and a control signal for controlling the polarity of the shutter 4 (more specifically, the polarity of the shutter 4 in the closed state).

[0036] Figure 6 is a diagram illustrating an example of the operation of the display system 1 according to this embodiment. In Figure 6, for example, it is assumed that the refresh rate of the display device 2 and the frame rate of the camera 3 are the same. In other words, it is assumed that the one-frame period FPd of the display device 2 and the one-frame period FPc of the camera 3 are the same (synchronized). Note that the one-frame period FPc of the camera 3 may be rephrased as the exposure period of the camera 3. Furthermore, in Figure 6, it is assumed that the display system 1 according to this embodiment operates periodically with a period of 4 frames of the display device 2 (or 4 frames of the camera 3) as one period.

[0037] As shown in Figure 6, camera 3 operates similarly during the first frame period FPc1, the second frame period FPc2, the third frame period FPc3, and the fourth frame period FPc4. Therefore, below, only the operation of camera 3 during the first frame period FPc1 will be described, and the operation of camera 3 during the other frame periods FPc2 to FPc4 will be omitted.

[0038] The first frame period FPc1 of camera 3 includes an exposure period EP and a blanking period BP. When the exposure period EP starts at the beginning of the first frame period FPc1, camera 3 sequentially begins exposure from the first pixel row to the last pixel row of the image sensor. The exposure period EP is the period from when the exposure of the first pixel row of the image sensor starts until the exposure of the last pixel row is completed. As described above, camera 3 in this embodiment is a rolling shutter type camera, and exposure is performed sequentially for each pixel row, so the exposure start time and exposure end time of each pixel row are different from each other. The blanking period BP is the period from when the exposure of the last pixel row of the image sensor is completed until the next frame period (in this case, the second frame period FPc2) starts. In this embodiment, the rolling shutter type camera is provided with a readout period (or reset period) in which data for each pixel is read out after exposure for a predetermined time for each pixel row. The readout period is significantly shorter than the exposure period of each pixel row. In this embodiment, the exposure period of each pixel row can be considered to include the readout period. Alternatively, as shown in Figure 6, if there is some leeway between the preceding and succeeding exposure periods, the exposure period itself may be considered the exposure period.

[0039] On the other hand, the display device 2 performs display operations with a cycle consisting of four frame periods: the first frame period FPd1, the second frame period FPd2, the third frame period FPd3, and the fourth frame period FPd4. Each frame period FPd1 to FPd4 includes a red subframe period SFR, a green subframe period SFG, a blue subframe period SFB, and a transparent subframe period SFT, respectively. Figure 6 shows the case where the lengths of each subframe period SFR, SFG, SFB, and SFT are equal.

[0040] The red subframe period (SFR) is the period during which red video data is written to each pixel PX located in the display area DA, and the red light-emitting element (LSR) is lit, thereby displaying a red image in the display area DA. The green subframe period (SFG) is the period during which green video data is written to each pixel PX located in the display area DA, and the green light-emitting element (LSG) is lit, thereby displaying a green image in the display area DA. The blue subframe period (SFB) is the period during which blue video data is written to each pixel PX located in the display area DA, and the blue light-emitting element (LSB) is lit, thereby displaying a blue image in the display area DA. The transparent subframe period (SFT) is the period during which a transparent voltage, which is at the same potential as the voltage applied to the common electrode (CE), is written to each pixel PX located in the display area DA, making the liquid crystal layer (LC1) (display device 2) transparent. Therefore, during the transparent subframe period (SFT), none of the light-emitting elements of any color are lit (i.e., the light-emitting elements are in a non-lit state).

[0041] In the first frame period FPd1, the subframe periods SFR, SFG, SFB, and SFT are arranged in the following order, as shown in Figure 6: transparent subframe period SFT, red subframe period SFR, green subframe period SFG, and blue subframe period SFB.

[0042] When the transparent subframe period SFT, which is the first subframe period included in the first frame period FPd1, begins, the display device 2 writes a transparent voltage to each pixel PX located in the display area DA according to a control signal from the display control unit 51 in the control device 5, making the display device 2 transparent. At the same time, the shutter 4 opens according to a control signal from the shutter control unit 53 in the control device 5. As a result, since the display device 2 is transparent and the shutter 4 is open, the camera 3 can capture an image of a subject (user U1 shown in Figure 1) located in front of the display device 2 through the display device 2.

[0043] When the transparent subframe period SFT included in the first frame period FPd1 ends and the red subframe period SFR begins, the display device 2 writes red video data to each pixel PX located in the display area DA according to a control signal from the display control unit 51 in the control device 5, and lights up the red light-emitting element LSR according to a control signal from the lighting control unit 52 in the control device 5. As a result, a red image is displayed on the display device 2. The shutter 4 transitions from an open state to a closed state according to a control signal from the shutter control unit 53 in the control device 5. More specifically, the shutter 4 transitions from an open state to a closed state by applying a voltage higher than the reference potential to the first electrode 31 shown in Figure 4, for example. As a result, the shutter 4 can block light directed toward the camera 3. Here, as described above, the shutter 4 is transitioned to a closed state by applying a voltage higher than the reference potential to the first electrode 31, so the polarity of the shutter 4 is positive (+ polarity).

[0044] When the red subframe period SFR included in the first frame period FPd1 ends and the green subframe period SFG begins, the display device 2 writes green video data to each pixel PX located in the display area DA according to a control signal from the display control unit 51 in the control device 5, and lights up the green light-emitting element LSG according to a control signal from the lighting control unit 52 in the control device 5. As a result, a green image is displayed on the display device 2. The shutter 4 maintains its closed state according to a control signal from the shutter control unit 53 in the control device 5. As a result, the shutter 4 can block light directed toward the camera 3, similar to the red subframe period SFR. In addition, since a voltage higher than the reference potential is continuously applied to the first electrode 31 of the shutter 4 in order to maintain the closed state, the polarity of the shutter 4 is positive, similar to the red subframe period SFR.

[0045] When the green subframe period SFG, included in the first frame period FPd1, ends and the blue subframe period SFB begins, the display device 2 writes blue video data to each pixel PX located in the display area DA according to a control signal from the display control unit 51 in the control device 5, and lights up the blue light-emitting element LSB according to a control signal from the lighting control unit 52 in the control device 5. As a result, a blue image is displayed on the display device 2. The shutter 4 maintains its closed state according to a control signal from the shutter control unit 53 in the control device 5. As a result, the shutter 4 can block light directed towards the camera 3, similar to the red subframe period SFR and the green subframe period SFG. In addition, since a voltage higher than the reference potential is continuously applied to the first electrode 31 of the shutter 4 in order to maintain the closed state, the polarity of the shutter 4 exhibits positive polarity, similar to the red subframe period SFR and the green subframe period SFG.

[0046] When the first frame period FPc1 begins, camera 3 starts exposure sequentially from the first pixel row to the last pixel row of the image sensor, as described above. Light enters camera 3 only during the transparent subframe period SFT included in the first frame period FPd1 of the display device 2. Therefore, camera 3 captures image I1 as the image corresponding to the pixel rows (pixel rows with diagonal lines) exposed during the transparent subframe period SFT described above.

[0047] The brightness of the image captured by camera 3 changes depending on the amount of light incident on each pixel row. Therefore, as shown in Figure 6, the image I1 captured during the first frame period FPc1 is brighter in the upper region of the image corresponding to pixel rows with longer exposure times, and darker as it approaches the middle region of the image corresponding to pixel rows with shorter exposure times. Since no light is incident on the pixel rows corresponding to the lower region of the image (i.e., pixel rows whose exposure started after the transparent subframe period SFT), nothing is visible in the lower region of the image, as shown in Figure 6, and it appears black.

[0048] In the second frame period FPd2, the subframe periods SFR, SFG, SFB, and SFT are arranged in the following order, as shown in Figure 6: red subframe period SFR, green subframe period SFG, transparent subframe period SFT, and blue subframe period SFB. The operation of the display device 2 in each subframe period SFR, SFG, SFB, and SFT is the same as in the first frame period FPd1, so a detailed explanation is omitted here. Below, only the operation of the shutter 4 and the image I2 captured by the camera 3 will be explained.

[0049] When the green subframe period SFG included in the second frame period FPd2 of the display device 2 ends and the transparent subframe period SFT begins, the shutter 4 transitions from a closed state to an open state according to a control signal from the shutter control unit 53 in the control device 5. More specifically, the shutter 4 transitions from a closed state to an open state by applying a voltage to the first electrode 31 that is the same potential as the reference potential.

[0050] Camera 3 receives light only during the transparent subframe period (SFT). Therefore, camera 3 captures image I2 as the image corresponding to the pixel rows (pixel rows with diagonal lines) that were exposed during the transparent subframe SFT included in the second frame period (FPd2). As shown in Figure 6, the lower region of image I2 captured during the second frame period (FPc2) is brighter as it approaches the middle region of the image, which corresponds to pixel rows with longer exposure times. Note that no light is incident on the pixel rows corresponding to the upper region of the image (i.e., pixel rows whose exposure was completed before the transparent subframe period (SFT)). As a result, as shown in Figure 6, nothing is visible in the upper region of image I2, and it appears black.

[0051] When the transparent subframe period SFT included in the second frame period FPd2 ends and the blue subframe period SFB begins, the shutter 4 transitions from the open state to the closed state according to a control signal from the shutter control unit 53 in the control device 5. More specifically, the shutter 4 transitions from the open state to the closed state by applying a voltage lower than the reference potential to the first electrode 31. Here, as described above, the shutter 4 is transitioned to the closed state by applying a voltage lower than the reference potential to the first electrode 31, so the polarity of the shutter 4 becomes negative (-polarity).

[0052] In the third frame period FPd3, the subframe periods SFR, SFG, SFB, and SFT are arranged in the following order, as shown in Figure 6: red subframe period SFR, green subframe period SFG, blue subframe period SFB, and transparent subframe period SFT. The operation of the display device 2 in each subframe period SFR, SFG, SFB, and SFT is the same as in the first frame period FPd1, so a detailed explanation is omitted here. Below, only the operation of the shutter 4 and the image I3 captured by the camera 3 will be explained.

[0053] When the blue subframe period SFB included in the third frame period FPd3 of the display device 2 ends and the transparent subframe period SFT begins, the shutter 4 transitions from a closed state to an open state according to a control signal from the shutter control unit 53 in the control device 5. More specifically, the shutter 4 transitions from a closed state to an open state by applying a voltage to the first electrode 31 that is the same potential as the reference potential.

[0054] Camera 3 is only exposed to light during the transparent subframe period (SFT). Therefore, camera 3 captures image I3 as the image corresponding to the pixel rows (shaded pixel rows) exposed during the transparent subframe SFT included in the third frame period (FPd3). As shown in Figure 6, during the transparent subframe period (SFT) included in the third frame period (FPd3), light is not incident on any pixel rows other than those corresponding to the lower region of the image. Therefore, in image I3, the subject is only visible in the lower region of the image, and nothing is visible in the other regions, resulting in black.

[0055] In the fourth frame period FPd4, the subframe periods SFR, SFG, SFB, and SFT are arranged in the following order, as shown in Figure 6: red subframe period SFR, transparent subframe period SFT, green subframe period SFG, and blue subframe period SFB. The operation of the display device 2 in each subframe period SFR, SFG, SFB, and SFT is the same as in the first frame period FPd1, so a detailed explanation is omitted here. Below, only the operation of the shutter 4 and the image I4 captured by the camera 3 will be explained.

[0056] When the red subframe period SFR, which is the first subframe period included in the fourth frame period FPd4, begins, the shutter 4 transitions from the open state to the closed state according to a control signal from the shutter control unit 53 in the control device 5. More specifically, the shutter 4 transitions from the open state to the closed state by applying a voltage higher than the reference potential to the first electrode 31. Here, as described above, the shutter 4 is transitioned to the closed state by applying a voltage higher than the reference potential to the first electrode 31, so the polarity of the shutter 4 becomes positive.

[0057] When the red subframe period SFR included in the fourth frame period FPd4 ends and the transparent subframe period SFT begins, the shutter 4 transitions from a closed state to an open state according to a control signal from the shutter control unit 53 in the control device 5. More specifically, the shutter 4 transitions from a closed state to an open state by applying a voltage to the first electrode 31 that is the same potential as the reference potential.

[0058] Camera 3 is only exposed to light during the transparent subframe period (SFT). Therefore, camera 3 captures image I4 as the image corresponding to the pixel rows (shaded pixel rows) exposed during the transparent subframe SFT included in the fourth frame period (FPd4). As shown in Figure 6, the image I4 captured in the fourth frame period (FPc4) is brighter in the middle region of the image corresponding to pixel rows with longer exposure times, and darker in the upper and lower regions of the image corresponding to pixel rows with shorter exposure times.

[0059] When the transparent subframe period SFT included in the fourth frame period FPd4 ends and the green subframe period SFG begins, the shutter 4 transitions from the open state to the closed state according to a control signal from the shutter control unit 53 in the control device 5. More specifically, the shutter 4 transitions from the open state to the closed state by applying a voltage lower than the reference potential to the first electrode 31. Here, as described above, the shutter 4 is transitioned to the closed state by applying a voltage lower than the reference potential to the first electrode 31, so the polarity of the shutter 4 becomes negative.

[0060] According to the operation shown in Figure 6 described above, in the four transparent subframe periods SFT included in the four frame periods FPd1 to FPd4 of the display device 2, each pixel row of the image sensor constituting the camera 3 is exposed for the duration of one frame period FPc of the camera 3 (in other words, the time equivalent to the difference between the exposure start time and the exposure end time). That is, each pixel row of the image sensor constituting the camera 3 is exposed for the same amount of time as the others. For this reason, as will be described in more detail later, the image IA that can be displayed based on images I1 to I4 can be made into an image with uniform brightness from the upper region to the lower region of the image (in short, an image with uniform brightness throughout the entire image region), as shown in Figure 6.

[0061] Furthermore, in the operation shown in Figure 6 described above, the polarity of the shutter 4 is controlled to reverse each time the shutter 4 is transitioned to the closed state, compared to the polarity at the time of the previous transition to the closed state. As a result, as shown in Figure 6, it is possible to match the time when the shutter 4 exhibits positive polarity with the time when the shutter 4 exhibits negative polarity in one cycle, thereby suppressing the occurrence of so-called burn-in.

[0062] In Figure 6, the control is shown to reverse the polarity of the shutter 4 each time it is transitioned to the closed state in order to suppress burn-in. However, the control is not limited to this, and burn-in can also be suppressed by, for example, reversing the polarity of the shutter 4 every cycle, as shown in Figure 7 (specifically, in the first cycle, all shutters 4 in the closed state are set to positive polarity, and in the next cycle, all shutters 4 in the closed state are set to negative polarity).

[0063] Furthermore, while Figure 6 illustrates the operation when the FPd of the display device 2 and the FPc of the camera 3 are the same (i.e., when the display device 2 and the camera 3 operate synchronously), the system is not limited to this. The display system 1 according to this embodiment can also operate similarly when the FPd of the display device 2 and the FPd of the camera 3 are not the same, for example, when the FPc of the camera 3 is slightly longer than the FPd of the display device 2, as shown in Figure 8. In this case, although it is not possible to expose each pixel row of the image sensor constituting the camera 3 for the duration of one frame period FPc of the camera 3 during the transparent subframe period SFT included in the four frame periods FPd1 to FPd4 of the display device 2, if the difference between the one frame period FPd of the display device 2 and the one frame period FPc of the camera 3 is around a few percent, it is possible to suppress the difference in exposure time for each pixel row to an extent that does not affect image quality, and the image IA' that can be displayed based on images I1' to I4' can be an image with almost uniform brightness from the upper region to the lower region of the image (in short, an image with almost uniform brightness throughout the entire image region).

[0064] Figure 9 illustrates an example of a method for displaying image IA, based on images I1 to I4 captured by the operation shown in Figure 6, on a display device (not shown) on the other party's side of the call. Images I1 to I4 captured by the operation shown in Figure 6 are sequentially transmitted by computer 6 to the other party's computer 6', for example, as shown in Figure 9(a). Images I1 to I4 received by computer 6' are sequentially displayed on the other party's display device over a period of, for example, four frames. Generally, people cannot directly perceive an image displayed in a single frame; they perceive an image displayed over multiple frames as a single image. Therefore, by sequentially transmitting images I1 to I4 to the other party's computer 6' and sequentially displaying images I1 to I4 on the other party's display device, the other party can perceive an image IA, which is a combination of images I1 to I4. This makes it possible to provide the other party with an image of uniform brightness (in this case, image IA).

[0065] Alternatively, the images I1 to I4 captured by the operation shown in Figure 6 may be combined by computer 6 as shown in Figure 9(b) and then transmitted to the other party's computer 6'. This allows the image IA obtained by combining images I1 to I4 to be transmitted to the other party's computer 6', making it possible to provide the other party with an image of uniform brightness (in this case, image IA) without using visual effects.

[0066] The effects of the display system 1 according to this embodiment will be explained below using comparative examples. Note that the comparative examples are intended to illustrate some of the effects that the display system 1 according to this embodiment may achieve, and do not exclude any configurations or effects common to this embodiment and the comparative examples from the scope of the present invention.

[0067] Figure 10 is a diagram illustrating a schematic configuration example of a display system according to the first comparative example and the operation of the display system. For the sake of explanation, elements relating to the first comparative example will be denoted with "A" below. The display system 1A according to the first comparative example differs from the display system 1 according to this embodiment in that, as shown in Figure 10(a), the camera 3A is a global shutter type camera and its operation is controlled by the control device 5A, and the configuration corresponding to the shutter 4 is omitted.

[0068] Unlike a rolling shutter camera (i.e., camera 3 in this embodiment), which sequentially exposes each pixel row of the image sensor, the global shutter camera 3A can expose all pixel rows of the image sensor at once. For this reason, the control device 5A controls the operation of camera 3A in synchronization with the operation of display device 2A, and controls the opening of the shutter built into camera 3A at the timing when the transparent subframe period SFT included in one frame period FPd of display device 2A starts and the display device 2A becomes transparent, as shown in Figure 10(b). Note that the operation of display device 2A in each subframe period SFR, SFG, SFB, and SFT included in one frame period FPd of display device 2A is the same as that of display device 2 in this embodiment, so a detailed explanation is omitted here.

[0069] According to the display system 1A of the first comparative example, the camera 3A can equally expose all pixel rows of the image sensor during one transparent subframe period SFT, and then read out the data of each pixel during the subsequent readout period. Therefore, the camera 3A can capture an image with uniform brightness from the upper region to the lower region of the image during one frame period FPc.

[0070] On the other hand, the global shutter camera 3A is very expensive, and using it for web conferencing or video calls is not practical. Furthermore, as mentioned above, using the global shutter camera 3A requires synchronizing its operation with the display device 2A, and a special function must be implemented in the control device 5A to control the operation of the camera 3A in synchronization with the operation of the display device 2A. As described above, the display system 1A according to the first comparative example has the problem of requiring a great deal of cost to realize.

[0071] In contrast, the display system 1 according to this embodiment uses a rolling shutter camera 3, which is less expensive than the global shutter camera 3A. Furthermore, since the display system 1 according to this embodiment does not require synchronization of the operation of the display device 2 and the camera 3, it is not necessary to implement the special functions described above in the control device 5. For this reason, the display system 1 according to this embodiment can be realized without requiring the significant costs associated with the display system 1A in the first comparative example.

[0072] Figure 11 is a diagram illustrating the operation of the display system according to the second comparative example. For convenience of explanation, elements relating to the second comparative example will be denoted with "B" in the following description. The display system 1B according to the second comparative example has the same configuration as the display system 1 according to the present embodiment. On the other hand, as shown in Figure 11, the display system 1B according to the second comparative example differs from the display system 1 according to the present embodiment in that the 1 frame period FPd of the display device 2B is composed of a red subframe period SFR, a green subframe period SFG, a blue subframe period SFB, and a transparent subframe period SFT inserted between each of the subframe periods SFR, SFG, and SFB.

[0073] When the red subframe period SFR, which is the first subframe period included in the FPd frame period of the display device 2B, begins, the display device 2B writes red video data to each pixel placed in the display area and lights up the red light-emitting element. After a predetermined time has elapsed since lighting up the red light-emitting element, the display device 2B switches the red light-emitting element to an unlit state. Subsequently, when the transparent subframe period SFT, which is inserted between the red subframe period SFR and the green subframe period SFG described later, begins, the display device 2B writes a transparent voltage to each pixel placed in the display area and makes the display device 2B transparent. When the transparent subframe period SFT begins, the shutter 4B transitions from a closed state to an open state according to a control signal from the control device 5B.

[0074] When the transparent subframe period SFT, which follows the red subframe period SFR, ends and the green subframe period SFG begins, the display device 2B writes green video data to each pixel placed in the display area and lights up the green light-emitting element. After a predetermined time has elapsed since lighting up the green light-emitting element, the display device 2B switches the green light-emitting element to an unlit state. Subsequently, when the transparent subframe period SFT, which is inserted between the green subframe period SFG and the blue subframe period SFB described later, begins, the display device 2B writes a transparency voltage to each pixel placed in the display area and makes the display device 2B transparent. When the transparent subframe period SFT begins, the shutter 4B transitions from a closed state to an open state according to a control signal from the control device 5B.

[0075] When the transparent subframe period SFT, which follows the green subframe period SFG, ends and the blue subframe period SFB begins, the display device 2B writes blue video data to each pixel placed in the display area and lights up the blue light-emitting element. After a predetermined time has elapsed since lighting up the blue light-emitting element, the display device 2B switches the blue light-emitting element to an unlit state. Subsequently, when the transparent subframe period SFT, which is inserted between the blue subframe period SFB and the red subframe period SFR included in the next 1 frame period FPd, begins, the display device 2B writes a transparency voltage to each pixel placed in the display area and makes the display device 2B transparent. When the transparent subframe period SFT begins, the shutter 4B transitions from a closed state to an open state according to a control signal from the control device 5B.

[0076] When a frame period FPc begins, camera 3B sequentially starts exposure from the first pixel row to the last pixel row of the image sensor, capturing images corresponding to the pixel rows exposed during the transparent subframe period SFT, which is inserted between each subframe period SFR, SFG, and SFB. In other words, as shown in Figure 11, camera 3B captures images corresponding to the pixel rows exposed during the transparent subframe period SFT, which is inserted between the red subframe period SFR and the green subframe period SFG (pixel rows with diagonal lines), and the pixel rows exposed during the transparent subframe period SFT, which is inserted between the green subframe period SFG and the blue subframe period SFB (pixel rows with diagonal lines).

[0077] During the transparent subframe period SFT, which is inserted between the red subframe period SFR and the green subframe period SFG, the upper to middle row of pixels in the image sensor constituting camera 3B is exposed. During the transparent subframe period SFT, which is inserted between the green subframe period SFG and the blue subframe period SFB, the middle to lower row of pixels in the image sensor constituting camera 3B is exposed. As a result, as shown in Figure 11, the exposure time of the middle row of pixels (the pixel rows enclosed by the dashed line) in the image sensor constituting camera 3B becomes longer than the exposure time of the upper and lower row of pixels. Consequently, in the camera 3B included in the display system 1B according to the second comparative example, an image is captured in which the middle region of the image is brighter than other regions. In other words, interference fringes are generated in the captured image.

[0078] In contrast, the display system 1 according to this embodiment exposes each pixel row of the image sensor constituting the camera 3 for the duration of one frame period FPc of the camera 3 during the four transparent subframe periods SFT included in the four frame periods FPd1 to FPd4 of the display device 2, thereby equalizing the exposure time for each pixel row. This suppresses the generation of interference fringes and makes it possible to provide an image with uniform brightness from the upper to the lower regions of the image (in short, an image with uniform brightness throughout the entire image area).

[0079] The following describes some modified examples of the display system 1 according to this embodiment. (First variation) Figure 12 is a diagram illustrating the operation of the display system 1 according to the first modified example. The operation of the display system 1 according to the first modified example differs from the operation of the display system 1 shown in Figure 6, in that the FPc of one frame period of the camera 3 corresponds to the exposure period of each pixel row, as shown in Figure 12. Note that the operation of the display device 2 and the operation of the shutter 4 are the same as those shown in Figure 6, so a detailed explanation of them is omitted here. In the following, only the operation of the camera 3 and the image captured by the camera 3 will be described.

[0080] When a frame period FPc begins, camera 3 sequentially starts exposure from the first pixel row to the last pixel row of the image sensor. The exposure time for each pixel row is equal to the frame period FPc. For example, when exposure of a predetermined pixel row for capturing a predetermined image is completed and the readout period for that pixel row is finished, exposure of the same predetermined pixel row for capturing the next image is immediately started.

[0081] Camera 3 captures image I11 as an image corresponding to the pixel rows (pixel rows with a downward-sloping line on the left) exposed during the transparent subframe period SFT included in the first frame period FPd1 of the display device 2, and the pixel rows (pixel rows with a downward-sloping line on the left) exposed during the transparent subframe period SFT included in the second frame period FPd2 of the display device 2.

[0082] Furthermore, camera 3 captures image I12 as an image corresponding to the pixel rows exposed during the transparent subframe period SFT included in the second frame period FPd2 of display device 2 (pixel rows with a thick line sloping downwards to the left) and the pixel rows exposed during the transparent subframe period SFT included in the third frame period FPd3 of display device 2 (pixel rows with a thick line sloping downwards to the left).

[0083] Furthermore, camera 3 captures image I13 as an image corresponding to the pixel rows exposed during the transparent subframe period SFT included in the third frame period FPd3 of display device 2 (pixel rows with downward-sloping lines), and the pixel rows exposed during the transparent subframe period SFT included in the fourth frame period FPd4 of display device 2 (pixel rows with downward-sloping lines).

[0084] In the operation shown in Figure 12 described above, similar to the operation shown in Figure 6, in the four transparent subframe periods SFT included in the four frame periods FPd1 to FPd4 of the display device 2, each pixel row of the image sensor constituting the camera 3 is exposed for the duration of one frame period FPc of the camera 3, making the exposure time of each pixel row equal. As a result, the image IB that can be displayed based on images I11 to I13 can be made into an image with uniform brightness from the upper to the lower region of the image (in short, an image with uniform brightness throughout the entire image region), as shown in Figure 12, and the same effect as the operation shown in Figure 6 can be obtained.

[0085] As shown in Figure 13, the camera 3 may further capture image I14 as an image corresponding to the pixel rows exposed during the transparent subframe period SFT included in the fourth frame period FPd4 of the display device 2 (pixel rows with thick lines sloping downwards to the right) and the pixel rows exposed during the transparent subframe period SFT included in the next first frame period FPd1 (pixel rows with thick lines sloping downwards to the right). In this case as well, since the exposure time of each pixel row of the image sensor constituting the camera 3 can be made equal, the image IC that can be displayed based on images I11 to I14 can be made into an image with uniform brightness from the upper region to the lower region of the image (in short, an image with uniform brightness throughout the entire image region), as shown in Figure 13, and it is possible to obtain the same effect as the operation shown in Figure 6. Furthermore, the image IC that can be displayed based on images I11 to I14 can be made into a brighter image than the image IB that can be displayed based on images I11 to I13 because it includes image I14.

[0086] (Second variation) Figure 14 is a diagram illustrating the operation of the display system 1 according to the second modified example. The operation of the display system 1 according to the second modified example differs from the operation of the display system 1 shown in Figure 6 in that, as shown in Figure 14, the 1-frame period FPc of the camera 3 is shorter than the 1-frame period FPd of the display device 2 (i.e., the 1-frame period FPc of the camera 3 does not coincide with the 1-frame period FPd of the display device 2), and the 4-frame period (i.e., 1 cycle) of the display device 2 coincides with the 8-frame period of the camera 3. In other words, in the second modified example, the operation of the display system 1 will be described when the time of the 4-frame period included in one cycle of the display device 2 is equal to the time obtained by multiplying the 1-frame period of the camera 3 by N (in this case, N=8). Note that the operation of the display device 2 and the operation of the shutter 4 are the same as those shown in Figure 6, so a detailed explanation of them will be omitted here. In the following, only the operation of the camera 3 and the images captured by the camera 3 will be described.

[0087] When the first frame period FPc1 begins, camera 3 sequentially starts exposure from the first pixel row to the last pixel row of the image sensor, and captures image I21 as an image corresponding to the pixel rows (pixel rows with diagonal lines) exposed during the transparent subframe period SFT included in the first frame period FPd1 of the display device 2.

[0088] Furthermore, during the fourth frame period FPc4, the camera 3 captures image I22 as an image corresponding to the pixel rows (pixel rows with diagonal lines) exposed during the transparent subframe period SFT included in the second frame period FPd2 of the display device 2.

[0089] Furthermore, during the sixth frame period FPc6, the camera 3 captures image I23 as an image corresponding to the pixel rows (pixel rows with diagonal lines) exposed during the transparent subframe period SFT included in the third frame period FPd3 of the display device 2.

[0090] Furthermore, during the seventh frame period FPc7, camera 3 captures image I24 as an image corresponding to the pixel rows (pixel rows with diagonal lines) exposed during the transparent subframe period SFT included in the fourth frame period FPd4 of display device 2.

[0091] Furthermore, during the second frame period FPc2, the third frame period FPc3, the fifth frame period FPc5, and the eighth frame period FPc8, camera 3 sequentially scans the image sensor from the first to the last pixel row. However, because the light directed towards camera 3 is blocked by the closed shutter 4, nothing is captured, and the image appears black. This black image may, for example, be discarded.

[0092] In the operation shown in Figure 14 described above, during the four transparent subframe periods SFT included in the four frame periods FPd1 to FPd4 of the display device 2, each pixel row of the image sensor constituting the camera 3 is exposed for the duration of two frames of the camera 3, making the exposure time of each pixel row equal. As a result, the image IDs that can be displayed based on images I21 to I24 can be made into an image with uniform brightness from the upper to the lower region of the image (in short, an image with uniform brightness throughout the entire image region), as shown in Figure 14, and it is possible to obtain the same effect as the operation shown in Figure 6.

[0093] (Third variation) Figure 15 is a diagram illustrating the operation of the display system 1 according to the third modified example. The operation cycle of the display system 1 according to the third modified example corresponds to the 2-frame period of the display device 2 (or the 2-frame period of the camera 3), as shown in Figure 15, and in this respect, it differs from the operation shown in Figure 6. Furthermore, the operation of the display system 1 according to the third modified example differs from the operation of the display system 1 shown in Figure 6 in that, as shown in Figure 15, the length of the transparent subframe period SFT included in the 1-frame period FPd of the display device 2 is half the length of the other subframe periods SFR, SFG, and SFB included in the 1-frame period FPd of the display device 2. In addition, the operation of the display system 1 according to the third modified example differs from the operation of the display system 1 shown in Figure 6 in that, as shown in Figure 15, the light-emitting element LS is controlled to be lit for half the time of the subframe periods SFR, SFG, and SFB included in the 1-frame period FPd of the display device 2, and to be unlit for the remaining half of the time. Furthermore, the operation of the display system 1 according to the third modified example differs from the operation of the display system 1 shown in Figure 6, in that, as shown in Figure 15, the shutter 4 is controlled to be in an open state for half the duration of each subframe period SFR, SFG, SFB (more specifically, during the time when the light-emitting element LS is not lit), and in a closed state for the remaining half duration of each subframe period SFR, SFG, SFB (more specifically, during the time when the light-emitting element LS is lit). In addition, the operation of the display system 1 according to the third modified example differs from the operation of the display system 1 shown in Figure 6, in that, as shown in Figure 15, the shutter 4 is controlled to be in an open state during the transparent subframe period SFT included in the first frame period FPd1, and in a closed state during the transparent subframe period SFT included in the second frame period FPd2. For the sake of explanation, it is assumed here that the 1-frame period FPd1 of the display device 2 and the 1-frame period FPc of the camera 3 coincide, and that the 1-frame period FPc of the camera 3 includes only the exposure period EP.

[0094] In the first frame period FPd1 of the display device 2, the subframe periods SFR, SFG, SFB, and SFT are arranged in the following order, as shown in Figure 15: red subframe period SFR, green subframe period SFG, blue subframe period SFB, and transparent subframe period SFT.

[0095] As shown in Figure 15, when the red subframe period SFR, which is the first subframe period included in the first frame period FPd1 of the display device 2, begins, the display device 2 writes red video data to each pixel PX located in the display area DA in accordance with a control signal from the display control unit 51 in the control device 5 during the first half of the red subframe period SFR, SFR1. During the first half of the red subframe period SFR, SFR1, although red video data is written to each pixel PX, the light-emitting element LS is not lit, so the display device 2 transmits light with a certain polarization component. The shutter 4 opens in accordance with a control signal from the shutter control unit 53 in the control device 5. As a result, light with a certain polarization component is emitted from the display device 2, and the shutter 4 is open, so the camera 3 can capture an image of a subject located in front of the display device 2 through the display device 2.

[0096] During the latter half of the red subframe period SFR2, which is included in the first frame period FPd1, the display device 2 lights up the red light-emitting element LSR. As a result, a red image is displayed on the display device 2. The shutter 4 transitions from an open state to a closed state according to a control signal from the shutter control unit 53 in the control device 5. More specifically, the shutter 4 transitions from an open state to a closed state by applying a voltage higher than the reference potential to the first electrode 31. As a result, the shutter 4 can block light directed toward the camera 3. Note that, as described above, since the shutter 4 is transitioned to a closed state by applying a voltage higher than the reference potential to the first electrode 31, the polarity of the shutter 4 becomes positive (+ polarity).

[0097] When the red subframe period SFR included in the first frame period FPd1 ends and the green subframe period SFG begins, the display device 2 writes green video data to each pixel PX located in the display area DA in accordance with a control signal from the display control unit 51 in the control device 5 during the first half of the green subframe period SFG, SFG1. Although green video data is written to each pixel PX during the first half of the green subframe period SFG, the light-emitting element LS is not lit, so, similar to the first half of the red subframe period SFR1, the display device 2 transmits light with a certain polarization component. The shutter 4 transitions from a closed state to an open state in accordance with a control signal from the shutter control unit 53 in the control device 5. More specifically, the shutter 4 transitions from a closed state to an open state by applying a voltage to the first electrode 31 that is the same potential as the reference potential.

[0098] During the latter half of the green subframe period SFG, which is included in the first frame period FPd1, in time SFG2, the display device 2 lights up the green light-emitting element LSG. As a result, a green image is displayed on the display device 2. The shutter 4 transitions from an open state to a closed state according to a control signal from the shutter control unit 53 in the control device 5. More specifically, the shutter 4 transitions from an open state to a closed state by applying a voltage lower than the reference potential to the first electrode 31. As a result, the shutter 4 can block light directed toward the camera 3. Note that, as described above, since the shutter 4 is transitioned to a closed state by applying a voltage lower than the reference potential to the first electrode 31, the polarity of the shutter 4 becomes negative (-polarity).

[0099] When the green subframe period SFG included in the first frame period FPd1 ends and the blue subframe period SFB begins, the display device 2 writes blue video data to each pixel PX located in the display area DA in accordance with a control signal from the display control unit 51 in the control device 5 during the first half of the blue subframe period SFB, SFB1. In the first half of the blue subframe period SFB, SFB1, although blue video data is written to each pixel PX, the light-emitting element LS is not lit, so the display device 2 transmits light with a certain polarization component, similar to the first half of the red subframe period SFR1 and the first half of the green subframe period SFG1. The shutter 4 applies a voltage equal to the reference potential to the first electrode 31 in accordance with a control signal from the shutter control unit 53 in the control device 5, causing the shutter 4 to transition from a closed state to an open state.

[0100] During the latter half of the blue subframe period SFB2, which is included in the first frame period FPd1, the display device 2 lights up the blue light-emitting element LSB. As a result, a blue image is displayed on the display device 2. The shutter 4, in accordance with the control signal from the shutter control unit 53 in the control device 5, applies a voltage higher than the reference potential to the first electrode 31, causing the shutter 4 to transition from the open state to the closed state. Here, as described above, since the shutter 4 is transitioned to the closed state by applying a voltage higher than the reference potential to the first electrode 31, the polarity of the shutter 4 becomes positive.

[0101] When the blue subframe period SFB included in the first frame period FPd1 ends and the transparent subframe period SFT begins, the display device 2 writes a transparent voltage to each pixel PX located in the display area DA according to a control signal from the display control unit 51 in the control device 5, and becomes transparent. The shutter 4 applies a voltage to the first electrode 31 that is the same potential as the reference potential according to a control signal from the shutter control unit 53 in the control device 5, and transitions the shutter 4 from the closed state to the open state.

[0102] When the first frame period FPc1 begins, camera 3 sequentially starts exposing the image sensor from the first pixel row to the last pixel row. Camera 3 captures image I41 as the image corresponding to the pixel rows (pixel rows with diagonal lines) that were exposed during the first half of the red subframe period SFR1, the first half of the green subframe period SFG1, the first half of the blue subframe period SFB1, and the transparent subframe period SFT.

[0103] Next, we will explain the second frame period FPd2 of the display device 2. In the second frame period FPd2 of the display device 2, the subframe periods SFR, SFG, SFB, and SFT are arranged in the order of transparent subframe period SFT, red subframe period SFR, green subframe period SFG, and blue subframe period SFB, as shown in Figure 15. The operation of the display device 2 in each subframe period SFR, SFG, SFB, and SFT is the same as in the first frame period FPd1, so a detailed explanation will be omitted here. Below, we will only explain the operation of the shutter 4 and the image captured by the camera 3.

[0104] As shown in Figure 15, when the transparent subframe period SFT, which is the first subframe period included in the second frame period FPd2 of the display device 2, begins, the shutter 4 applies a voltage lower than the reference potential to the first electrode 31 in accordance with the control signal from the shutter control unit 53 in the control device 5, causing the shutter 4 to transition from the open state to the closed state. Here, as described above, since the shutter 4 is transitioned to the closed state by applying a voltage lower than the reference potential to the first electrode 31, the polarity of the shutter 4 becomes negative.

[0105] When the transparent subframe period SFT included in the second frame period FPd2 ends and the red subframe period SFR begins, in the first half of the period SFR1, the shutter 4 applies a voltage equal to the reference potential to the first electrode 31 according to a control signal from the shutter control unit 53 in the control device 5, causing the shutter 4 to transition from the closed state to the open state. On the other hand, in the second half of the period SFR2, the shutter 4 applies a voltage higher than the reference potential to the first electrode 31 according to a control signal from the shutter control unit 53 in the control device 5, causing the shutter 4 to transition from the open state to the closed state. Note that, as described above, since the shutter 4 is transitioned to the closed state by applying a voltage higher than the reference potential to the first electrode 31, the polarity of the shutter 4 becomes positive.

[0106] When the red subframe period SFR, included in the second frame period FPd2, ends and the green subframe period SFG begins, in the first half of time SFG1, the shutter 4 applies a voltage equal to the reference potential to the first electrode 31 according to a control signal from the shutter control unit 53 in the control device 5, causing the shutter 4 to transition from the closed state to the open state. On the other hand, in the second half of time SFG2, the shutter 4 applies a voltage lower than the reference potential to the first electrode 31 according to a control signal from the shutter control unit 53 in the control device 5, causing the shutter 4 to transition from the open state to the closed state. Note that, as described above, since the shutter 4 is transitioned to the closed state by applying a voltage lower than the reference potential to the first electrode 31, the polarity of the shutter 4 becomes negative.

[0107] When the green subframe period SFG, included in the second frame period FPd2, ends and the blue subframe period SFB begins, in the first half of the period SFB1, the shutter 4 applies a voltage equal to the reference potential to the first electrode 31 according to a control signal from the shutter control unit 53 in the control device 5, causing the shutter 4 to transition from the closed state to the open state. On the other hand, in the second half of the period SFB2, the shutter 4 applies a voltage higher than the reference potential to the first electrode 31 according to a control signal from the shutter control unit 53 in the control device 5, causing the shutter 4 to transition from the open state to the closed state. Note that, as described above, since the shutter 4 is transitioned to the closed state by applying a voltage higher than the reference potential to the first electrode 31, the polarity of the shutter 4 becomes positive.

[0108] When the second frame period FPc2 begins, camera 3 sequentially starts exposing the image sensor from the first pixel row to the last pixel row, and captures image I42 as the image corresponding to the pixel rows (pixel rows with diagonal lines) that were exposed during the first half of the red subframe period SFR1, the first half of the green subframe period SFG1, and the first half of the blue subframe period SFB1.

[0109] As described above, according to the operation shown in Figure 15, the display device 2 can be made to transmit or be transparent to light with a certain polarization component for half the duration of the two frame periods FPd1 and FPd2, and each pixel row of the image sensor constituting the camera 3 can be exposed for the duration of one frame period FPc of the camera 3, making the exposure time of each pixel row equal. Therefore, the image IE that can be displayed based on images I41 and I42 can be made into an image with uniform brightness from the upper region to the lower region of the image (in short, an image with uniform brightness throughout the entire image region), as shown in Figure 15, and it is possible to obtain the same effect as the operation shown in Figure 6.

[0110] Furthermore, in the operation shown in Figure 15 described above, the polarity of the shutter 4 is controlled to reverse each time the shutter 4 is transitioned to the closed state, compared to the polarity at the time of the previous transition to the closed state. As a result, as shown in Figure 15, it is possible to match the time when the shutter 4 exhibits positive polarity with the time when the shutter 4 exhibits negative polarity over two cycles, and, similar to the operation shown in Figure 6, the occurrence of so-called burn-in can be suppressed.

[0111] The display system 1 according to one embodiment described above comprises a display device 2 having a display area DA that displays an image and transmits ambient light, and a light-emitting element LS; a rolling shutter type camera 3 positioned behind the display device 2; a shutter 4 positioned between the display device 2 and the camera 3; and a control device 5 that controls the operation of the display device 2 and the operation of the shutter 4. The display device 2 operates according to a 1-frame period that includes a red subframe period SFR, a green subframe period SFG, and a blue subframe period SFB in which the light-emitting element LS is lit and an image is displayed in the display area DA, and a transparent subframe period SFT in which the light-emitting element LS is not lit and the display area DA is made transparent. The control device 5 periodically controls the operation of the display device 2 with a plurality of 1-frame periods as one cycle, and controls the operation of the display device 2 so that the plurality of 1-frame periods included in one cycle start the transparent subframe period SFT at different timings from the start of the 1-frame period. The control device 5 also controls the operation of the shutter 4 so as to block light directed toward the camera 3 during the red subframe period, the green subframe period, and the blue subframe period, and so as to block light directed toward the camera 3 during the transparent subframe period SFT.

[0112] According to this, in the multiple transparent subframe periods (SFT) included in one cycle, each pixel row of the image sensor constituting the camera 3 can be exposed for the same amount of time. This suppresses the occurrence of interference fringes that may occur in images captured by the rolling shutter camera 3, making it possible to capture (provide) images with uniform brightness.

[0113] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols]

[0114] 1...Display system, 2...Display device, 3...Camera, 4...Shutter, 5...Control device, 6...Computer (user side), 6'...Computer (other party), U1...User, U2...Other party, NW...Network.

Claims

1. A display device having a display area that displays an image and transmits ambient light, and a light-emitting element, A rolling shutter type camera is positioned behind the display device, A shutter is disposed between the display device and the camera, The system includes a control device that controls the operation of the display device and the operation of the shutter, The display device operates according to a frame period that includes a first subframe period in which the light-emitting element is lit and an image is displayed in the display area, and a second subframe period in which the light-emitting element is not lit and the display area is made transparent. The control device is The operation of the display device is periodically controlled with multiple single-frame periods forming one cycle, and the operation of the display device is controlled such that the multiple single-frame periods included in the one cycle start the second sub-frame period at different timings from the start of each single-frame period. A display system that controls the operation of the shutter to block light directed towards the camera during the first subframe period and to transmit light directed towards the camera during the second subframe period.

2. The camera has an image sensor including multiple pixel rows, and operates to expose each pixel row in a time-division manner, thereby capturing an image of a subject in front of the display device. Each of the aforementioned pixel rows is exposed to each other for equal periods of time in a plurality of second subframe periods included in the one cycle. The display system according to claim 1.

3. The shutter comprises a first electrode, a second electrode facing the first electrode and at a reference potential, and a liquid crystal layer that functions as a shutter. The shutter transitions to a closed state, blocking light directed towards the camera, when a voltage of a different potential from the reference potential is applied to the first electrode, and transitions to an open state, allowing light directed towards the camera to pass through, when a voltage of the same potential as the reference potential is applied to the first electrode. The display system according to claim 1.

4. The polarity of the shutter is positive when a voltage greater than the reference potential is applied to the first electrode and it transitions to the closed state, and negative when a voltage less than the reference potential is applied to the first electrode and it transitions to the closed state. The display system according to claim 3.

5. The control device is Each time the shutter transitions from the open state to the closed state, the operation of the shutter is controlled so as to reverse the polarity of the shutter from the polarity it had when it was previously closed, thereby matching the time the shutter shows positive polarity and the time it shows negative polarity within one cycle. The display system according to claim 4.

6. The control device is Each of the aforementioned cycles, the operation of the shutter is controlled to reverse the polarity of the shutter from the polarity of the previous cycle, so that the time during which the shutter shows positive polarity and the time during which it shows negative polarity are matched over two consecutive cycles. The display system according to claim 4.

7. The system further includes a computer that acquires images captured by the aforementioned camera. The aforementioned computer is connected to other computers via a network so as to be able to communicate with them, and sequentially transmits images acquired from the camera to the other computers. The display system according to claim 1.

8. The system further includes a computer that acquires images captured by the aforementioned camera. The computer is connected to other computers via a network so as to be able to communicate with them, and when it acquires images for one cycle from the camera, it combines the acquired images for one cycle and transmits the combined image to the other computer. The display system according to claim 1.

9. When the computer receives an image transmitted from the other computer, it outputs the received image to the control device. The display system according to claim 7 or claim 8.

10. The first subframe period includes a red subframe period in which a red light-emitting element is lit and a red image is displayed in the display area, a green subframe period in which a green light-emitting element is lit and a green image is displayed in the display area, and a blue subframe period in which a blue light-emitting element is lit and a blue image is displayed in the display area. The display device operates according to the one frame period, which includes the red subframe period, the green subframe period, the blue subframe period, and the second subframe period, and the red subframe period, the green subframe period, the blue subframe period, and the second subframe period are set to be equal in length. The control device periodically controls the operation of the display device with a period of 4 frames as one cycle. The display system according to claim 1.

11. A display device having a display area that displays an image and transmits ambient light, and a light-emitting element, A rolling shutter type camera is positioned behind the display device, A shutter is disposed between the display device and the camera, The system includes a control device that controls the operation of the display device and the operation of the shutter, The display device operates according to a frame period that includes a first subframe period in which the light-emitting element is lit and an image is displayed in the display area, and a second subframe period in which the light-emitting element is not lit and the display area is made transparent. The first subframe period includes a red subframe period in which a red light-emitting element is lit and a red image is displayed in the display area, a green subframe period in which a green light-emitting element is lit and a green image is displayed in the display area, and a blue subframe period in which a blue light-emitting element is lit and a blue image is displayed in the display area. The display device operates according to the one-frame period, which includes the red subframe period, the green subframe period, the blue subframe period, and the second subframe period, wherein the red subframe period, the green subframe period, and the blue subframe period are set to be equal in length, and the second subframe period is set to be half the length of the red subframe period, the green subframe period, and the blue subframe period. The control device is The operation of the display device is periodically controlled with a two-frame period as one cycle, and in the first frame period included in the cycle, the operation of the display device is controlled so that the first subframe period and the second subframe period start in that order, and in the next frame period included in the cycle, the operation of the display device is controlled so that the second subframe period and the first subframe period start in that order. A display system that controls the operation of the shutter to transmit light toward the camera during half the time of the red subframe period, the green subframe period, and the blue subframe period included in the first subframe period, and during the second subframe period included in the first frame period, and to block light toward the camera during the other half of the time of the red subframe period, the green subframe period, and the blue subframe period included in the first subframe period, and during the second subframe period included in the next frame period.

Images