Imaging system

Abstract

To provide an imaging system capable of performing remote image display with high quality.SOLUTION: An imaging system 200 comprises: a display device 100 with a display panel P; and an imaging apparatus 300 which is disposed with the display panel P sandwiched between itself and a subject PA and captures an image of the subject PA transmitted through the display panel P. The imaging apparatus 300 acquires exposure data of a color different from a display color of the display device 100 and generates imaging data of the subject PA.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging system. [Background technology]

[0002] Remote conference systems via networks have become widespread. In such remote conference systems, typically, images of participants captured by a Web camera installed or embedded in the top of a monitor are displayed on the screen of each terminal. In such a configuration, the line of sight of the participant looking at the monitor may not match the line of sight of the participant displayed on the screen, resulting in a decrease in the quality of communication. For example, a camera-equipped display device having an imaging module (camera) built into a display element (display unit), a communication device, and a communication system have been disclosed (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2005-176151 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned conventional technology, the display drive of the display element and the image capture drive of the image capture module are alternately performed, which may cause flickering in the displayed image or captured image, resulting in a deterioration in display quality or image capture quality.

[0005] The present disclosure has been made in consideration of the above problems, and has an object to provide an imaging system capable of remotely displaying high-quality images. [Means for solving the problem]

[0006] An imaging system according to one aspect of the present disclosure includes a display device having a display panel, and an imaging device arranged between a display panel and a subject, and configured to capture an image of the subject through the display panel, wherein the imaging device acquires exposure data of a color different from the display color of the display device and generates imaging data of the subject. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an imaging system according to an embodiment. [Diagram 2] FIG. 2 is a schematic circuit diagram showing the main configuration of the display device. [Diagram 3] FIG. 3 is a schematic cross-sectional view of the display panel. [Figure 4] FIG. 4 is a timing chart showing sub-frame periods and light emission periods in one frame period during which image data for display is displayed. [Diagram 5] FIG. 5 is a diagram showing a color filter array of an image sensor. [Figure 6] FIG. 6 is a timing chart showing the relationship between image display timing and imaging data acquisition timing according to the first embodiment. [Figure 7] FIG. 7 is a timing chart showing the relationship between image display timing and imaging data acquisition timing according to the second embodiment. [Figure 8] FIG. 8 is a diagram showing a color filter array of an image sensor according to a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] The form (embodiment) for carrying out the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited by the contents described in the following embodiment. In addition, the components described below include those that a person skilled in the art can easily imagine and those that are substantially the same. Furthermore, the components described below can be appropriately combined. In addition, the disclosure is merely an example, and those that a person skilled in the art can easily imagine appropriate modifications while maintaining the gist of the disclosure are naturally included in the scope of the present disclosure. In addition, in order to make the explanation clearer, the drawings may be schematic in terms of the width, thickness, shape, etc. of each part compared to the actual embodiment, but they are merely examples and do not limit the interpretation of the present disclosure. In addition, in this specification and each figure, elements similar to those described above with respect to the previously mentioned figures may be given the same reference numerals, and detailed explanations may be omitted as appropriate.

[0009] 1 is a diagram showing a schematic configuration of an imaging system according to an embodiment. As shown in FIG. 1, an imaging system 200 according to an embodiment includes a display device 100 and an imaging device 300.

[0010] In the present disclosure, the display device 100 is a liquid crystal display device that performs display output by a so-called field sequential color (FSC) method in which pixels are controlled so as to transmit light of multiple colors from the same pixel at different timings.

[0011] 2 is a schematic circuit diagram showing the main configuration of the display device 100. The display device 100 includes a display panel module DPM and an image processing circuit 70. The display panel module DPM includes a display panel P and a light source device L.

[0012] The display panel P includes a display area 7, a signal output circuit 8, a scanning circuit 9, a VCOM drive circuit 10, a timing controller 13, and a power supply circuit 14. Hereinafter, one side of the display panel P facing the display area 7 will be referred to as a display surface, and the other side will be referred to as a rear surface. Also, when describing a side of the display device 100, it is located in a direction intersecting (for example, perpendicular to) the opposing direction of the display surface and the rear surface with the display device 100 as a reference.

[0013] In the display area 7, a plurality of pixels Pix are arranged in a matrix. Each pixel Pix includes a switching element 1 and two electrodes. Fig. 3 is a schematic cross-sectional view of the display panel. In Figs. 2 and 3, a pixel electrode 2 and a common electrode 6 are illustrated as two electrodes.

[0014] The display panel P has two opposing substrates and liquid crystal 3 sealed between the two substrates. Hereinafter, one of the two substrates will be referred to as a first substrate 30, and the other as a second substrate 20.

[0015] The first substrate 30 includes a light-transmitting glass substrate 35, a pixel electrode 2 laminated on the second substrate 20 side of the glass substrate 35, and an insulating layer 55 laminated on the second substrate 20 side so as to cover the pixel electrode 2. The pixel electrode 2 is provided individually for each pixel Pix. The second substrate 20 includes a light-transmitting glass substrate 21, a common electrode 6 laminated on the first substrate 30 side of the glass substrate 21, and an insulating layer 56 laminated on the first substrate 30 side so as to cover the common electrode 6. The common electrode 6 has a plate-like or film-like shape shared by multiple pixels Pix.

[0016] The liquid crystal 3 in the first embodiment is a polymer-dispersed liquid crystal. Specifically, the liquid crystal 3 includes a bulk 51 and fine particles 52. The orientation of the fine particles 52 changes in the bulk 51 according to the potential difference between the pixel electrode 2 and the common electrode 6. The potential of the pixel electrode 2 is individually controlled for each pixel Pix, thereby controlling the scattering state of the liquid crystal 3 for each pixel Pix.

[0017] FIG. 3 shows an example in which the pixel electrode 2 and the common electrode 6 are arranged opposite each other with the liquid crystal 3 therebetween. However, the display panel P may also be configured such that the pixel electrode 2 and the common electrode 6 are provided on a single substrate, and the orientation is changed by the electric field generated by the pixel electrode 2 and the common electrode 6, thereby controlling the scattering state of the liquid crystal 3.

[0018] Next, a mechanism for controlling the potentials of the pixel electrode 2 and the common electrode 6 will be described.

[0019] The switching element 1 is a switching element using a semiconductor, such as a thin film transistor (TFT). One of the source or drain of the switching element 1 is connected to one of two electrodes (pixel electrode 2). The other of the source or drain of the switching element 1 is connected to a signal line 4. The gate of the switching element 1 is connected to a scanning line 5. The scanning line 5 applies a potential to open and close between the source and drain of the switching element 1 under the control of a scanning circuit 9. The scanning circuit 9 controls the potential.

[0020] In the example shown in FIG. 2, multiple signal lines 4 are aligned along one of the arrangement directions of the pixels Pix (row direction). The signal lines 4 extend along the other arrangement direction of the pixels Pix (column direction). The signal lines 4 are shared by the switching elements 1 of multiple pixels Pix aligned in the column direction. Multiple scanning lines 5 are aligned along the column direction. The scanning lines 5 extend along the row direction. The scanning lines 5 are shared by the switching elements 1 of multiple pixels Pix aligned in the row direction.

[0021] In the present disclosure, the extension direction of the scanning lines 5 is the X direction, and the direction in which the multiple scanning lines 5 are arranged is the Y direction. In addition, in Fig. 2, one of the multiple scanning lines 5 arranged at both ends in the Y direction is designated as scanning line 5a, and the other is designated as scanning line 5b.

[0022] The common electrode 6 is connected to a VCOM drive circuit 10. The VCOM drive circuit 10 applies a common potential to the common electrode 6.

[0023] When the scanning circuit 9 supplies a potential that functions as a drive signal to the scanning line 5 and the switching element 1 is controlled to be on, the signal output circuit 8 supplies a pixel signal to the signal line 4, thereby charging the storage capacitance formed between the pixel electrode 2 and the common electrode 6 and the liquid crystal 3 (particles 52) that is a capacitive load. As a result, a voltage according to the pixel signal is applied between the pixel electrode 2 and the common electrode 6.

[0024] After the switching element 1 is turned off, the voltage applied between the pixel electrode 2 and the common electrode 6 is held by the liquid crystal 3 (particles 52) which is a storage capacitance and a capacitive load. The degree of scattering of the liquid crystal 3 (particles 52) is controlled according to the voltage applied between the pixel electrode 2 and the common electrode 6 for each pixel Pix. The liquid crystal 3 may be, for example, a polymer-dispersed liquid crystal in which the degree of scattering increases as the voltage applied between the pixel electrode 2 and the common electrode 6 for each pixel Pix increases, or may be a polymer-dispersed liquid crystal in which the degree of scattering increases as the voltage applied between the pixel electrode 2 and the common electrode 6 for each pixel Pix decreases.

[0025] As shown in Fig. 3, a light source device L is disposed on the side of the display panel P (below the display panel P in Fig. 2). The light source device L includes a light source 11 that irradiates light onto the side surface of the display panel P, and a light source drive circuit 12 that controls the light source 11. The light source 11 includes a first light source 11R, a second light source 11G, and a third light source 11B.

[0026] The first light source 11R, the second light source 11G, and the third light source 11B each emit light under the control of a light source drive circuit 12. The first light source 11R, the second light source 11G, and the third light source 11B are light sources using light emitting elements such as light emitting diodes (LEDs), but are not limited thereto and may be any light source whose emission timing can be controlled.

[0027] The light source driving circuit 12 controls the light emission timing of the first light source 11R, the second light source 11G, and the third light source 11B under the control of the timing controller 13. In the present disclosure, the emission color (first color) of the first light source 11R is red (R), the emission color (second color) of the second light source 11G is green (G), and the emission color (third color) of the third light source 11B is blue (B).

[0028] When light is emitted from the light source 11, the display area 7 is illuminated by the light (first color, second color, third color) emitted from one side in the Y direction. Each pixel Pix transmits or scatters the light emitted from one side in the Y direction. The degree of scattering of the liquid crystal 3 for each pixel Pix depends on the state of the liquid crystal 3 controlled in response to the pixel signal for each pixel Pix.

[0029] The timing controller 13 is a circuit that controls the operation timing of the signal output circuit 8, the scanning circuit 9, the VCOM driving circuit 10, and the light source driving circuit 12. In the present disclosure, the timing controller 13 operates based on a signal input via the image processing circuit 70.

[0030] The image processing circuit 70 outputs a signal based on the display image data I to the signal output circuit 8 and the timing controller 13. If pixel data is data indicating RGB gradation values ​​assigned to one pixel Pix among the multiple pixels Pix provided in the display area 7, the display image data I input to the image processing circuit 70 to output a display image is a collection of multiple pixel data for each pixel Pix in the display area 7. The image processing circuit 70 may be provided on one of the substrates constituting the display panel P, or may be mounted on a flexible printed circuit board on which wiring and the like extending from the display panel P are provided, or may be configured to be provided outside the display panel P.

[0031] Fig. 4 is a timing chart showing sub-frame periods and light emission periods of one frame period during which display image data is displayed. Fig. 4 shows an example of an N-th frame period. Specifically, the display device 100 displays display image data I at, for example, 60 FPS.

[0032] In a display device 100 that performs display output using the FSC method, the image display period (one frame period) of one frame based on display image data I is time-divided into a first sub-frame period RF, a second sub-frame period GF, and a third sub-frame period BF, as shown in FIG. 4.

[0033] In the vertical scanning period GateScan of the first sub-frame period RF, pixel data corresponding to the output gradation value of each pixel Pix corresponding to the first color (red (R)) of the display image data I(N) is written. As a result, a voltage corresponding to each pixel data of each pixel Pix is ​​applied to the pixel electrode 2, and the scattering state of the liquid crystal 3 of each pixel Pix is ​​controlled according to the applied voltage of the pixel electrode 2.

[0034] In the subsequent light emission period RON, the first light source 11R is caused to emit light. During this light emission period RON, light of a first color (red (R)) corresponding to the pixel data for each pixel Pix written in the immediately preceding vertical scanning period GateScan is scattered and displayed.

[0035] In the vertical scanning period GateScan of the second sub-frame period GF, pixel data corresponding to the output gradation value of each pixel Pix corresponding to the second color (green (G)) of the display image data I(N) is written. As a result, a voltage corresponding to each pixel data of each pixel Pix is ​​applied to the pixel electrode 2, and the scattering state of the liquid crystal 3 of each pixel Pix is ​​controlled according to the applied voltage of the pixel electrode 2.

[0036] In the subsequent light emission period GON, the second light source 11G is caused to emit light. In this light emission period GON, light of a second color (green (G)) corresponding to the pixel data for each pixel Pix written in the immediately preceding vertical scanning period GateScan is scattered and displayed.

[0037] In the vertical scanning period GateScan of the third sub-frame period BF, pixel data corresponding to the output gradation value of each pixel Pix corresponding to the third color (blue (B)) of the display image data I(N) is written. As a result, a voltage corresponding to each pixel data of each pixel Pix is ​​applied to the pixel electrode 2, and the scattering state of the liquid crystal 3 of each pixel Pix is ​​controlled according to the applied voltage of the pixel electrode 2.

[0038] In the subsequent light emission period BON, the third light source 11B is caused to emit light. In this light emission period BON, light of a third color (blue (B)) corresponding to the pixel data for each pixel Pix written in the immediately preceding vertical scanning period GateScan is scattered and displayed.

[0039] In the above-mentioned FSC type display device 100, an image in which three colors, a first color (red (R)), a second color (green (G)), and a third color (blue (B)), are synthesized (mixed) is recognized due to the afterimage phenomenon that occurs due to the limitations of the temporal resolution of the human eye. Furthermore, in the FSC type display device 100, since it is not necessary to provide a color filter for each pixel Pix, the light transmittance in the display area 7 can be increased.

[0040] 1, the imaging device 300 is a digital camera including an imaging element, an imaging lens for focusing light on the imaging element, etc. The imaging element may be, for example, a solid-state imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor, but is not limited thereto, and may be, for example, a CCD (Charge Coupled Devices) image sensor.

[0041] The image sensor is equipped with color filters that selectively transmit a first color (red (R)), a second color (green (G)), and a third color (blue (B)). Figure 5 is a diagram showing the color filter array of the image sensor.

[0042] The color filter array of the image sensor shown in Fig. 5 is composed of three primary color filters of red (R), green (G), and blue (B), and is an example of a Bayer array in which a combination of four pixels of R / G / G / B is regularly repeated (specifically, an array in which one first color (red (R)), one third color (blue (B)), and two second colors (green (G)) are arranged in a 2 x 2 pixel array). Note that the color filter array of the image sensor shown in Fig. 5 is only an example, and is not limited to the array shown in Fig. 5.

[0043] 1, in the present disclosure, the imaging device 300 is provided between a subject PA, which is, for example, a participant of a remote conference system, and the above-mentioned FSC-type display device 100. More preferably, in the imaging system 200 according to the embodiment, the imaging device 300 is provided on a vertical line of a line of sight A when the subject PA views the display panel P.

[0044] The image of the subject PA is transmitted through the display panel P, formed on the imaging element of the imaging device 300, and captured as image data. Specifically, the imaging device 300 captures image data of the subject PA at, for example, 90 FPS. In the following description, the image data acquired by the imaging device 300 is also referred to as "imaging data."

[0045] The image data captured by the imaging device 300 is distributed as video data to, for example, an information terminal 500 of a participant of a remote conference system via a network 400. For example, a desktop or notebook personal computer is exemplified as the information terminal 500. The imaging system 200 according to the embodiment may be configured, for example, as a display or a Web camera constituting the information terminal 500.

[0046] In the imaging system 200 according to the embodiment described above, light of the display image data I displayed on the display device 100 may affect the imaging data acquired by the imaging device 300. Hereinafter, a method for acquiring imaging data in the imaging system 200 according to the embodiment will be described.

[0047] (Embodiment 1) FIG. 6 is a timing chart showing the relationship between image display timing and imaging data acquisition timing according to the first embodiment.

[0048] In the example shown in Fig. 6, the image display timing in the display device 100 is similar to the timing chart described in Fig. 4. That is, an image display period of one frame (one frame period F) is time-divided into a first sub-frame period RF, a second sub-frame period GF, and a third sub-frame period BF.

[0049] FIG. 6 shows frame periods F1, F2, F3, and F4 spanning four frames, with the first subframe period in the frame period F1 designated as RF1, the second subframe period as GF1, and the third subframe period as BF1.

[0050] In addition, in the frame period F2, the first subframe period is designated RF2, the second subframe period is designated GF2, and the third subframe period is designated BF2.

[0051] In addition, in the frame period F3, the first subframe period is designated RF3, the second subframe period is designated GF3, and the third subframe period is designated BF3.

[0052] In addition, in the frame period F4, the first subframe period is designated as RF4, the second subframe period is designated as GF4, and the third subframe period is designated as BF4.

[0053] Synchronous control between the image display timing in the display device 100 and the acquisition timing (exposure timing) of imaging data in the imaging device 300 is performed, for example, by the display device 100. In this case, a synchronization signal output from the display device 100 is input to the imaging device 300 (see FIG. 1). Note that, without being limited to this, synchronization control between the image display timing in the display device 100 and the acquisition timing (exposure timing) of imaging data in the imaging device 300 may be performed, for example, by the imaging device 300. In this case, a synchronization signal output from the imaging device 300 is input to the display device 100.

[0054] In the example shown in FIG. 6, first, the imaging device 300 acquires exposure data B1 of a third color (blue (B)) different from the display colors (first color (red (R)), second color (green (G))) in two subframe periods, that is, a first subframe period RF1 of a frame period F1 in which a first color (red (R)) is displayed and a second subframe period GF1 in which a second color (green (G)) is displayed on the display device 100.

[0055] Next, the imaging device 300 acquires exposure data G1 of a second color (green (G)) different from the display colors (third color (blue (B)), first color (red (R))) in two sub-frame periods, that is, the third sub-frame period BF1 of the frame period F1 in which the third color (blue (B)) is displayed on the display device 100 and the first sub-frame period RF2 of the frame period F2 in which the first color (red (R)) is displayed.

[0056] Next, the imaging device 300 acquires exposure data R1 of a first color (red (R)) different from the display colors (second color (green (G), third color (blue (B)))) in two subframe periods, that is, the second subframe period GF2 of the frame period F2 in which the second color (green (G)) is displayed and the third subframe period BF2 in which the third color (blue (B)) is displayed on the display device 100.

[0057] Then, the imaging device 300 combines the exposure data R1 for the first color (red (R)), the exposure data G1 for the second color (green (G)), and the exposure data B1 for the third color (blue (B)) to generate imaging data RGB1 (=R1, G1, B1).

[0058] Next, the imaging device 300 acquires exposure data B2 of a third color (blue (B)) different from the display colors (first color (red (R)), second color (green (G))) in two sub-frame periods, that is, the first sub-frame period RF3 of the frame period F3 in which the first color (red (R)) is displayed and the second sub-frame period GF3 in which the second color (green (G)) is displayed on the display device 100.

[0059] Then, the imaging device 300 combines the exposure data R1 for the first color (red (R)), the exposure data G1 for the second color (green (G)), and the exposure data B2 for the third color (blue (B)) to generate imaging data RGB2 (=R1, G1, B2).

[0060] Next, the imaging device 300 acquires exposure data G2 of a second color (green (G)) different from the display colors (third color (blue (B)), first color (red (R))) in two sub-frame periods, that is, the third sub-frame period BF3 of the frame period F3 in which the third color (blue (B)) is displayed on the display device 100 and the first sub-frame period RF4 of the frame period F4 in which the first color (red (R)) is displayed.

[0061] Then, the imaging device 300 combines the exposure data R1 for the first color (red (R)), the exposure data G2 for the second color (green (G)), and the exposure data B2 for the third color (blue (B)) to generate imaging data RGB3 (=R1, G2, B2).

[0062] Next, the imaging device 300 acquires exposure data R2 of a first color (red (R)) different from the display color (second color (green (G), third color (blue (B)))) in two subframe periods, that is, the second subframe period GF4 of the frame period F4 in which the second color (green (G)) is displayed and the third subframe period BF4 in which the third color (blue (B)) is displayed on the display device 100.

[0063] Then, the imaging device 300 combines the exposure data R2 of the first color (red (R)), the exposure data G2 of the second color (green (G)), and the exposure data B2 of the third color (blue (B)) to generate imaging data RGB4 (=R2, G2, B2).

[0064] Thereafter, in a similar manner, the imaging device 300 acquires imaging data RGB5, RGB6, .... In this way, for example, when the display image data I is displayed on the display device 100 at 60 FPS, the imaging device 300 can capture imaging data of the subject PA at 90 FPS.

[0065] As described above, at the timing of acquiring imaging data according to the first embodiment, the imaging device 300 acquires exposure data of a color different from the display color of the display device 100 in two subframe periods. Then, the exposure data acquired in a plurality of subframe periods is synthesized to generate imaging data of the subject PA. This makes it possible to prevent the light of the display image data I displayed on the display device 100 from affecting the imaging data acquired by the imaging device 300, enabling high-quality remote image display.

[0066] (Embodiment 2) FIG. 7 is a timing chart showing the relationship between image display timing and imaging data acquisition timing according to the second embodiment.

[0067] 7, the image display timing in the display device 100 is the same as the timing chart described in Fig. 4, as in embodiment 1. That is, an image display period of one frame (one frame period F) is time-divided into a first sub-frame period RF, a second sub-frame period GF, and a third sub-frame period BF.

[0068] FIG. 7 illustrates frame periods F1 and F2 spanning two frames, with the first subframe period in the frame period F1 designated as RF1, the second subframe period as GF1, and the third subframe period as BF1.

[0069] In addition, in the frame period F2, the first subframe period is designated RF2, the second subframe period is designated GF2, and the third subframe period is designated BF2.

[0070] In the example shown in FIG. 7, first, the imaging device 300 acquires exposure data G1 of a second color (green (G)) different from the display color (first color (red (R))) and exposure data B1 of a third color (blue (B)) in a first sub-frame period RF1 of a frame period F1 in which a first color (red (R)) is displayed on the display device 100.

[0071] Next, in the second sub-frame period GF1 of the frame period F1 in which the second color (green (G)) is displayed on the display device 100, the imaging device 300 acquires exposure data B2 of a third color (blue (B)) different from the display color (second color (green (G))) and exposure data R1 of the first color (red (R)).

[0072] The imaging device 300 then synthesizes the exposure data R1 of the first color (red (R)), the exposure data G1 of the second color (green (G)), and the exposure data B1 / 2, B2 / 2 of the third color (blue (B)) to generate imaging data RGB1 (=R1, G1, B1 / 2+B2 / 2). The exposure data B1 / 2 indicates half the value of the exposure data B1, and the exposure data B2 / 2 indicates half the value of the exposure data B2. The imaging device 300 may be configured to generate imaging data RGB1 (=R1, G1, B1) by combining exposure data R1 of the first color (red (R)), exposure data G1 of the second color (green (G)), and exposure data B1 of the third color (blue (B)), or may be configured to generate imaging data RGB1 (=R1, G1, B2) by combining exposure data R1 of the first color (red (R)), exposure data G1 of the second color (green (G)), and exposure data B2 of the third color (blue (B)).

[0073] Next, in the third sub-frame period BF1 of the frame period F1 in which the display device 100 displays a third color (blue (B)), the imaging device 300 acquires exposure data R2 of a first color (red (R)) different from the display color (third color (blue (B))) and exposure data G2 of a second color (green (G)).

[0074] The imaging device 300 then synthesizes the exposure data R1 / 2, R2 / 2 of the first color (red (R)), the exposure data G2 of the second color (green (G)), and the exposure data B2 of the third color (blue (B)) to generate imaging data RGB2 (=R1 / 2+R2 / 2, G2, B2). The exposure data R1 / 2 indicates half the value of the exposure data R1, and the exposure data R2 / 2 indicates half the value of the exposure data R2. The imaging device 300 may be configured to generate imaging data RGB2 (=R1, G2, B2) by combining exposure data R1 of the first color (red (R)), exposure data G2 of the second color (green (G)), and exposure data B2 of the third color (blue (B)), or may be configured to generate imaging data RGB2 (=R2, G2, B2) by combining exposure data R2 of the first color (red (R)), exposure data G2 of the second color (green (G)), and exposure data B2 of the third color (blue (B)).

[0075] Next, in the first sub-frame period RF2 of the frame period F2 in which the first color (red (R)) is displayed on the display device 100, the imaging device 300 acquires exposure data G3 of a second color (green (G)) different from the display color (first color (red (R))) and exposure data B3 of a third color (blue (B)).

[0076] The imaging device 300 then synthesizes the exposure data R2 of the first color (red (R)), the exposure data G2 / 2, G3 / 2 of the second color (green (G)), and the exposure data B3 of the third color (blue (B)) to generate imaging data RGB3 (=R2, G2 / 2 + G3 / 2, B3). The exposure data G2 / 2 indicates half the value of the exposure data G2, and the exposure data G3 / 2 indicates half the value of the exposure data G3. The imaging device 300 may be configured to generate imaging data RGB3 (=R2, G2, B3) by combining exposure data R2 of the first color (red (R)), exposure data G2 of the second color (green (G)), and exposure data B3 of the third color (blue (B)), or may be configured to generate imaging data RGB3 (=R2, G3, B3) by combining exposure data R2 of the first color (red (R)), exposure data G3 of the second color (green (G)), and exposure data B3 of the third color (blue (B)).

[0077] Next, in the second sub-frame period GF2 of the frame period F2 in which the second color (green (G)) is displayed on the display device 100, the imaging device 300 acquires exposure data B4 of a third color (blue (B)) different from the display color (second color (green (G))) and exposure data R3 of the first color (red (R)).

[0078] The imaging device 300 then synthesizes the exposure data R3 of the first color (red (R)), the exposure data G3 of the second color (green (G)), and the exposure data B3 / 2, B4 / 2 of the third color (blue (B)) to generate imaging data RGB4 (=R3, G3, B3 / 2 + B4 / 2). The exposure data B3 / 2 represents half the value of the exposure data B3, and the exposure data B4 / 2 represents half the value of the exposure data B4. The imaging device 300 may be configured to generate imaging data RGB4 (=R3, G3, B3) by combining exposure data R3 of the first color (red (R)), exposure data G3 of the second color (green (G)), and exposure data B3 of the third color (blue (B)), or may be configured to generate imaging data RGB4 (=R3, G3, B4) by combining exposure data R3 of the first color (red (R)), exposure data G3 of the second color (green (G)), and exposure data B4 of the third color (blue (B)).

[0079] Next, in the third sub-frame period BF2 of the frame period F2 in which the display device 100 displays the third color (blue (B)), the imaging device 300 acquires exposure data R4 of a first color (red (R)) different from the display color (third color (blue (B))) and exposure data G4 of a second color (green (G)).

[0080] The imaging device 300 then synthesizes the exposure data R3 / 2, R4 / 2 of the first color (red (R)), the exposure data G4 of the second color (green (G)), and the exposure data B4 of the third color (blue (B)) to generate imaging data RGB5 (=R3 / 2+R4 / 2, G4, B4). The exposure data R3 / 2 indicates half the value of the exposure data R3, and the exposure data R4 / 2 indicates half the value of the exposure data R4. The imaging device 300 may be configured to synthesize the exposure data R3 of the first color (red (R)), the exposure data G4 of the second color (green (G)), and the exposure data B4 of the third color (blue (B)) to generate imaging data RGB5 (=R3, G4, B4), or may be configured to synthesize the exposure data R4 of the first color (red (R)), the exposure data G4 of the second color (green (G)), and the exposure data B4 of the third color (blue (B)) to generate imaging data RGB5 (=R4, G4, B4).

[0081] Thereafter, in a similar manner, the imaging device 300 acquires imaging data RGB6, RGB7, .... In this way, for example, when the display image data I is displayed on the display device 100 at 60 FPS, the imaging device 300 can capture imaging data of the subject PA at 180 FPS.

[0082] As described above, in the timing of acquiring imaging data according to the second embodiment, the imaging device 300 acquires exposure data of a color different from the display color of the display device 100 in each subframe period. Then, the data acquired in the multiple subframe periods is synthesized to generate imaging data of the subject PA. This makes it possible to prevent the light of the display image data I displayed on the display device 100 from affecting the imaging data acquired by the imaging device 300, enabling high-quality remote image display.

[0083] (Modification) Fig. 8 is a diagram showing a color filter array of an image sensor according to a modified example. The color filter array of the image sensor shown in Fig. 8 is composed of three complementary color filters of yellow (Y), cyan (C), and magenta (M), and illustrates a Bayer array in which a combination of four pixels of C / Y / Y / M is regularly repeated (specifically, an array in which one cyan (C), one magenta (M), and two yellow (Y) are arranged in a 2 x 2 pixel array). Note that, instead of the color filter array of the image sensor shown in Fig. 8, for example, an array in which a combination of four pixels of C / Y / G / M is regularly repeated (specifically, an array in which one cyan (C), one yellow (Y), one green (G), and one magenta (M) are arranged in a 2 x 2 pixel array) may be used.

[0084] As shown in FIG. 8, in a configuration in which the image sensor of the imaging device 300 has a color filter composed of complementary color filters, the image display period (one frame period) of one frame based on the display image data I may be time-divided into a first subfield period in which the first light source 11R and the second light source 11G are made to emit light to display yellow (Y), a second subfield period in which the second light source 11G and the third light source 11B are made to emit light to display cyan (C), and a third subfield period in which the third light source 11B and the first light source 11R are made to emit light to display magenta (M).

[0085] In this configuration, similarly to the first embodiment, the imaging device 300 captures exposure data of a color different from the display color of the display device 100 in two subframe periods. Then, the exposure data acquired in the multiple subframe periods is synthesized to generate imaging data of the subject PA. This makes it possible to prevent the light of the display image data I displayed on the display device 100 from affecting the imaging data acquired by the imaging device 300, enabling high-quality remote image display.

[0086] Also, similar to the second embodiment, the imaging device 300 captures exposure data of a color different from the display color of the display device 100 in each subframe period. Then, the exposure data acquired in the multiple subframe periods is synthesized to generate imaging data of the subject PA. This makes it possible to prevent the light of the display image data I displayed on the display device 100 from affecting the imaging data acquired by the imaging device 300, enabling high-quality remote image display.

[0087] Alternatively, as shown in FIG. 8, in a configuration in which the image sensor of the imaging device 300 has a color filter composed of complementary color filters, the image display period (one frame period) of one frame based on the display image data I may be time-divided into a first subfield period in which the first light source 11R is made to emit light to display a first color (red (R)), a second subfield period in which the second light source 11G is made to emit light to display a second color (green (G)), and a third subfield period in which the third light source 11B is made to emit light to display a third color (blue (B)), as in the first and second embodiments described above.

[0088] In such a configuration, similarly to the first embodiment, the imaging device 300 captures exposure data of a color different from the display color of the display device 100 in two sub-frame periods.

[0089] Specifically, for example, in the first subfield period of the first frame displaying red (R) (first color), exposure data of cyan (C) (fourth color), which is the complementary color of red (R) (first color), is acquired, and in the second subfield period of the first frame displaying green (G) (second color), exposure data of magenta (M) (fifth color), which is the complementary color of green (G) (second color), is acquired. Then, exposure data of blue (B) (third color) is extracted from the exposure data of cyan (C) (fourth color) acquired in the first subfield period of the first frame and the exposure data of magenta (M) (fifth color) acquired in the second subfield period.

[0090] Also, for example, in the third subfield period of the first frame displaying blue (B) (third color), exposure data of yellow (Y) (sixth color), which is the complementary color of blue (B) (third color), is acquired, and in the first subfield period of the second frame displaying red (R) (first color), exposure data of cyan (C) (fourth color), which is the complementary color of red (R) (first color), is acquired. Then, exposure data of green (G) (second color) is extracted from the exposure data of yellow (Y) (sixth color) acquired in the first subfield period of the first frame and the exposure data of cyan (C) (fourth color) acquired in the first subfield period of the second frame.

[0091] Also, for example, in the second subfield period of the second frame displaying green (G) (second color), exposure data of magenta (M) (fifth color), which is the complementary color of green (G) (second color), is acquired, and in the third subfield period of the second frame displaying blue (B) (third color), exposure data of yellow (Y) (sixth color), which is the complementary color of blue (B) (third color), is acquired. Then, exposure data of red (R) (first color) is extracted from the exposure data of magenta (M) (fifth color) acquired in the second subfield period of the second frame and the exposure data of yellow (Y) (sixth color) acquired in the third subfield period of the second frame.

[0092] Then, the exposure data for each color acquired during a plurality of subframe periods is synthesized to generate imaging data of the subject PA. This makes it possible to prevent the light of the display image data I displayed on the display device 100 from affecting the imaging data acquired by the imaging device 300, enabling high-quality remote image display.

[0093] Furthermore, similarly to the second embodiment, the imaging device 300 captures exposure data of a color different from the display color of the display device 100 in each sub-frame period.

[0094] Specifically, for example, in the first subfield period of the first frame displaying red (R) (first color), exposure data for cyan (C) (fourth color), which is the complementary color of red (R) (first color), is acquired, and exposure data for green (G) (second color) and blue (B) (third color) are extracted from the acquired exposure data for cyan (C) (fourth color).

[0095] Also, for example, in the second subfield period of the first frame displaying green (G) (second color), exposure data for magenta (M) (fifth color), which is the complementary color of green (G) (second color), is acquired, and exposure data for blue (B) (third color) and red (R) (first color) are extracted from the acquired exposure data for magenta (M) (fifth color).

[0096] Also, for example, in the third subfield period of the first frame in which blue (B) (third color) is displayed, exposure data for yellow (Y) (sixth color), which is the complementary color of blue (B) (third color), is acquired, and exposure data for red (R) (first color) and green (G) (second color) are extracted from the acquired exposure data for yellow (Y) (sixth color).

[0097] Then, the exposure data acquired in the multiple subframe periods is synthesized to generate imaging data of the subject PA. This makes it possible to prevent the light of the display image data I displayed on the display device 100 from affecting the imaging data acquired by the imaging device 300, enabling high-quality remote image display.

[0098] Although the preferred embodiment of the present disclosure has been described above, the present disclosure is not limited to such an embodiment. The contents disclosed in the embodiment are merely examples, and various modifications are possible within the scope of the present disclosure. For example, appropriate modifications made within the scope of the present disclosure naturally belong to the technical scope of the present invention. [Explanation of symbols]

[0099] 3 Liquid crystal 7 Display area 11 Light source 11R 1st light source 11G 2nd light source 11B Third light source 70 Image processing circuit 100 display device 200 Imaging System 300 Imaging device 400 Network 500 Information terminal DPM Display Panel Module L light source device P Display Panel PA Subject

Claims

1. A display device having a display panel; an imaging device disposed between the display panel and a subject, the imaging device capturing an image of the subject through the display panel; having The imaging device includes: acquiring exposure data of a color different from a display color of the display device, and generating imaging data of the subject; Imaging system.

2. The display device includes: a frame period during which one frame of image data is displayed on the display panel is time-divided into a plurality of subframe periods during which different colors are displayed; The imaging device includes: synthesizing the exposure data of the plurality of colors acquired during the plurality of sub-frame periods to generate the imaging data; The imaging system according to claim 1 .

3. The display device includes: the one frame period is time-divided into a first subframe period in which a first color is displayed, a second subframe period in which a second color different from the first color is displayed, and a third subframe period in which a third color different from the first color and the second color is displayed; The imaging system according to claim 2 .

4. The display device includes: a light source that irradiates a side surface of the display panel with light; The light source is a first light source that emits light at least during the first subframe period; a second light source that emits light at least during the second subframe period; a third light source that emits light at least during the third subframe period; Including, The imaging system according to claim 3 .

5. The imaging device includes: an image sensor including color filters that selectively transmit the first color, the second color, and the third color, The imaging system according to claim 4 .

6. the first color is red; the second color is green; the third color is blue; The imaging system according to claim 5 .

7. the first color is yellow; the second color is cyan; the third color is magenta; The imaging system according to claim 5 .

8. The imaging device includes: an image sensor including color filters that selectively transmit a fourth color that is a complementary color of the first color, a fifth color that is a complementary color of the second color, and a sixth color that is a complementary color of the third color, The imaging system according to claim 4 .

9. the first color is red; the second color is green; the third color is blue; The imaging system according to claim 8 .

10. The imaging device includes: acquiring exposure data of the third color in a first subframe period and a second subframe period following the first subframe period; acquiring exposure data of the second color in a third subframe period and a first subframe period following the third subframe period; acquiring exposure data of the first color in a second subframe period and a third subframe period subsequent to the second subframe period; 10. The imaging system according to claim 6, 7 or 9.

11. The imaging device includes: synthesizing the exposure data of the first color, the second color, and the third color, which are successively acquired, to generate the imaging data; The imaging system according to claim 10.

12. The imaging device includes: acquiring exposure data for the second color and the third color in a first subframe period; acquiring exposure data for the third color and the first color in a second subframe period; acquiring exposure data of the first color and the second color in a third subframe period; 10. The imaging system according to claim 6, 7 or 9.

13. The imaging device includes: generating the imaging data by synthesizing the exposure data of the second color and the third color acquired in a first subframe period and the exposure data of the third color and the first color acquired in a second subframe period subsequent to the first subframe period; The imaging system according to claim 12.

14. The imaging device includes: generating the imaging data by synthesizing the exposure data of the third color and the first color acquired in a second subframe period and the exposure data of the first color and the second color acquired in a third subframe period subsequent to the second subframe period; The imaging system according to claim 12.

15. The imaging device includes: generating the imaging data by synthesizing the exposure data of the first color and the second color acquired in a third subframe period and the exposure data of the second color and the third color acquired in a first subframe period subsequent to the third subframe period; The imaging system according to claim 12.

16. The imaging device includes: generating the imaging data by synthesizing the exposure data of the second color acquired in a first sub-frame period and the exposure data of the third color and the first color acquired in a second sub-frame period subsequent to the first sub-frame period; The imaging system according to claim 12.

17. The imaging device includes: generating the imaging data by combining the exposure data of the third color acquired in a second sub-frame period and the exposure data of the first color and the second color acquired in a third sub-frame period subsequent to the second sub-frame period; The imaging system according to claim 12.

18. The imaging device includes: generating the imaging data by synthesizing the exposure data of the first color acquired in a third subframe period and the exposure data of the second color and the third color acquired in a first subframe period subsequent to the third subframe period; The imaging system according to claim 12.

19. The imaging device includes: generating the imaging data by synthesizing the exposure data of the second color and the third color acquired in a first subframe period and the exposure data of the first color acquired in a second subframe period following the first subframe period; The imaging system according to claim 12.

20. The imaging device includes: generating the imaging data by synthesizing the exposure data of the third color and the first color acquired in a second sub-frame period and the exposure data of the second color acquired in a third sub-frame period following the second sub-frame period; The imaging system according to claim 12.

21. The imaging device includes: generating the imaging data by synthesizing the exposure data of the first color and the second color acquired in a third subframe period and the exposure data of the third color acquired in a first subframe period subsequent to the third subframe period; The imaging system according to claim 12.

Images