Imaging system

Abstract

To provide an imaging system capable of high-quality remote image display.SOLUTION: An imaging system includes a display device including a display panel in which a plurality of pixels are arranged in a first direction and a second direction intersecting with the first direction, and a light source that delivers light to a side surface of the display panel, and an imaging device that is disposed with the display panel held between the imaging device and a subject and captures an image of the subject having transmitted the display panel. The display device includes a first period for which image data is written in the pixels and a second period for which the light source is caused to emit light after the first period. The imaging device generates imaging data of the subject using the exposure data acquired in the first period.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an imaging system. [Background technology]

[0002] Remote conference systems via networks have become widespread. In such remote conference systems, typically, images of participants captured by a Web camera installed or embedded in the top of a monitor are displayed on the screen of each terminal. In such a configuration, the line of sight of the participant looking at the monitor may not match the line of sight of the participant displayed on the screen, resulting in a decrease in the quality of communication. For example, a camera-equipped display device having an imaging module (camera) built into a display element (display unit), a communication device, and a communication system have been disclosed (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2005-176151 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned conventional technology, the display drive of the display element and the image capture drive of the image capture module are alternately performed, which may cause flickering in the displayed image or captured image, resulting in a deterioration in display quality or image capture quality.

[0005] The present disclosure has been made in consideration of the above problems, and has an object to provide an imaging system capable of remotely displaying high-quality images. [Means for solving the problem]

[0006] An imaging system according to one embodiment of the present disclosure includes a display device having a display panel in which a plurality of pixels are arranged in a first direction and a second direction intersecting the first direction, and a light source that irradiates light onto a side of the display panel, and an imaging device arranged between a subject and the display panel and capturing an image of the subject transmitted through the display panel, wherein the display device has a first period in which pixel data is written to a plurality of the pixels, and a second period in which the light source is caused to emit light after the first period, and the imaging device generates imaging data of the subject using exposure data acquired in the first period. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an imaging system according to an embodiment. [Diagram 2] FIG. 2 is a schematic circuit diagram showing the main configuration of the display device. [Diagram 3] FIG. 3 is a schematic cross-sectional view of the display panel. [Figure 4] FIG. 4 is a timing chart showing sub-frame periods and light emission periods in one frame period during which image data for display is displayed. [Diagram 5] FIG. 5 is a diagram showing a color filter array of an image sensor. [Figure 6] FIG. 6 is a timing chart showing an example of acquisition timing of imaging data according to the first embodiment. [Figure 7] FIG. 7 is a timing chart showing an example of acquisition timing of imaging data according to the comparative example. [Figure 8] FIG. 8 is a timing chart showing an example of acquisition timing of imaging data according to the modification of the first embodiment. [Figure 9] FIG. 9 is a diagram showing a schematic configuration of an imaging system according to the second embodiment. [Figure 10] FIG. 10 is a timing chart showing an example of the timing of acquiring imaging data according to the second embodiment. [Figure 11]FIG. 11 is a timing chart showing an example of acquisition timing of imaging data according to a modification of the second embodiment. [Figure 12] FIG. 12 is a timing chart showing an example of the timing of acquiring imaging data according to the third embodiment. [Figure 13] FIG. 13 is a diagram showing an imaging range that overlaps with a display panel when the imaging device acquires exposure data in the third embodiment. [Figure 14] FIG. 14 is a timing chart showing an example of acquisition timing of imaging data according to the modified example of the third embodiment. [Figure 15] FIG. 15 is a diagram showing an imaging range that overlaps with a display panel when an imaging device acquires exposure data in a modification of the third embodiment. [Figure 16] FIG. 16 is a diagram showing a modified example of the color filter array of the image sensor. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] The form (embodiment) for carrying out the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited by the contents described in the following embodiment. In addition, the components described below include those that a person skilled in the art can easily imagine and those that are substantially the same. Furthermore, the components described below can be appropriately combined. In addition, the disclosure is merely an example, and those that a person skilled in the art can easily imagine appropriate modifications while maintaining the gist of the disclosure are naturally included in the scope of the present disclosure. In addition, in order to make the explanation clearer, the drawings may be schematic in terms of the width, thickness, shape, etc. of each part compared to the actual embodiment, but they are merely examples and do not limit the interpretation of the present disclosure. In addition, in this specification and each figure, elements similar to those described above with respect to the previously mentioned figures may be given the same reference numerals, and detailed explanations may be omitted as appropriate.

[0009] 1 is a diagram showing a schematic configuration of an imaging system according to an embodiment. As shown in FIG. 1, an imaging system 200 according to an embodiment includes a display device 100 and an imaging device 300.

[0010] In the present disclosure, display device 100 is a transmissive liquid crystal display device that performs display output using a so-called field sequential color (FSC) method in which pixels are controlled so as to transmit multiple colors of light from the same pixel at different timings.

[0011] 2 is a schematic circuit diagram showing the main configuration of the display device 100. The display device 100 includes a display panel module DPM and an image processing circuit 70. The display panel module DPM includes a display panel P and a light source device L.

[0012] The display panel P includes a display area 7, a signal output circuit 8, a scanning circuit 9, a VCOM drive circuit 10, a timing controller 13, and a power supply circuit 14. Hereinafter, one side of the display panel P facing the display area 7 will be referred to as a display surface, and the other side will be referred to as a rear surface. Also, when describing a side of the display device 100, it is located in a direction intersecting (for example, perpendicular to) the opposing direction of the display surface and the rear surface with the display device 100 as a reference.

[0013] In the display area 7, a plurality of pixels Pix are arranged in a matrix in the X direction (first direction) and the Y direction (second direction). The Y direction (second direction) is a direction that intersects with the X direction (first direction). More specifically, in the example shown in FIG. 2, the Y direction (second direction) is a direction that is perpendicular to the X direction (first direction).

[0014] The pixel Pix includes a switching element 1 and two electrodes. Fig. 3 is a schematic cross-sectional view of a display panel. In Figs. 2 and 3, a pixel electrode 2 and a common electrode 6 are illustrated as the two electrodes.

[0015] The display panel P has two opposing substrates and liquid crystal 3 sealed between the two substrates. Hereinafter, one of the two substrates will be referred to as a first substrate 30, and the other as a second substrate 20.

[0016] The first substrate 30 includes a light-transmitting glass substrate 35, a pixel electrode 2 laminated on the second substrate 20 side of the glass substrate 35, and an insulating layer 55 laminated on the second substrate 20 side so as to cover the pixel electrode 2. The pixel electrode 2 is provided individually for each pixel Pix. The second substrate 20 includes a light-transmitting glass substrate 21, a common electrode 6 laminated on the first substrate 30 side of the glass substrate 21, and an insulating layer 56 laminated on the first substrate 30 side so as to cover the common electrode 6. The common electrode 6 has a plate-like or film-like shape shared by multiple pixels Pix.

[0017] The liquid crystal 3 in the first embodiment is a polymer dispersed liquid crystal (PDLC). In other words, in this embodiment, the display panel P is a liquid crystal panel in which a polymer dispersed liquid crystal is sealed. Specifically, the liquid crystal 3 includes a bulk 51 and fine particles 52. The orientation of the fine particles 52 changes in the bulk 51 according to the potential difference between the pixel electrode 2 and the common electrode 6. The potential of the pixel electrode 2 is individually controlled for each pixel Pix, thereby controlling the scattering state of the liquid crystal 3 for each pixel Pix.

[0018] FIG. 3 shows an example in which the pixel electrode 2 and the common electrode 6 are arranged opposite each other with the liquid crystal 3 therebetween. However, the display panel P may also be configured such that the pixel electrode 2 and the common electrode 6 are provided on a single substrate, and the orientation is changed by the electric field generated by the pixel electrode 2 and the common electrode 6, thereby controlling the scattering state of the liquid crystal 3.

[0019] Next, a mechanism for controlling the potentials of the pixel electrode 2 and the common electrode 6 will be described.

[0020] The switching element 1 is a switching element using a semiconductor such as a thin film transistor (TFT). One of the source or drain of the switching element 1 is connected to one of two electrodes (pixel electrode 2). The other of the source or drain of the switching element 1 is connected to a signal line SDL(m) (m is an integer from 1 to M, M is the total number of signal lines). The gate of the switching element 1 is connected to a scanning line SCL(n) (n is an integer from 1 to N, N is the total number of scanning lines). The scanning line SCL(n) applies a potential to open and close between the source and drain of the switching element 1 under the control of a scanning circuit 9. The scanning circuit 9 controls the potential.

[0021] In the example shown in FIG. 2, multiple signal lines SDL(n) are arranged along one of the arrangement directions of the pixels Pix (row direction). Signal line SDL(m) extends along the other arrangement direction of the pixels Pix (column direction). The signal line SDL(m) is shared by the switching elements 1 of multiple pixels Pix arranged in the column direction. Multiple scanning lines SCL(n) are arranged along the column direction. The scanning line SCL(n) extends along the row direction. The scanning line SCL(n) is shared by the switching elements 1 of multiple pixels Pix arranged in the row direction.

[0022] In the present disclosure, the extension direction of the scanning line SCL(n) is defined as the X direction (first direction), and the direction in which the multiple scanning lines SCL(n) are arranged is defined as the Y direction (second direction).

[0023] The common electrode 6 is connected to a VCOM drive circuit 10. The VCOM drive circuit 10 applies a common potential to the common electrode 6.

[0024] The scanning circuit 9 sequentially supplies a driving signal that functions as an on-potential (driving potential) of the switching element 1 to the scanning line SCL(n) to which the pixels Pix aligned in the X direction (first direction) are connected. In other words, the scanning circuit 9 simultaneously supplies the driving signal to the pixels Pix aligned in the X direction (first direction). The scanning circuit 9 also sequentially supplies the driving signal to the pixels Pix aligned in the Y direction (second direction).

[0025] The signal output circuit 8 sequentially supplies a pixel signal that functions as data of a pixel corresponding to each pixel Pix (hereinafter also referred to as "pixel data") to a signal line SDL(m) to which a plurality of pixels Pix arranged in the Y direction (second direction) are connected. In other words, the signal output circuit 8 sequentially supplies pixel data to a plurality of pixels Pix arranged in the Y direction (second direction). Further, the signal output circuit 8 simultaneously supplies pixel data to a plurality of pixels Pix arranged in the X direction (first direction).

[0026] When the scanning circuit 9 supplies a driving signal to the scanning line SCL(n) and the switching elements 1 of a plurality of pixels Pix arranged in the X direction (first direction) are under on-control, the signal output circuit 8 supplies a pixel signal to the signal line SDL(m), thereby charging a storage capacitor formed between the pixel electrode 2 and the common electrode 6 of a plurality of pixels Pix arranged in the X direction (first direction) and the liquid crystal 3 (fine particles 52) which is a capacitive load. Thereby, a voltage corresponding to the pixel data corresponding to each pixel Pix is applied between the pixel electrode 2 and the common electrode 6 of a plurality of pixels Pix arranged in the X direction (first direction). When the scanning circuit 9 sequentially supplies a driving signal to the scanning lines SCL(n) arranged in the Y direction (second direction) and the signal output circuit 8 supplies pixel data corresponding to a plurality of pixels Pix connected to the scanning line SCL(n) to which the driving signal is supplied by the scanning circuit 9, the pixel data of an image for one sub-frame (a plurality of monochromatic images constituting an image for one frame) is written.

[0027] After the switching element 1 is turned off, the applied voltage between the pixel electrode 2 and the common electrode 6 is held by the storage capacitor and the liquid crystal 3 (fine particles 52) which is a capacitive load. The degree of scattering of the liquid crystal 3 (fine particles 52) is controlled according to the applied voltage between the pixel electrode 2 and the common electrode 6 for each pixel Pix. The liquid crystal 3 may be, for example, a polymer dispersed liquid crystal in which the degree of scattering increases as the applied voltage between the pixel electrode 2 and the common electrode 6 for each pixel Pix increases, or a polymer dispersed liquid crystal in which the degree of scattering increases as the applied voltage between the pixel electrode 2 and the common electrode 6 for each pixel Pix decreases.

[0028] As shown in Fig. 3, a light source device L is disposed on the side of the display panel P (below the display panel P in Fig. 2). The light source device L includes a light source 11 that irradiates light onto the side surface of the display panel P, and a light source drive circuit 12 that controls the light source 11. The light source 11 includes a first light source 11R, a second light source 11G, and a third light source 11B.

[0029] The first light source 11R, the second light source 11G, and the third light source 11B each emit light under the control of a light source drive circuit 12. The first light source 11R, the second light source 11G, and the third light source 11B are light sources using light emitting elements such as light emitting diodes (LEDs), but are not limited thereto and may be any light source whose emission timing can be controlled.

[0030] The light source driving circuit 12 controls the light emission timing of the first light source 11R, the second light source 11G, and the third light source 11B under the control of the timing controller 13. In the present disclosure, the emission color (first color) of the first light source 11R is red (R), the emission color (second color) of the second light source 11G is green (G), and the emission color (third color) of the third light source 11B is blue (B).

[0031] When light is emitted from the light source 11, the display area 7 is illuminated by the light (first color, second color, third color) emitted from one side in the Y direction. Each pixel Pix transmits or scatters the light emitted from one side in the Y direction. The degree of scattering of the liquid crystal 3 for each pixel Pix depends on the state of the liquid crystal 3 controlled in response to the pixel signal for each pixel Pix.

[0032] The timing controller 13 is a circuit that controls the operation timing of the signal output circuit 8, the scanning circuit 9, the VCOM driving circuit 10, and the light source driving circuit 12. In the present disclosure, the timing controller 13 operates based on a signal input via the image processing circuit 70.

[0033] The image processing circuit 70 outputs a signal based on the display image data to the signal output circuit 8 and the timing controller 13. If pixel data is data indicating RGB gradation values ​​assigned to one pixel Pix among the multiple pixels Pix provided in the display area 7, the display image data input to the image processing circuit 70 to output a display image is a collection of multiple pixel data for each pixel Pix in the display area 7. The image processing circuit 70 may be provided on one of the substrates constituting the display panel P, or may be mounted on a flexible printed circuit board on which wiring and the like extending from the display panel P are provided, or may be configured to be provided outside the display panel P.

[0034] Fig. 4 is a timing chart showing the sub-frame periods and light emission periods of one frame period in which display image data is displayed. In Fig. 4, the image display period FP of one frame is set to 20 ms. At this time, the image display frame rate of the display device 100 is set to 50 FPS.

[0035] In display device 100 that performs display output in the FSC system, an image display period FP of one frame based on display image data is time-divided into a first sub-frame period RF, a second sub-frame period GF, and a third sub-frame period BF, as shown in Fig. 4. Each of the first sub-frame period RF, the second sub-frame period GF, and the third sub-frame period BF is set to 6.67 ms.

[0036] In the vertical scanning period GateScan (first period) of the first sub-frame period RF, pixel data is written according to the output gradation value of each pixel Pix corresponding to the first color (red (R)) of the display image data. As a result, a voltage according to the pixel data for each pixel Pix is ​​applied to the pixel electrode 2, and the scattering state of the liquid crystal 3 for each pixel Pix is ​​controlled according to the applied voltage to the pixel electrode 2. The vertical scanning period GateScan (first period) of the first sub-frame period RF is set to, for example, 2.5 ms.

[0037] In the subsequent light emission period RON (second period), the first light source 11R emits light. In this light emission period RON (second period), light of a first color (red (R)) corresponding to each pixel data for each pixel Pix written in the immediately preceding vertical scanning period GateScan is scattered and displayed.

[0038] In the vertical scanning period GateScan (first period) of the second sub-frame period GF, pixel data is written according to the output gradation value of each pixel Pix corresponding to the second color (green (G)) of the display image data. As a result, a voltage according to the pixel data for each pixel Pix is ​​applied to the pixel electrode 2, and the scattering state of the liquid crystal 3 for each pixel Pix is ​​controlled according to the applied voltage to the pixel electrode 2. The vertical scanning period GateScan (first period) of the second sub-frame period GF is set to, for example, 2.5 ms.

[0039] In the subsequent light emission period GON (second period), the second light source 11G is caused to emit light. In this light emission period GON (second period), light of a second color (green (G)) corresponding to the pixel data for each pixel Pix written in the immediately preceding vertical scanning period GateScan is scattered and displayed.

[0040] In the vertical scanning period GateScan (first period) of the third sub-frame period BF, pixel data is written according to the output gradation value of each pixel Pix corresponding to the third color (blue (B)) of the display image data. As a result, a voltage according to the pixel data for each pixel Pix is ​​applied to the pixel electrode 2, and the scattering state of the liquid crystal 3 for each pixel Pix is ​​controlled according to the applied voltage to the pixel electrode 2. The vertical scanning period GateScan (first period) of the third sub-frame period BF is set to, for example, 2.5 ms.

[0041] In the subsequent light emission period BON (second period), the third light source 11B emits light. In this light emission period BON (second period), light of a third color (blue (B)) corresponding to the pixel data for each pixel Pix written in the immediately preceding vertical scanning period GateScan is scattered and displayed.

[0042] In the above-described FSC display device 100, due to the afterimage phenomenon caused by the limitation of the temporal resolution of the human eye, an image in which three colors, i.e., a first color (red (R)), a second color (green (G)), and a third color (blue (B)), are synthesized (color-mixed) is recognized. Further, in the FSC display device 100, since it is not necessary to provide a color filter for each pixel Pix, the light transmittance in the display area 7 can be increased.

[0043] Returning to FIG. 1, the imaging device 300 is a digital camera including an imaging element and an imaging lens that condenses light on the imaging element. Examples of the imaging element include solid-state imaging elements such as CMOS (Complementary Metal Oxide Semiconductor) image sensors, but are not limited thereto, and for example, a CCD (Charge Coupled Devices) image sensor may be used.

[0044] The image sensor includes a color filter that selectively transmits a first color (red (R)), a second color (green (G)), and a third color (blue (B)). FIG. 5 is a diagram showing the color filter arrangement of the image sensor.

[0045] In the color filter arrangement of the image sensor shown in FIG. 5, it is composed of primary color filters of three colors, red (R), green (G), and blue (B), and a Bayer array in which a combination of 4 pixels of R / G / G / B is regularly repeated (specifically, in a 2×2 pixel in the vertical and horizontal directions, one first color (red (R)), one third color (blue (B)), and two second colors (green (G)) are arranged) is exemplified. Note that the color filter arrangement of the image sensor shown in FIG. 5 is an example and is not limited to the arrangement shown in FIG. 5.

[0046] As shown in FIG. 1, in the present disclosure, the imaging device 300 is provided with the above-described FSC display device 100 interposed therebetween and, for example, a subject PA who is a participant in a remote conference system. More preferably, in the imaging system 200 according to the embodiment, the imaging device 300 is provided on the vertical line of the line of sight A when the subject PA views the display panel P.

[0047] 1 indicates the imaging range of the imaging device 300. An image of the subject PA is transmitted through the display panel P, formed on the imaging element of the imaging device 300, and captured as image data. In the following description, the image data acquired by the imaging device 300 is also referred to as "imaging data." Specifically, if the imaging period FI of one frame for acquiring imaging data is set to, for example, 20 ms, which is the same as the image display period FP of one frame in the display device 100, the imaging frame rate in the imaging device 300 is set to 50 FPS.

[0048] Synchronous control between the image display timing in the display device 100 and the acquisition timing (exposure timing) of imaging data in the imaging device 300 is performed, for example, by the display device 100. In this case, a synchronization signal output from the display device 100 is input to the imaging device 300 (see FIG. 1). Note that, without being limited to this, synchronization control between the image display timing in the display device 100 and the acquisition timing (exposure timing) of imaging data in the imaging device 300 may be performed, for example, by the imaging device 300. In this case, a synchronization signal output from the imaging device 300 is input to the display device 100.

[0049] The imaging data captured by the imaging device 300 is distributed as video data to, for example, an information terminal 500 of a participant of a remote conference system via a network 400. For example, a desktop or notebook personal computer is exemplified as the information terminal 500. The imaging system 200 according to the embodiment may be configured, for example, as a display or a Web camera constituting the information terminal 500.

[0050] In the imaging system 200 according to the embodiment described above, the imaging device 300 captures an image of the subject PA transmitted through the display panel P. For this reason, it is necessary to control the display timing of the image data for display on the display device 100 (hereinafter also referred to as "image display timing") and the acquisition timing of the imaging data in the imaging device 300 so that the light of the image data for display displayed on the display device 100 does not affect the imaging data acquired by the imaging device 300. The acquisition timing of the imaging data in the imaging system 200 according to the embodiment will be described below.

[0051] (Embodiment 1) FIG. 6 is a timing chart showing an example of acquisition timing of imaging data in the imaging system according to the first embodiment.

[0052] In the example shown in Fig. 6, the image display timing in the display device 100 is similar to the timing chart described in Fig. 4. That is, an image display period FP of one frame is time-divided into a first sub-frame period RF, a second sub-frame period GF, and a third sub-frame period BF.

[0053] 6, the image display period FP of one frame is set to 20 ms. At this time, the image display frame rate in the display device 100 is set to 50 FPS. The first subframe period RF, the second subframe period GF, and the third subframe period BF are each set to 6.67 ms, and the vertical scanning period GateScan (first period) of each subframe period is set to, for example, 2.5 ms.

[0054] In the first embodiment, the imaging device 300 generates imaging data of the subject PA using exposure data acquired during an exposure period EXP that overlaps with a vertical scanning period GateScan (first period) of each subframe period. The vertical scanning period GateScan (first period) of each subframe period is a period during which light emission of the light source device L (first light source 11R, second light source 11G, third light source 11B) is stopped. This makes it possible to suppress the light of the display image data from affecting the imaging data acquired by the imaging device 300. This makes it possible to acquire imaging data in which the influence of the display image data is suppressed.

[0055] The exposure period EXP is set to be equal to or shorter than the vertical scanning period GateScan (first period) of each subframe period. Specifically, in the example shown in FIG. 6, the exposure period EXP is set to be equal to or shorter than 2.5 ms.

[0056] In the example shown in FIG. 6, first, the imaging device 300 acquires first exposure data during an exposure period EXP that overlaps with a vertical scanning period GateScan (first period) of the first sub-frame period RF.

[0057] Next, the imaging device 300 obtains second exposure data during the exposure period EXP that overlaps with the vertical scanning period GateScan (first period) of the second sub-frame period GF.

[0058] Next, the imaging device 300 obtains third exposure data during the exposure period EXP that overlaps with the vertical scanning period GateScan (first period) of the third sub-frame period BF.

[0059] Then, the imaging device 300 synthesizes the first exposure data, the second exposure data, and the third exposure data to generate imaging data for one frame imaging period FI. The substantial exposure period at the timing of acquiring the imaging data shown in Fig. 6 corresponds to the length obtained by adding the exposure period EXP overlapping with the vertical scanning period GateScan (first period) of each subframe period during which the first exposure data, the second exposure data, and the third exposure data were acquired. Specifically, in the example shown in Fig. 6, the substantial exposure time is 7.5 ms or less.

[0060] FIG. 7 is a timing chart showing an example of acquisition timing of imaging data according to the comparative example.

[0061] In the timing of acquiring imaging data according to the comparative example shown in Fig. 7, an example is shown in which a non-display period HF of the display device 100 is provided after the third sub-frame period BF, and an exposure period EXP of the imaging device 300 is provided in the non-display period HF. Even when the timing of acquiring imaging data according to the comparative example shown in Fig. 7 is applied, it is possible to acquire imaging data in which the influence of the image data for display is suppressed.

[0062] On the other hand, in the comparative example shown in Fig. 7, if the image display period FP of one frame in the display device 100 is set to 20 ms (50 FPS) similar to that of the first embodiment shown in Fig. 6, the light emission period (second period) of each subframe period becomes shorter. In other words, in the image display period FP of one frame, the period during which the light source device L is turned off becomes relatively shorter. Also, in the comparative example shown in Fig. 7, if the imaging period FI of one frame in the imaging device 300 is set to 20 ms (50 FPS) similar to that of the first embodiment shown in Fig. 6, the exposure period EXP for the imaging period FI of one frame becomes relatively shorter.

[0063] In contrast, in the acquisition timing of the imaging data according to the first embodiment, the exposure data acquired during the exposure period EXP overlapping with the vertical scanning period GateScan (first period) of each subframe period is synthesized to generate imaging data of the subject PA. As a result, the light emission period (second period) of each subframe period can be relatively longer than that of the comparative example shown in FIG. 7. As a result, the luminance of the display image of the display device 100 can be increased compared to that of the comparative example. Also, the substantial exposure period for the imaging period FI of one frame can be relatively longer than that of the acquisition timing of the imaging data according to the comparative example shown in FIG. 7. As a result, the luminance of the image captured by the imaging device 300 can be increased compared to that of the comparative example.

[0064] (Modification) 8 is a timing chart showing an example of the timing of acquiring imaging data according to a modification of embodiment 1. Here, a description similar to that of the timing of acquiring imaging data according to embodiment 1 may be omitted.

[0065] In the example shown in FIG. 8, first, the imaging device 300 acquires first exposure data in an exposure period EXP that overlaps with a vertical scanning period GateScan (first period) of the first sub-frame period RF1 in an image display period FP1.

[0066] Next, the imaging device 300 obtains second exposure data during the exposure period EXP that overlaps with the vertical scanning period GateScan (first period) of the second sub-frame period GF1.

[0067] Next, the imaging device 300 obtains third exposure data during the exposure period EXP that overlaps with the vertical scanning period GateScan (first period) of the third sub-frame period BF1.

[0068] Next, in the image display period FP2 of the next frame, the imaging device 300 obtains the fourth exposure data in the exposure period EXP that overlaps with the vertical scanning period GateScan (first period) of the first sub-frame period RF2.

[0069] Then, the imaging device 300 synthesizes the first exposure data, the second exposure data, the third exposure data, and the fourth exposure data to generate imaging data for one frame imaging period FI (26.67 ms). The substantial exposure period at the timing of acquiring the imaging data shown in Fig. 8 corresponds to the length obtained by adding the exposure period EXP overlapping with the vertical scanning period GateScan (first period) of each subframe period during which the first exposure data, the second exposure data, the third exposure data, and the fourth exposure data were acquired. Specifically, in the example shown in Fig. 8, the substantial exposure time is 10 ms or less.

[0070] 8, the imaging frame rate (37.5 FPS in this example) is lower than the image display frame rate (50 FPS in this example), but the effective exposure time can be longer than in Example 1. This allows the brightness of the image captured by the imaging device 300 to be higher than in Example 1.

[0071] In the first embodiment, an example is shown in which the exposure data acquired in the exposure period EXP overlapping the vertical scanning period GateScan (first period) of three subframe periods is synthesized to generate imaging data, and in the modified example of the first embodiment, an example is shown in which the exposure data acquired in the exposure period EXP overlapping the vertical scanning period GateScan (first period) of four subframe periods is synthesized to generate imaging data, but the present invention is not limited to this. For example, it is also possible to use a mode in which the exposure data acquired in the exposure period EXP overlapping the vertical scanning period GateScan (first period) of five or more subframe periods is synthesized to generate imaging data.

[0072] (Embodiment 2) 9 is a diagram showing a schematic configuration of an imaging system according to embodiment 2. Here, detailed description of the same configuration as that of the imaging system 200 may be omitted.

[0073] 9, the imaging system 200a according to the second embodiment includes a liquid crystal shutter 600 disposed between a display panel P and an imaging device 300. The liquid crystal shutter 600 is provided on the optical axis of the imaging device 300. The liquid crystal shutter 600 and the imaging device 300 overlap on a vertical line of a line of sight A when the display panel P is viewed from a subject PA.

[0074] 10 is a timing chart showing an example of the timing of acquiring imaging data according to embodiment 2. Here, explanations similar to those of the timing of acquiring imaging data according to embodiment 1 and its modified examples may be omitted.

[0075] 10, the image display timing in the display device 100 is similar to the timing chart described in embodiment 1. That is, an image display period FP of one frame is time-divided into a first sub-frame period RF, a second sub-frame period GF, and a third sub-frame period BF.

[0076] 10, the image display period FP of one frame is set to 20 ms, as in the first embodiment. At this time, the image display frame rate in the display device 100 is set to 50 FPS. The first sub-frame period RF, the second sub-frame period GF, and the third sub-frame period BF are each set to 6.67 ms, and the vertical scanning period GateScan (first period) of each sub-frame period is set to, for example, 2.5 ms.

[0077] The liquid crystal shutter 600 transmits light from the subject PA during a light transmission period TS that overlaps with a vertical scanning period GateScan (first period) of each subframe period, and blocks light from the subject PA during a light blocking period SP that overlaps with a light emission period (second period) of each subframe period. The light transmission period TS is set to be equal to or shorter than the vertical scanning period GateScan (first period) of each subframe period. Specifically, in the example shown in FIG. 10, the light transmission period TS is set to be equal to or shorter than 2.5 ms.

[0078] The synchronous control of the liquid crystal shutter 600 is performed, for example, by the display device 100. In this case, a synchronous signal output from the display device 100 is input to the liquid crystal shutter 600 (see FIG. 9 ). Note that, without being limited to this, the synchronous control of the liquid crystal shutter 600 may be performed, for example, by the imaging device 300. In this case, a synchronous signal output from the imaging device 300 is input to the liquid crystal shutter 600.

[0079] The imaging device 300 generates imaging data of the object PA using exposure data acquired during a light transmission period TS overlapping with a vertical scanning period GateScan (first period) of each subframe period. This makes it possible to acquire imaging data in which the influence of the display image data is suppressed, as in the first embodiment.

[0080] In the second embodiment, the exposure period EXP of the imaging device 300 is a period that overlaps with the vertical scanning period GateScan (first period) of each subframe period. Specifically, in the example shown in Fig. 10, the exposure period EXP is, for example, 16 ms to 20 ms. Note that, in the second embodiment, the exposure period EXP of the imaging device 300 is continuous with both the vertical scanning period GateScan (first period) of each subframe period and the light emission period (second period) of each subframe period, but the substantial exposure period is a period that includes the light transmission period TS when the liquid crystal shutter 600 is in the transmitting state.

[0081] In the example shown in FIG. 10, first, the imaging device 300 acquires first exposure data in a light transmission period TS that overlaps with a vertical scanning period GateScan (first period) of the first sub-frame period RF.

[0082] Next, the imaging device 300 obtains second exposure data during the light transmission period TS that overlaps with the vertical scanning period GateScan (first period) of the second sub-frame period GF.

[0083] Next, the imaging device 300 obtains third exposure data during the light transmitting period TS that overlaps with the vertical scanning period GateScan (first period) of the third sub-frame period BF.

[0084] Then, the imaging device 300 synthesizes the first exposure data, the second exposure data, and the third exposure data to generate imaging data for one frame imaging period FI. The substantial exposure period at the timing of acquiring the imaging data shown in Fig. 10 corresponds to the length obtained by adding the light transmission period TS overlapping with the vertical scanning period GateScan (first period) of each subframe period during which the first exposure data, the second exposure data, and the third exposure data were acquired. Specifically, in the example shown in Fig. 10, the substantial exposure time is 7.5 ms or less.

[0085] At the timing of acquiring the imaging data according to the second embodiment, the exposure data acquired during the light transmission period TS of the liquid crystal shutter 600 overlapping with the vertical scanning period GateScan (first period) of each subframe period is synthesized to generate imaging data of the subject PA. As a result, the light emission period (second period) of each subframe period can be relatively longer than that of the comparative example shown in FIG. 7. As a result, the luminance of the display image of the display device 100 can be increased compared to that of the comparative example. Also, the substantial exposure period for the imaging period FI of one frame can be relatively longer than that of the imaging data acquisition timing according to the comparative example shown in FIG. 7. As a result, the luminance of the image captured by the imaging device 300 can be increased compared to that of the comparative example.

[0086] (Modification) 11 is a timing chart showing an example of the timing of acquiring imaging data according to a modification of embodiment 2. Here, a description similar to that of the timing of acquiring imaging data according to embodiment 2 may be omitted.

[0087] In the modification of the second embodiment, the exposure period EXP of the imaging device 300 is a period overlapping the first subframe period RF1, the second subframe period GF2, the third subframe period GF2 of the image display period FP1, and the vertical scanning period GateScan (first period) of the first subframe period RF2 of the image display period FP2 of the next frame. Specifically, in the example shown in FIG. 10, the exposure period EXP is, for example, 22.5 ms to 26.67 ms. Note that, in the modification of the second embodiment, the exposure period EXP of the imaging device 300 is continuous over both the vertical scanning period GateScan (first period) of each subframe period and the light emission period (second period) of each subframe period, but the substantial exposure period is a period obtained by adding the light transmission period TS when the liquid crystal shutter 600 is in the transmitting state.

[0088] In the example shown in FIG. 11, first, the imaging device 300 acquires first exposure data in a light transmission period TS that overlaps with a vertical scanning period GateScan (first period) of the first sub-frame period RF1 in an image display period FP1.

[0089] Next, the imaging device 300 obtains second exposure data during the light transmission period TS that overlaps with the vertical scanning period GateScan (first period) of the second sub-frame period GF1.

[0090] Next, the imaging device 300 obtains third exposure data during the light transmitting period TS that overlaps with the vertical scanning period GateScan (first period) of the third sub-frame period BF1.

[0091] Next, in the image display period FP2 of the next frame, the imaging device 300 obtains the fourth exposure data in the light transmitting period TS that overlaps with the vertical scanning period GateScan (first period) of the first sub-frame period RF2.

[0092] Then, the imaging device 300 synthesizes the first exposure data, the second exposure data, the third exposure data, and the fourth exposure data to generate imaging data for one frame imaging period FI (26.67 ms). The substantial exposure period at the timing of acquiring the imaging data shown in Fig. 11 corresponds to the length obtained by adding the light transmission period TS overlapping with the vertical scanning period GateScan (first period) of each subframe period during which the first exposure data, the second exposure data, the third exposure data, and the fourth exposure data were acquired. Specifically, in the example shown in Fig. 11, the substantial exposure time is 10 ms or less.

[0093] 11, the imaging frame rate (37.5 FPS in this example) is lower than the image display frame rate (50 FPS in this example), but the effective exposure time can be longer than in Example 2. This allows the brightness of the image captured by the imaging device 300 to be higher than in Example 2.

[0094] In the second embodiment, an example is shown in which the exposure data acquired in the light transmission period TS overlapping the vertical scanning period GateScan (first period) of three subframe periods is synthesized to generate imaging data, and in the modified example of the second embodiment, an example is shown in which the exposure data acquired in the light transmission period TS overlapping the vertical scanning period GateScan (first period) of four subframe periods is synthesized to generate imaging data, but the present invention is not limited to this. For example, the imaging data may be generated by synthesizing the exposure data acquired in the light transmission period TS overlapping the vertical scanning period GateScan (first period) of five or more subframe periods.

[0095] Furthermore, the component that transmits or blocks light from the subject PA is not limited to the liquid crystal shutter. Specifically, instead of the liquid crystal shutter 600, for example, a mechanical shutter may be provided.

[0096] (Embodiment 3) 12 is a timing chart showing an example of the timing of acquiring imaging data according to the third embodiment. Here, detailed description of the same configuration as that of the imaging system 200 may be omitted. Also, description of the same timing of acquiring imaging data according to the first embodiment and its modified examples may be omitted.

[0097] In the third embodiment, all the pixels Pix of the display panel P are reset at the reset timing RST immediately before the vertical scanning period GateScan (first period) of each sub-frame period.

[0098] Specifically, the signal output circuit 8 of the display device 100 supplies pixel data to all pixels Pix of the display panel P such that the display panel P transmits maximum light from the subject PA at the reset timing RST immediately before the vertical scanning period GateScan (first period) of each subframe period.

[0099] Then, in the vertical scanning period GateScan (first period) after the reset, the scanning circuit 9 of the display device 100 sequentially supplies driving signals to the scanning lines SCL(n), and the signal output circuit 8 supplies pixel data corresponding to the multiple pixels Pix connected to the scanning line SCL(n) to which the driving signal is supplied by the scanning circuit 9.

[0100] The imaging device 300 acquires exposure data during an exposure period EXP that overlaps with a vertical scanning period GateScan (first period) of each subframe period. Fig. 13 is a diagram showing an imaging range that overlaps with a display panel when the imaging device acquires exposure data in the third embodiment.

[0101] In the vertical scanning period GateScan after the reset period RST, pixel data corresponding to the display image data is written sequentially to each pixel Pix of the display panel P. In the third embodiment, the period during which the area in which pixel data corresponding to the display image data is not written (the area below the dashed lines shown in Figs. 12 and 13) overlaps with the imaging range of the imaging device 300 (the area shown by the dashed line in Fig. 13) is set as the exposure period EXP. In other words, in the timing of acquiring the imaging data according to the third embodiment, the imaging range of the imaging device 300 and the area in which pixel data corresponding to the display image data is written do not overlap when acquiring the exposure data. This makes it possible to acquire imaging data in which the influence of the display image data is suppressed more than in the first embodiment.

[0102] 12 shows an example in which the exposure data acquired during the exposure period EXP overlapping the vertical scanning period GateScan (first period) of three subframe periods is synthesized to generate imaging data, but the present invention is not limited to this. For example, the exposure data acquired during the exposure period EXP overlapping the vertical scanning period GateScan (first period) of four or more subframe periods may be synthesized to generate imaging data.

[0103] (Modification) FIG. 14 is a timing chart showing an example of the acquisition timing of imaging data according to a modified example of Embodiment 3. Here, detailed description of the same configuration as that of the imaging system 200 may be omitted. Also, the same description as that of the acquisition timing of imaging data according to Embodiment 2 and its modified example may be omitted.

[0104] In the modified example of Embodiment 3, all the pixels Pix of the display panel P are reset at the reset timing RST immediately before the vertical scanning period GateScan (first period) of each sub-frame period.

[0105] Specifically, the signal output circuit 8 of the display device 100 supplies pixel data that allows the display panel P to transmit the maximum amount of light from the subject PA to all the pixels Pix of the display panel P at the reset timing RST immediately before the vertical scanning period GateScan (first period) of each sub-frame period.

[0106] Then, in the vertical scanning period GateScan (first period) after reset, the scanning circuit 9 of the display device 100 sequentially supplies drive signals to the scanning lines SCL(n), and the signal output circuit 8 supplies pixel data corresponding to a plurality of pixels Pix connected to the scanning lines SCL(n) to which drive signals are supplied by the scanning circuit 9.

[0107] Also in the modified example of Embodiment 3, similar to Embodiment 2, the exposure period EXP of the imaging device 300 is continuous over both the vertical scanning period GateScan (first period) of each sub-frame period and the light emission period (second period) of each sub-frame period. However, the substantial exposure period is the period obtained by adding the light transmission period TS when the liquid crystal shutter 600 is in the transmission state.

[0108] The imaging device 300 acquires exposure data in the light transmission period TS overlapping with the vertical scanning period GateScan (first period) of each sub-frame period. FIG. 15 is a diagram showing the imaging range overlapping the display panel when the imaging device acquires exposure data in the modified example of Embodiment 3.

[0109] In the vertical scanning period GateScan after the reset period RST, pixel data corresponding to the display image data is written sequentially to each pixel Pix of the display panel P. In the modification of the third embodiment, the period during which the area in which pixel data corresponding to the display image data is not written (the area below the dashed lines shown in Figs. 14 and 15) overlaps with the imaging range of the imaging device 300 (the area shown by the dashed line in Fig. 13) is set as the light transmission period TS. In other words, in the timing of acquiring the imaging data according to the modification of the third embodiment, the imaging range of the imaging device 300 and the area in which pixel data corresponding to the display image data is written do not overlap when acquiring the exposure data. This makes it possible to acquire imaging data in which the influence of the display image data is suppressed more than in the second embodiment.

[0110] 14 shows an example in which the exposure data acquired during the light transmission periods TS overlapping the vertical scanning periods GateScan (first periods) of three subframe periods are synthesized to generate imaging data, but the present invention is not limited to this. For example, the exposure data acquired during the light transmission periods TS overlapping the vertical scanning periods GateScan (first periods) of four or more subframe periods may be synthesized to generate imaging data.

[0111] Fig. 16 is a diagram showing a modified example of a color filter array of an image sensor. The color filter array of the image sensor shown in Fig. 16 is composed of three complementary color filters of yellow (Y), cyan (C), and magenta (M), and illustrates a Bayer array in which a combination of four pixels of C / Y / Y / M is regularly repeated (specifically, an array in which one cyan (C), one magenta (M), and two yellow (Y) are arranged in a 2 x 2 pixel array). Note that, instead of the color filter array of the image sensor shown in Fig. 8, for example, an array in which a combination of four pixels of C / Y / G / M is regularly repeated (specifically, an array in which one cyan (C), one yellow (Y), one green (G), and one magenta (M) are arranged in a 2 x 2 pixel array) may be used.

[0112] Although the preferred embodiment of the present disclosure has been described above, the present disclosure is not limited to such an embodiment. The contents disclosed in the embodiment are merely examples, and various modifications are possible within the scope of the present disclosure. For example, appropriate modifications made within the scope of the present disclosure naturally belong to the technical scope of the present invention. [Explanation of symbols]

[0113] 3 Liquid crystal 7 Display area 11 Light source 11R 1st light source 11G 2nd light source 11B Third light source 70 Image processing circuit 100 display device 200,200a Imaging System 300 Imaging device 400 Network 500 Information terminal 600 Liquid Crystal Shutter DPM Display Panel Module L light source device P Display Panel PA Subject

Claims

1. A display device including: a display panel in which a plurality of pixels are arranged in a first direction and a second direction intersecting the first direction; and a light source that irradiates light onto a side surface of the display panel; an imaging device disposed between the display panel and a subject, the imaging device capturing an image of the subject through the display panel; having The display device includes: a first period for writing pixel data to the plurality of pixels; a second period in which the light source is caused to emit light after the first period; having The imaging device includes: generating imaging data of the subject using the exposure data acquired during the first period; Imaging system.

2. A one-frame period for displaying one frame of an image on the display panel is a first subframe period in which a first color is displayed; a second subframe period in which a second color different from the first color is displayed; a third subframe period in which a third color different from the first color and the second color is displayed; Including, The light source is a first light source that emits light during a second period of the first subframe period; a second light source that emits light during a second period of the second subframe period; a third light source that emits light during a second period of the third subframe period; Including, The imaging device includes: generating the imaging data by combining first exposure data acquired during a first period of the first sub-frame period, second exposure data acquired during a first period of the second sub-frame period, and third exposure data acquired during a first period of the third sub-frame period; The imaging system according to claim 1 .

3. a shutter disposed between the display panel and the imaging device, the shutter transmitting light during the first period and blocking light during the second period; A one-frame period for displaying one frame of an image on the display panel is a first subframe period in which a first color is displayed; a second subframe period in which a second color different from the first color is displayed; a third subframe period in which a third color different from the first color and the second color is displayed; Including, The light source is a first light source that emits light during a second period of the first subframe period; a second light source that emits light during a second period of the second subframe period; a third light source that emits light during a second period of the third subframe period; Including, The imaging device includes: generating the imaging data using exposure data acquired over a first period of the first sub-frame period, a first period of the second sub-frame period, and a first period of the third sub-frame period; The imaging system according to claim 1 .

4. The shutter is a liquid crystal shutter. The imaging system according to claim 3 .

5. A frame rate for acquiring the imaging data is the same as a frame rate for displaying an image on the display panel. The imaging system according to claim 2 or 3.

6. a frame rate at which the imaging data is acquired is lower than a frame rate at which an image is displayed on the display panel; The imaging system according to claim 2 or 3.

7. The display device includes: a scanning circuit that simultaneously supplies a drive signal to a plurality of pixels aligned in the first direction and sequentially supplies a drive signal to a plurality of pixels aligned in the second direction; a signal output circuit that supplies pixel data of an image to be displayed on the display panel to a plurality of pixels to which a drive signal is supplied; Equipped with All pixels of the display panel are reset immediately before the first period; a region where pixel data of an image to be displayed on the display panel is not supplied during the first period and an imaging range of the imaging device when acquiring the exposure data overlap with each other; The imaging system according to claim 1 .

8. The display panel is a liquid crystal panel in which a polymer dispersed liquid crystal is enclosed. The imaging system according to claim 1 .

Images