Line-of-sight detection device, method for manufacturing line-of-sight detection device, glasses terminal, and camera

Abstract

Provided are: a line-of-sight detection device capable of improving detection accuracy in eye tracking; a method for manufacturing the line-of-sight detection device; and a glasses terminal and a camera provided with the line-of-sight detection device. This line-of-sight detection device is configured to have a display device for displaying an image, and an imaging element for detecting a viewpoint of a user, the imaging element being arranged on the outer periphery of the display device in close contact with an effective pixel region of the display device. In addition, a display element is sealed between the semiconductor substrate and the light guide body, and the display device displays a formed image, and the imaging element is arranged on the semiconductor substrate in close contact with the light guide body and is used for detecting the viewpoint of a user. The display device is configured so as to have a display device for displaying an image, an imaging element for detecting a viewpoint of a user, and a substrate on which the display device and the imaging element are arranged in close contact with each other. Further, a glasses terminal and a camera having the same are configured.

Description

Technical Field

[0001] The present disclosure relates to a line-of-sight detection device, a manufacturing method of the line-of-sight detection device, a glasses terminal having the line-of-sight detection device, and a camera. Background Art

[0002] In recent years, for example, in the fields of virtual reality (VR: Virtual Reality), augmented reality (AR: Augmented Reality), etc., a head-mounted display configured as follows is known. By fixedly disposing a small display device in front of the eyes of a user, the user can visually confirm a display image on the display device.

[0003] In the display device of the head-mounted display, there are a non-transmissive type in which the user cannot observe the outside world other than the display image at all, and a transmissive type (perspective type) configured to be able to visually confirm the outside world in addition to the display image.

[0004] In such a head-mounted display, it is configured that the display image moves in conjunction with the movement of the user's line of sight. Therefore, an eye tracking function for capturing the movement of the user's line of sight is provided in the head-mounted display.

[0005] Eye tracking refers to a method of measuring the position of the viewing point and the movement of the eyeball relative to the head position and tracking them.

[0006] As a technique for inferring the line-of-sight direction, for example, the corneal reflection method is used. In the corneal reflection method, infrared light is irradiated onto the eyeball, and the line-of-sight direction is inferred using a reflection image (corneal reflection image) on the corneal surface and an image obtained by photographing the pupil.

[0007] In the eye tracking of such a VR head-mounted display, improving the detection accuracy is a major technical problem. Therefore, distinguishing a fine difference in the line-of-sight angle has become the key to line-of-sight detection.

[0008] In addition, similarly, this technical problem is the same in an electronic viewfinder (EVF: Electronic View Finder, hereinafter referred to as "EVF") of a camera.

[0009] In cameras, cameras with built-in electronic viewfinders (EVFs) have become practical. With such cameras, through the viewfinder, it is possible to observe the imaging through the lens onto a display device, i.e., an imaging element, provided inside the camera. Since in cameras with built-in EVFs, it is possible to observe an image in a state where photographic condition settings such as exposure and white balance have been made, it is possible to confirm what kind of shooting method will be used under the set photographic conditions before taking a photo.

[0010] Patent Document 1 discloses a display device in which the display screen of a display element that displays an image based on an input video signal and the light-receiving surface of an imaging element that photographs the user's eye are arranged on the same optical axis. During each predetermined unit period, the display element is made to operate during a first period within the unit period, and the imaging element is made to operate in place of the display element during a second period following the first period.

[0011] Patent Document 2 discloses an eye observation device and a glasses terminal equipped with the eye observation device. The eye observation device includes at least one infrared light source and at least one imaging device. The at least one infrared light source irradiates the user's eye with polarized infrared light, and the at least one imaging device photographs an image of the eye irradiated with the polarized infrared light and is capable of photographing polarized images in at least three directions simultaneously.

[0012] Prior Art Documents

[0013] Patent Documents

[0014] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2010-72188

[0015] Patent Document 2: International Publication No. WO 2017 / 014137 Summary of the Invention

[0016] Technical Problem to be Solved by the Invention

[0017] The display device disclosed in Patent Document 1 aims to improve the accuracy of viewpoint detection by arranging the display device and the imaging element on the same optical axis. However, when arranged on the same optical axis, it is impossible to arrange the display element in the viewpoint detection unit of the viewpoint detection device, and there is a problem that it is difficult to achieve high definition and high brightness of the display device.

[0018] The eye observation device disclosed in Patent Document 2 aims to improve the detection accuracy through polarization in three directions. However, the position of sensors such as the imaging device for detecting the pupil of the eye is not mentioned. Also, when there is a deviation in the position of the sensors such as the imaging device, there is a problem that the detection accuracy is inconsistent accordingly.

[0019] The present disclosure is proposed in view of such problems, and its purpose is to provide a line of sight detection device that can improve the detection accuracy in eye tracking, a manufacturing method of the line of sight detection device, and an eyeglass terminal and a camera having the line of sight detection device.

[0020] Solutions for solving technical problems

[0021] The present disclosure is proposed to solve the above-mentioned problems. In a first aspect, a sight line detection device is provided, comprising:

[0022] A display device that displays an image; and

[0023] The imaging element for detecting the user's viewpoint is arranged in close contact with the effective pixel area of ​​the display device and on the periphery of the effective pixel area.

[0024] In addition, in the first aspect, the display device may include:

[0025] Semiconductor substrates;

[0026] A display element is arranged on the semiconductor substrate; and

[0027] A light guide is arranged on the semiconductor substrate with the display element interposed therebetween, and seals the display element between the light guide and the semiconductor substrate.

[0028] In addition, in the first aspect, the image pickup element may be arranged on a short side of the outer periphery of the effective pixel region.

[0029] In addition, in the first aspect, the image pickup element may be arranged on a long side of the outer periphery of the effective pixel region.

[0030] In addition, in the first aspect, the image pickup element may be arranged at a corner portion of an outer periphery of the effective pixel region.

[0031] In the first aspect, one end of the imaging element may be placed on or bonded to the upper surface of the light guide, and the other end of the imaging element may be electrically connected to the semiconductor substrate via a height correction object having conductivity.

[0032] In the first aspect, one end of the imaging element may be bonded to the upper surface of the light guide by an adhesive, and the other end of the imaging element may be electrically connected to the semiconductor substrate via the height correction object by welding.

[0033] In addition, in the first aspect, the height correction object may be a conductive pillar.

[0034] Alternatively, in this first aspect, it is also possible that the height correction object is a silicon chip.

[0035] Alternatively, in this first aspect, it is also possible that the height correction object is an active element chip.

[0036] Alternatively, in this first aspect, it is also possible that the height correction object is an insulating substrate having TSVs.

[0037] Alternatively, in this first aspect, it is also possible that the height correction object is the insulating substrate having a driving substrate.

[0038] A second aspect thereof is a line-of-sight detection device, comprising:

[0039] A display device that seals the display element between a semiconductor substrate and a light guide and displays the formed image; and

[0040] An imaging element for detecting the user's viewing point, which is disposed on the semiconductor substrate in close contact with the light guide.

[0041] A third aspect thereof is a line-of-sight detection device, comprising:

[0042] A display device that displays an image;

[0043] An imaging element for detecting the user's viewing point; and

[0044] A substrate that disposes the display device and the imaging element in close contact with each other.

[0045] Alternatively, in the first to third aspects, it is also possible that the imaging element is electrically connected to the display device via the conductor.

[0046] Alternatively, in the first to third aspects, it is also possible that the electrical connection between the imaging element and the display device via the conductor is achieved by self-alignment by soldering.

[0047] Alternatively, in the first to third aspects, it is also possible that the normal of the imaging surface of the imaging element is arranged to face the same direction as the optical axis of the display device.

[0048] A fourth aspect thereof is a method for manufacturing a line-of-sight detection device, comprising:

[0049] A step of preparing a display device including a semiconductor substrate and a light guide;

[0050] A step of forming or bonding a height correction object having conductivity on the semiconductor substrate;

[0051] A step of forming protrusions on the height correction object;

[0052] The process of applying solder to the pads of the imaging element;

[0053] The process of placing the pads of the imaging element on the upper surface of the light guide body and the upper surface of the protrusion of the height correction object respectively; and

[0054] The process of soldering the imaging element to the upper surface of the light guide body and the upper surface of the protrusion by reflow soldering.

[0055] Its fifth aspect is a glasses terminal, having:

[0056] A line-of-sight detection device,

[0057] The line-of-sight detection device has:

[0058] A display device for displaying an image; and

[0059] An imaging element for detecting the user's viewing point, which is arranged close to the outer periphery of the effective pixel region of the display device and adjacent to the effective pixel region, or

[0060] The line-of-sight detection device has:

[0061] A display device for sealing a display element between a semiconductor substrate and a light guide body and displaying the formed image; and

[0062] An imaging element for detecting the user's viewing point, which is arranged close to the light guide body on the semiconductor substrate, or

[0063] The line-of-sight detection device has:

[0064] A display device for displaying an image;

[0065] An imaging element for detecting the user's viewing point; and

[0066] A substrate for arranging the display device and the imaging element in close contact.

[0067] Its sixth aspect is a camera, having:

[0068] A line-of-sight detection device,

[0069] The line-of-sight detection device has:

[0070] A display device for displaying an image; and

[0071] An imaging element for detecting the user's viewing point, which is arranged close to the outer periphery of the effective pixel region of the display device and adjacent to the effective pixel region, or

[0072] The line-of-sight detection device has:

[0073] A display device that seals a display element between a semiconductor substrate and a light guide and displays the formed image; and

[0074] An imaging element for detecting the user's viewing point, which is disposed on the semiconductor substrate in close contact with the light guide, or

[0075] The line-of-sight detection device includes:

[0076] A display device that displays an image;

[0077] An imaging element for detecting the user's viewing point; and

[0078] A substrate that disposes the display device and the imaging element in close contact with each other.

[0079] By adopting the above aspects, it is possible to provide a line-of-sight detection device capable of improving the detection accuracy in eye tracking, a manufacturing method of the line-of-sight detection device, a glasses terminal having the line-of-sight detection device, and a camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 It is an explanatory diagram (one) showing the relationship between the display device and the imaging element in the camera and the distance from the eyeball.

[0081] Figure 2 It is an explanatory diagram (two) showing the relationship between the display device and the imaging element in the camera and the distance from the eyeball.

[0082] Figure 3 It is an explanatory diagram (three) showing the relationship between the display device and the imaging element in the camera and the distance from the eyeball.

[0083] Figure 4 It is a top view of the first embodiment of the line-of-sight detection device according to the present disclosure.

[0084] Figure 5 It is a sectional view taken along line A-A of the first embodiment of the line-of-sight detection device according to the present disclosure.

[0085] Figure 6 It shows an example in Figure 4 where the imaging element is disposed on the left short side of the outer periphery thereof in close contact with the effective pixel region of the display device. It is a top view.

[0086] Figure 7 It shows an example in Figure 4 where the imaging element is disposed on the upper long side of the outer periphery thereof in close contact with the effective pixel region of the display device. It is a top view.

[0087] Figure 8 It shows an example in Figure 4A plan view showing an example in which an imaging element is arranged on the lower long side of the outer periphery thereof in close contact with the effective pixel region of the display device.

[0088] Figure 9 It shows in Figure 4 A plan view showing an example in which an imaging element is arranged at the lower right corner of the outer periphery thereof in close contact with the effective pixel region of the display device.

[0089] Figure 10 It shows in Figure 4 A plan view showing an example in which an imaging element is arranged at the lower left corner of the outer periphery thereof in close contact with the effective pixel region of the display device.

[0090] Figure 11 A partial enlarged view of the A-A line cut end face view of the second embodiment of the line-of-sight detection device according to the present disclosure.

[0091] Figure 12 A partial enlarged view of the A-A line cut end face view before the imaging element of the second embodiment of the line-of-sight detection device according to the present disclosure is joined to the display device.

[0092] Figure 13 An explanatory view of the manufacturing process of the second embodiment of the line-of-sight detection device according to the present disclosure.

[0093] Figure 14 A partial enlarged view of the A-A line cut end face view of the basic form of the third embodiment of the line-of-sight detection device according to the present disclosure.

[0094] Figure 15 A partial enlarged view of the A-A line cut end face view of a modified example of the third embodiment of the line-of-sight detection device according to the present disclosure.

[0095] Figure 16 A partial enlarged view of the A-A line cut end face view of the fourth embodiment of the line-of-sight detection device according to the present disclosure.

[0096] Figure 17 A partial enlarged view of the A-A line cut end face view of the fifth embodiment of the line-of-sight detection device according to the present disclosure.

[0097] Figure 18 A partial enlarged view of the A-A line cut end face view of the sixth embodiment of the line-of-sight detection device according to the present disclosure.

[0098] Figure 19 A partial enlarged view of the A-A line cut end face view of the seventh embodiment of the line-of-sight detection device according to the present disclosure.

[0099] Figure 20It is a partial enlarged view of the A-A line cut end face view of the eighth embodiment of the line-of-sight detection device related to the present disclosure.

[0100] Figure 21 It is a schematic configuration diagram of a glasses terminal having the line-of-sight detection device related to the present disclosure.

[0101] Figure 22 It is a block diagram of a glasses terminal having the line-of-sight detection device related to the present disclosure.

[0102] Figure 23 It is a schematic configuration diagram of a camera having the line-of-sight detection device related to the present disclosure. Detailed Embodiments

[0103] Next, with reference to the accompanying drawings, the modes for implementing the line-of-sight detection device related to the present disclosure, the manufacturing method of the line-of-sight detection device, and the glasses terminal and camera having the detection device (hereinafter referred to as "embodiments") will be described in the following order. In the following drawings, the same or similar parts are labeled with the same or similar reference numerals. However, the drawings are schematic, and the ratios of the dimensions of each part are not necessarily the same as those in reality. In addition, of course, there are also parts where the dimensional relationships and ratios are different between the drawings.

[0104] 1. First Embodiment of the Line-of-Sight Detection Device Related to the Present Disclosure

[0105] 2. Second Embodiment of the Line-of-Sight Detection Device Related to the Present Disclosure

[0106] 3. Manufacturing Process of the Second Embodiment of the Line-of-Sight Detection Device Related to the Present Disclosure

[0107] 4. Third Embodiment of the Line-of-Sight Detection Device Related to the Present Disclosure

[0108] 5. Fourth Embodiment of the Line-of-Sight Detection Device Related to the Present Disclosure

[0109] 6. Fifth Embodiment of the Line-of-Sight Detection Device Related to the Present Disclosure

[0110] 7. Sixth Embodiment of the Line-of-Sight Detection Device Related to the Present Disclosure

[0111] 8. Seventh Embodiment of the Line-of-Sight Detection Device Related to the Present Disclosure

[0112] 9. Eighth Embodiment of the Line-of-Sight Detection Device Related to the Present Disclosure

[0113] 10. Glasses Terminal Having the Line-of-Sight Detection Device Related to the Present Disclosure

[0114] 11. Camera Having the Line-of-Sight Detection Device Related to the Present Disclosure

[0115] <1. First Embodiment of the Gaze Detection Device of the Present Disclosure>

[0116] [Relationship between the Display Device and the Imaging Element of the Gaze Detection Device Constituting the Camera and the Distance from the Eyeball]

[0117] Figures 1 to 3 It is an explanatory diagram showing the relationship between the display device 20 and the imaging element 10 of the gaze detection device 100 constituting the camera 300 and the distance from the eyeball 400. Figure 1 It shows the distance X1 from the lens 302 of the camera 300 that enables the imaging element 10 to observe the center of the eyeball 400 when the imaging element 10 is arranged at a position (Y1) 2 mm from the end of the effective pixel region 24 of the display device 20 for displaying images and the like.

[0118] Similarly, Figure 2 It shows the distance X2 from the lens 302 that enables the imaging element 10 to observe the center of the eyeball 400 when the imaging element 10 is arranged at a position (Y2) 1 mm from the end of the effective pixel region 24 of the display device 20.

[0119] Similarly, Figure 3 It shows the distance X3 from the lens 302 that enables the imaging element 10 to observe the center of the eyeball 400 when the imaging element 10 is arranged at a position (Y3) 0.5 mm from the end of the effective pixel region 24 of the display device 20.

[0120] Comparing Figures 1 to 3 the distances X1, X2, and X3 from the lens 302 of each camera 300 that enables the imaging element 10 to observe the center of the eyeball 400, it can be seen that there is a relationship of X1 < X2 < X3. That is, it can be known that if the imaging element 10 is arranged closer to the effective pixel region 24 of the display device 20, the distances X1, X2, and X3 that can observe the eyeball 400 become longer in this order.

[0121] Conversely, it can be known that if the interval between the effective pixel region 24 of the display device 20 and the imaging element 10 becomes farther, there will be a problem that the center of the eyeball 400 cannot be captured by the imaging element 10 at a position farther from the display device 20. Therefore, in order to accurately detect the movement of the eyeball 400, it is necessary to arrange the imaging element 10 as close as possible to the effective pixel region 24 of the display device 20, and it is more preferable to arrange it closely.

[0122] It can be known that even in the glasses terminal 200 described later, similar to the case of the camera 300 shown in Figures 1 to 3 it is necessary to make the imaging element 10 as close as possible to the effective pixel region 24 of the display device 20, and further, it is necessary to arrange it closely.

[0123] Here, in eye tracking, as described above, the corneal reflection method is generally used. In the corneal reflection method, for example, infrared rays or near-infrared rays are irradiated onto the eyeball 400, and the reflected image on the corneal surface or the image obtained by photographing the pupil is used to infer the line-of-sight direction. In the corneal reflection method, infrared light is irradiated onto the eyeball 400, and the reflected image (corneal reflection image) on the corneal surface and the image obtained by infrared photographing of the pupil are used to infer the line-of-sight direction. Specifically, by calculating the rotation angle of the eyeball 400 based on the position of the corneal reflection image, the position of the pupil center, and the dimensional relationship of each part of the eyeball 400, etc., the line of sight can be detected. Therefore, in the EVF for the glasses terminal 200 and the camera 300, in order to accurately calculate the rotation angle, differentiating the minute angular difference of the line of sight becomes the key to line-of-sight detection.

[0124] Here, in order to accurately calculate the angle of the line of sight, the arrangement positions (the distance between the two) of the display device 20 and the imaging element 10 serving as the reference thereof must be accurate. That is, there are high-precision requirements for the arrangement dimensions of the display device 20, the imaging element 10, etc. constituting the line-of-sight detection device 100.

[0125] As one method for improving the calculation accuracy of the line-of-sight angle, for example, there is a method of reducing errors by providing a plurality of light sources composed of a large number of light-emitting diodes, etc. and image sensors such as solid-state imaging devices. However, arranging a large number of such components will increase the cost and the device will become complicated.

[0126] The line-of-sight detection device 100 according to the present disclosure is a device that can achieve the following, that is, by bringing the distance between the imaging element 10 and the effective pixel region 24 of the display device 20 close and arranging them in contact, the angular difference between the line-of-sight direction and the line-of-sight detection direction is reduced, and by arranging the positions of the imaging element 10 and the display device 20 with high precision, line-of-sight detection is performed with high precision.

[0127] [Configuration of the First Embodiment]

[0128] Figure 4 is a top view of the first embodiment of the line-of-sight detection device 100 according to the present disclosure. In addition, Figure 5 is Figure 4 a partial enlarged view of the cut end face along the line A-A in Figure 4 and Figure 5 As shown in

[0129] Specifically, as shown in Figure 4As shown, a sealing agent 22 is coated on the periphery of a predetermined area on a semiconductor substrate 30 to form a sealed area 23 surrounded by the sealing agent 22. And, within the sealed area 23 surrounded by this sealing agent 22, a display element 25 composed of a liquid crystal or an organic light-emitting material is arranged. That is, as Figure 5 shown, the display element 25 is sealed between the semiconductor substrate 30 and the light guide 21 joined by the sealing agent 22. The display screen of the display element 25 within the sealed area 23 forms an effective pixel area 24.

[0130] Here, the light guide 21 transmits the light from the display element 25 and is a protective material that protects the display element 25. For example, it is formed of a transparent material having light transmittance such as glass.

[0131] However, the material of the light guide 21 only needs to be a light-guiding material and is not limited to glass. In the following, regarding the example where the light guide 21 is formed of glass, the light guide 21 will be described as "glass 21".

[0132] As Figure 4 shown, on the upper surface of the glass 21, the imaging element 10 is disposed closely at approximately the central portion of the right short side of the outer periphery of the effective pixel area 24. That is, the area for disposing the imaging element 10 is a pixel non-display area on the upper surface of the glass 21 that becomes the outer periphery of the effective pixel area 24 and does not overlap with the effective pixel area 24.

[0133] The imaging element 10 is an image sensor that images an image of the eyeball 400 etc. As Figure 5 shown, it includes a predetermined color filter 12 and a lens array 13 on the imaging surface. It should be noted that the imaging element 10 can also be a CMOS image sensor for imaging used in a general camera etc. Additionally, as Figure 4 shown, the imaging element 10 is disposed at approximately the central portion of the right end of the upper surface of the glass 21, but its disposed position is not specified at this position and can be any position. Additionally, the imaging element 10 does not need to be disposed to converge within the upper surface of the glass 21 and can also be disposed to protrude from the upper surface of the glass 21 to the upper surface of the semiconductor substrate 30.

[0134] As Figure 5 shown, near the imaging element 10, the light source 90 is disposed at a position on the side that will not become an obstacle to imaging. The light source 90 uses, for example, an infrared light-emitting element. The light source 90 irradiates the eyeball 400 with the light source light 91 and forms an imaging image including its reflected light 92 on the light-receiving surface of the imaging element 10.

[0135] Therefore, the imaging surface of the imaging element 10 and the display device 20 are both arranged to face the direction of the eyeball 400. That is, the normal line of the imaging surface of the imaging element 10 is arranged to face the same direction as the optical axis of the normal line that is the center of the display device 20.

[0136] In addition, in this case, preferably, the imaging element 10 and the display device 20 are arranged such that the normal line of the imaging surface of the imaging element 10 and the optical axis of the display device 20 are as parallel as possible.

[0137] As Figure 5 shown, the imaging element 10 is connected to an image control unit 220 of a glasses terminal 200 described later, for example, via a connection line 93. And, as Figure 4 shown, an external terminal 31 is formed on the right side of the upper surface of the semiconductor substrate 30, and the display device 20 is connected to the image control unit 220 via the external terminal 31 through a connection line 94.

[0138] In addition, the connection line 93 of the imaging element 10 may not be directly connected to the image control unit 220, but may be first connected to the semiconductor substrate 30 and then connected to the image control unit 220 together with the display device 20 via the external terminal 31 through the connection line 94.

[0139] The image control unit 220 outputs an image for the user to use to the image display device 20 based on the imaging signal from the imaging element 10 and a video signal 96 from an external entertainment device (not shown), etc.

[0140] Thus, the user of the glasses terminal 200 having the line-of-sight detection device 100 described later can observe a realistic image corresponding to the movement of the line of sight. In addition, in addition to displaying the image, a stereo field is provided using a separately prepared speaker (not shown), etc., so that the user can enter a virtual space, a virtual reality world, and enjoy that world.

[0141] The first embodiment of the line-of-sight detection device 100 according to the present disclosure is configured as described above, and since the display device 20 and the imaging element 10 are joined by self-alignment of welding described later, a high-precision line-of-sight detection device 100 can be provided at low cost.

[0142] [First Variation of the First Embodiment]

[0143] Next, a first variation of the first embodiment of the line-of-sight detection device 100 according to the present disclosure will be described. In this first variation, as Figure 6 shown, the imaging element 10 is used as Figure 4 or Figure 5In the upper surface of the glass 21 at the position opposite to the shown position, it is arranged at the approximate center of the left short side that closely adheres to the outer periphery of the effective pixel region 24, which is different from the basic form of the first embodiment. It should be noted that the position for arranging the imaging element 10 only needs to be a short side on the upper surface of the glass 21, and is not limited to the approximate center of the short side.

[0144] Except for the above, since it is the same as the basic form of the first embodiment, the description is omitted.

[0145] [Second Variant of the First Embodiment]

[0146] Next, a second variant of the first embodiment of the line-of-sight detection device 100 according to the present disclosure will be described. In this second variant, as Figure 7 shown, the imaging element 10 is arranged at the approximate center of the upper long side that closely adheres to the outer periphery of the effective pixel region 24 on the upper surface of the glass 21, or as Figure 8 shown, the imaging element 10 is arranged at the approximate center of the lower long side that closely adheres to the outer periphery of the effective pixel region 24 on the upper surface of the glass 21, which is different from the basic form of the first embodiment. It should be noted that the position for arranging the imaging element 10 only needs to be a long side on the upper surface of the glass 21, and is not limited to the approximate center of the long side.

[0147] Except for the above, since it is the same as the basic form of the first embodiment, the description is omitted.

[0148] [Third Variant of the First Embodiment]

[0149] Next, a third variant of the first embodiment of the line-of-sight detection device 100 according to the present disclosure will be described. In this third variant, as Figure 9 shown, the imaging element 10 is arranged at the lower right corner that closely adheres to the outer periphery of the effective pixel region 24 on the upper surface of the glass 21, or as Figure 10 shown, the imaging element 10 is arranged at the lower left corner that closely adheres to the outer periphery of the effective pixel region 24 on the upper surface of the glass 21, which is different from the basic form of the first embodiment. It should be noted that the imaging element 10 can also be arranged at the upper right corner or the upper left corner of the upper surface of the glass 21.

[0150] Except for the above, since it is the same as the basic form of the first embodiment, the description is omitted.

[0151] <2. Second Embodiment of the Line-of-Sight Detection Device According to the Present Disclosure>

[0152] [Configuration of the Second Embodiment]

[0153] Next, a second embodiment of the line-of-sight detection device 100 according to the present disclosure will be described. In the second embodiment, as Figure 11 shown, a support pillar 60 is formed near the right side surface of the glass 21. In a state where the support pillar 60 is not provided, if the imaging element 10 is to be welded to both the upper surface of the glass 21 and the upper surface of the semiconductor substrate 30, welding cannot be directly performed due to the different heights of the two. The support pillar 60 serves as a height corrector for aligning the welding surfaces to the same height and also serves as a conductor for electrically connecting the imaging element 10 and the semiconductor substrate 30.

[0154] The support pillar 60 is, for example, a substantially round bar-shaped column formed of a conductive metal such as copper (Cu). However, the shape of the support pillar 60 is not limited to a substantially round bar shape and may also be a substantially square bar shape, and the cross section may be substantially triangular, substantially hexagonal, substantially octagonal, or substantially elliptical, etc.

[0155] In addition, the height of the support pillar 60 can be set to any value. Therefore, for example, by forming the overall height of the support pillar 60 to be the same size as the height (plate thickness) of the glass 21, the welding height of the imaging element 10 can be aligned. Thereby, welding can be performed accurately and easily.

[0156] As Figure 11 shown, in the imaging element 10, one end, that is, the left end, is arranged at the right end of the upper surface of the glass 21, and the other end, that is, the right end, is arranged at the top of the support pillar 60. That is, the imaging element 10 is arranged on the right short side of the outer periphery of the effective pixel region 24 in contact with the upper surface of the glass 21 in such a manner as to exactly span the right end of the upper surface of the glass 21 and the top of the support pillar 60.

[0157] In addition, pads 10a, 10a are formed at the left end and the right end of the lower surface of the imaging element 10. Moreover, the left pad 10a is welded to the right end of the upper surface of the glass 21 by solder 70a, and the right pad 10a is welded to the top of the support pillar 60 by solder 70a. It should be noted that since a protrusion 60a formed by solder, which will be described later, is formed at the top of the support pillar 60, good welding can be performed.

[0158] The solder 70a uses, for example, lead-free tin-silver (Sn-Ag) type paste solder. In addition, welding is performed through a reflow soldering process. The welding based on this reflow soldering process is also performed in the same manner in the aforementioned first embodiment. In addition, thereby, the imaging element 10 and the semiconductor substrate 30 are electrically connected via the conductive support pillar 60.

[0159] Regarding the configurations other than the above, since they are the same as those of the first embodiment, the description thereof is omitted. In addition, the reflow soldering process will be described later.

[0160] [Principle of Calibration of Imaging Element]

[0161] Here, when the imaging element 10 is arranged to straddle the right end of the upper surface of the glass 21 and the top of the support column 60, as Figure 12 shown, there may be a positional deviation Z between the pad 10a on the lower surface of the imaging element 10 and the top of the support column 60. And if bonding is directly performed in a state where the positional deviation Z has occurred, since the positional relationship between the display device 20 and the imaging element 10 becomes inaccurate, an error will occur in the line-of-sight detection.

[0162] Therefore, in the second embodiment of the line-of-sight detection device 100 according to the present disclosure, as Figure 12 shown, it is configured such that a substantially flat hemispherical protrusion 60a formed of solder is formed on the top of the support column 60, and solders 70a, 70a are respectively coated on the pads 10a, 10a at the left and right ends of the lower surface of the imaging element 10 or on the upper surface of the right end of the glass 21 and the upper surface of the protrusion 60a, and welding is performed through a reflow soldering process.

[0163] Here, in Figure 11 and Figure 12 , it is configured such that a protrusion 60a is provided on the top of the support column 60, a metal layer (not shown) is formed on the upper surfaces of the protrusion 60a and the glass 21 by evaporation plating or the like, and it is bonded to the imaging element 10 through solders 70a, 70a (not shown). However, since there is no need for any electrical connection between the imaging element 10 and the glass 21, the two do not necessarily need to be welded. Therefore, the bonding of the imaging element 10 and the glass 21 is not limited to the Figure 11 and Figure 12 shown configuration, and various configurations can be adopted. For example, a protrusion 60a may be provided on the top of the support column 60, and a protrusion equivalent to the protrusion 60a may also be provided on the upper surface of the glass 21, and each protrusion is bonded to the imaging element 10 through solders 70a, 70a. Additionally, it may be that a protrusion 60a is provided on the top of the support column 60, and a protrusion equivalent to the protrusion 60a is also provided on the upper surface of the glass 21, but only the protrusion 60a on the top of the support column 60 is bonded to the imaging element 10 through the solder 70a. Additionally, it may be configured such that a protrusion 60a is provided on the top of the support column 60, no protrusion is provided on the upper surface of the glass 21, and only the protrusion 60a on the top of the support column 60 is bonded to the imaging element 10 through the solder 70a.

[0164] As described above, when the imaging element 10 is placed on the top of the support column 60, a positional deviation Z sometimes occurs between the pad 10a on the lower surface of the imaging element 10 and the support column 60. However, even if such a positional deviation Z occurs, if welding is performed through a reflow soldering process, the positional deviation Z will be automatically corrected by the self-alignment effect of the molten solder, and welding will be performed at the specified position. This self-alignment is also referred to as automatic alignment.

[0165] The principle of self-alignment is that the paste solder 70a melts when heated during the reflow soldering process of welding. Due to its surface tension, the molten solder 70a will act with a force that minimizes its surface area. Through the action of this force, the imaging element 10 moves to the specified position, and the positional deviation Z is eliminated. As a result, the imaging element 10 is welded at the specified position.

[0166] That is, in the case where there is a positional deviation Z, first, in this state, the imaging element 10 is melt-bonded to the right end of the upper surface of the glass 21 and the top of the support column 60 through the solder 70a. However, since the state of the molten solder 70a with a larger surface area is maintained while keeping the state with the positional deviation Z unchanged, a force that will move the imaging element 10 to a position where the surface area becomes smaller due to surface tension and there is no positional deviation Z will act. Through the action of such a force, as Figure 11 shown, welding can be performed at the specified position. Thus, the imaging element 10 can be configured at the predetermined position with high precision.

[0167] The above is the principle of the configuration of the line-of-sight detection device 100 and the arrangement of the imaging element 10 in the second embodiment that can be performed with high precision. It should be noted that the correction function of the positional deviation Z achieved by the self-alignment of the solder can be applied not only in the aforementioned first embodiment but also in each of the embodiments described below.

[0168] It should be noted that the details of the reflow soldering process of welding will be described in the manufacturing process of the second embodiment described later.

[0169] <3. Manufacturing Process of the Second Embodiment of the Line-of-Sight Detection Device According to the Present Disclosure>

[0170] Next, the manufacturing process of the second embodiment of the line-of-sight detection device 100 according to the present disclosure will be described. In the manufacturing process of the second embodiment, as Figure 13As shown, first, the display device 20 is initially prepared. Specifically, the display element 25 is sealed between the semiconductor substrate 30 and the glass 21 to form the display device 20. The display element 25 is sealed in the active pixel region 24 formed by coating the sealant 22 on the peripheral surface of the semiconductor substrate 30 that surrounds the region for arranging the display element 25 in a manner sandwiched between the semiconductor substrate 30 and the glass 21.

[0171] In the process of preparing the display device 20, the production of the display device 20 can be carried out using the prior art (step S01).

[0172] Next, the pillar 60, which is a height corrector having conductivity, is formed on the semiconductor substrate 30. The pillar 60 is formed on the semiconductor substrate 30 by the semi-additive method. In the process based on the semi-additive method, first, a metal layer is thinly formed on the entire surface of the semiconductor substrate 30. Next, the surface other than the region for forming the pillar 60 is covered with a resist. In this state, copper (Cu) is deposited to a predetermined film thickness in the region for forming the pillar 60, and then a solder layer is thinly deposited on its upper surface by plating. Then the resist is removed. Further, the thin metal layer initially formed on the surface from which the resist has been removed is removed. After that, heat near the melting point of the solder layer is applied and then cooled.

[0173] By going through such a process, the pillar 60 with the projection 60a having a predetermined height (thickness) formed of copper (Cu) and a substantially flat hemispherical solder layer protruding upward formed on its top can be formed (steps S02, S03).

[0174] It should be noted that the pillar 60 can also be formed through the following process. For example, the pre-formed copper (Cu) pillar 60 is mounted on a dedicated jig (not shown) and placed on the semiconductor substrate 30 in an erected state. Then, the pillar 60 is joined to the semiconductor substrate 30 by soldering or the like. Alternatively, the pillar 60 can be placed one by one at a predetermined position by a mounter (not shown) and joined by soldering or the like (step S02).

[0175] Next, the projection 60a formed of solder is formed on the top of the pillar 60. The projection 60a can be formed by inverting the semiconductor substrate 30 with the pillar 60 erected up and down and dipping the top of the pillar 60 into the solder bath (step S03).

[0176] In the foregoing (step S02) and (step S03), examples of forming the pillar 60 are described, but in this step, as long as a height corrector having conductivity is formed or joined, it is not limited to the pillar 60. For example, it can also be the silicon chip 65, the active element chip 40, etc. described later.

[0177] In addition, in other processes, the imaging element 10 is prepared. The imaging element 10 has pads 10a for soldering or projections, terminals, etc. equivalent thereto, and can be prepared, for example, through the manufacturing process of a general CMOS image sensor or the like. Therefore, detailed description is omitted (step S04).

[0178] Next, paste solder 70a, 70a is applied onto the pads 10a, 10a on the lower surface of the imaging element 10, the upper surface of the right end of the glass 21, and the upper surface of the projection 60a (step S05).

[0179] Then, the imaging element 10 is placed such that the pads 10a, 10a exactly straddle the upper surface of the glass 21 and the upper surface of the projection 60a at the top of the support column 60, and is placed on the right short side of the outer periphery in close contact with the effective pixel region 24 on the upper surface of the glass 21 (step S06).

[0180] Next, the imaging element 10 is soldered to the upper surface of the glass 21 and the projection 60a of the support column 60. Soldering is performed by melting the paste solder 70a, 70a in a reflow soldering process.

[0181] In this case, in the soldering on the upper surface of the glass 21, a metal layer (not shown in each figure) is formed in advance on the portion of the upper surface of the glass 21 for soldering by means of evaporation or the like. Alternatively, pads equivalent to the pads 60a may be formed in advance as described above. It should be noted that since no electrical connection is required between the imaging element 10 and the glass 21, the two may not be soldered. Therefore, when the two are not soldered, the pads 10a of the imaging element 10 opposite to the upper surface of the glass 21 may not be formed. Or, even if formed, the solder 70a may not be applied to the pads 10a. Whether such soldering between the imaging element 10 and the upper surface of the glass 21 is required is the same in other embodiments (step S07).

[0182] The soldering achieved through the reflow soldering process is specifically carried out as follows. That is, in a state where the imaging element 10 is placed on a predetermined position of the display device 20 via the paste solder 70a, 70a, the display device 20 is placed on a conveyor (not shown) and passed through a reflow soldering furnace (not shown).

[0183] In a reflow soldering furnace, temperature management is performed according to a predetermined temperature curve. For example, a process of preheating is carried out by making the surface temperature of the imaging element 10 approximately 180 to 190 °C for about 60 to 120 seconds. Next, the temperature is gradually increased, and a process is carried out in which the state where the surface temperature of the imaging element 10 is approximately 230 to 260 °C (peak temperature) is maintained for about 30 to 50 seconds. Thereby, the paste-like solders 70a, 70a are melted, and soldering with self-alignment is performed on the imaging element 10 and the display device 20.

[0184] Then, a process of reducing the temperature in the reflow soldering furnace is carried out, and thereby the melted solders 70a, 70a solidify. Thus, as Figure 11 shown, the imaging element 10 is soldered to the top of the support column 60 and the upper surface of the glass 21 through the solders 70a, 70a.

[0185] It should be noted that, precisely speaking, since the temperature curve is different for each imaging element 10, prior test confirmation and temperature condition setting are required. The soldering is managed by appropriately setting the temperature curve.

[0186] Therefore, the temperature, heating time, etc. of the reflow soldering process described above are only examples, and are set to the optimal values according to the object to be soldered. It should be noted that, in the case where the solder 70a is not a no-clean solder, etc., cleaning is performed after soldering.

[0187] By going through the soldering process as described above, the imaging element 10 can be soldered to the upper surface of the glass 21 and the top of the support column 60.

[0188] It should be noted that, during soldering, in order to reduce the thermal stress on the imaging element 10, etc., it is preferable to use a low-temperature solder. A low-temperature solder (low-melting-point solder) generally refers to a solder having a melting point lower than that of a eutectic solder (tin (Sn) 63%, lead (Pb) 37%) (below 184 °C). In the case of using such a low-temperature solder, the peak temperature of the temperature curve can be reduced.

[0189] In addition, from the viewpoint of environmental protection, lead-free solders have been widely used in recent years. As the name implies, a lead-free solder is a solder that does not contain lead (Pb). The main component of the lead-free solder is tin (Sn), and is configured to contain other metals such as silver (Ag) and copper (Cu) in the tin.

[0190] Regarding the solder 70a used in the soldering of the imaging element 10 according to each embodiment of the present disclosure, it is assumed to be a Sn-Ag-based lead-free solder, but it can be selected or combined according to the actual situation, and is not limited to using the Sn-Ag-based lead-free solder. Therefore, whether it is a lead-free solder or a leaded solder, or a low-temperature solder or a high-temperature solder.

[0191] It should be noted that the following situation is envisaged, that is, when passing through the reflow soldering furnace, in addition to the melting of the solders 70a and 70a, the solder of the component on the lower layer that has been soldered in other processes will also melt simultaneously. In such a case, solders with different melting points can also be used for the solder on the lower layer and the solder on the upper layer. Alternatively, it can also be configured to manage the temperature in the reflow soldering furnace and the conveying speed of the conveyor so that although solders with the same melting point are used, by setting a temperature gradient in the vertical direction, the solder of the component on the lower layer is not melted, and only the solder on the upper layer is melted.

[0192] After soldering, as described above, if necessary, cleaning is performed and a predetermined inspection process is passed, whereby a line-of-sight detection device 100 in which the imaging element 10 is arranged on the display device 20 including the semiconductor substrate 30 can be obtained.

[0193] It should be noted that the manufacturing processes in the following-described embodiments are based on the manufacturing process of this embodiment and can be realized by appropriately adding changes corresponding to each embodiment.

[0194] <4. Third Embodiment of the Line-of-Sight Detection Device According to the Present Disclosure>

[0195] [Configuration of the Basic Form of the Third Embodiment]

[0196] Next, the basic form of the third embodiment of the line-of-sight detection device 100 according to the present disclosure will be described. In this basic form, as Figure 14 shown, a silicon chip 65 is arranged near the right side surface of the glass 21 instead of forming the pillar 60 to serve as a height correction object and a conductor.

[0197] In the silicon chip 65, pads 65b and 65b formed on its lower surface are soldered to pads 30a and 30a formed on the semiconductor substrate 30 through solders 70b and 70b.

[0198] The silicon chip 65 is, for example, a substantially square conductor formed of silicon (Si). However, the shape of the silicon chip 65 is not limited to a substantially square shape, and it can also be substantially cylindrical, and the cross-section of the horizontal plane can also be substantially triangular, substantially hexagonal, substantially octagonal, or substantially elliptical, etc.

[0199] In addition, the height (thickness) of the silicon chip 65 can be set to any value. Therefore, for example, by forming the overall height when pads 65a are provided on the upper surface of the silicon chip 65 to be the same size as the height (plate thickness) of the glass 21, the soldering height of the imaging element 10 can be aligned. Thereby, soldering can be performed accurately and easily.

[0200] The left end of the imaging element 10 is disposed at the right end of the upper surface of the glass 21, and its right end is disposed at the upper surface of the silicon chip 65. That is, the imaging element 10 is disposed in such a manner as to exactly span the right end of the upper surface of the glass 21 and the upper surface of the silicon chip 65, and is disposed on the right short side of the outer periphery in close contact with the effective pixel region 24 in the upper surface of the glass 21.

[0201] Moreover, pads 65a are formed in the upper surface of the silicon chip 65. In addition, pads 10a, 10a are respectively formed at the left end and the right end of the lower surface of the imaging element 10. The pad 10a at the left end of the imaging element 10 is bonded to the upper surface of the glass 21 by an adhesive 71, and the pad 10a at the right end is soldered to the pad 65a on the upper surface of the silicon chip 65 by a solder 70a.

[0202] As described above, soldering is performed by a reflow process using the paste-like solder 70a. In this case, the adhesive 71 uses an adhesive formed of a thermoplastic material. Thus, when the paste-like solder 70a melts in the reflow process and the position deviation Z is corrected by self-alignment, since the adhesive 71 also melts by heating, it does not hinder the self-alignment. In addition, the imaging element 10 is electrically connected to the semiconductor substrate 30 via the conductive silicon chip 65.

[0203] In addition, since the height (thickness) of the silicon chip 65 can be set to an arbitrary value, for example, by forming it to have the same height as the height (plate thickness) of the glass 21, the soldering height of the imaging element 10 can be aligned as a height correction object. Thus, soldering can be performed accurately and easily.

[0204] Regarding the configuration other than the above, since it is the same as the configuration of the second embodiment described above, the description thereof is omitted.

[0205] In addition, regarding the manufacturing process, it is also basically the same as that of the second embodiment described above. Regarding the differences, since they can be achieved by adding appropriate changes, the description thereof is omitted.

[0206] [Configuration of the modified example of the third embodiment]

[0207] Next, a modified example of the third embodiment of the line-of-sight detection device 100 according to the present disclosure will be described. In this modified example, as Figure 15 shown, the silicon chip 65 is disposed near the right side surface of the glass 21 instead of forming the support column 60 to serve as a height correction object and a conductor, which is the same as the basic form of the third embodiment.

[0208] However, the pad 10a at the left end of the imaging element 10 is not bonded to the upper surface of the glass 21 by the adhesive 71, but is soldered by the solder 70a, which is different from the basic form of the third embodiment.

[0209] Regarding the configuration other than the above, since it is the same as the configuration of the basic form of the aforementioned third embodiment, the description thereof is omitted.

[0210] In addition, regarding the manufacturing process, it is also basically the same as that of the aforementioned second embodiment. Regarding the differences, since they can be achieved by adding appropriate modifications, the description thereof is omitted.

[0211] <5. Fourth Embodiment of the Gaze Detection Device According to the Present Disclosure>

[0212] Next, a fourth embodiment of the gaze detection device 100 according to the present disclosure will be described. In this embodiment, as Figure 16 shown, an active element chip 40 is arranged near the right side surface of the glass 21 instead of arranging the silicon chip 65 to serve as a height correction object and a conductor, which is different from the second embodiment or the third embodiment.

[0213] The active element chip 40 is, for example, a display driver IC (Display Driver IC) that drives the display device 20, and inputs a drive signal from the semiconductor substrate 30 to drive the display device 20.

[0214] In the imaging element 10, its left end is arranged at the right end of the upper surface of the glass 21, and its right end is arranged on the upper surface of the active element chip 40. That is, the imaging element 10 is arranged so as to exactly straddle the right end of the upper surface of the glass 21 and the upper surface of the active element chip 40, and is arranged on the right short side of the outer periphery closely adjacent to the effective pixel region 24 on the upper surface of the glass 21.

[0215] In addition, for the active element chip 40, the active element chip 40 is joined to the semiconductor substrate 30 via its pads 40b, 40b, solders 70b, 70b, and pads 30a, 30a. In addition, on the upper surface of the active element chip 40, the imaging element 10 is joined via its pad 40a, solder 70a, and pad 10a, and a CoC (Chip on Chip) structure is formed as a whole.

[0216] That is, compared with Figure 15Similarly, in the imaging element 10, its pads 10a, 10a are respectively soldered to the upper surface of the glass 21 and the pads 40a on the upper surface of the active element chip 40 through solders 70a, 70a. Additionally, in the active element chip 40, its pads 40b, 40b are respectively soldered to the pads 30a, 30a on the upper surface of the semiconductor substrate 30 through solders 70b, 70b. Further, for example, a through-silicon via (TSV) (not shown) is provided penetrating through the active element chip 40. Thus, the pad 10a of the imaging element 10 is electrically connected to the pad 30a of the semiconductor substrate 30 via this through-silicon via (TSV).

[0217] Regarding the configurations other than the above, since they are the same as those of the modified example of the foregoing third embodiment, the description thereof is omitted.

[0218] In addition, regarding the manufacturing process, it is also basically the same as that of the foregoing second embodiment. Regarding the differences, since they can be achieved by making appropriate changes, the description thereof is omitted.

[0219] <6. Fifth Embodiment of the Line-of-Sight Detection Device According to the Present Disclosure>

[0220] Next, the basic form of the fifth embodiment of the line-of-sight detection device 100 according to the present disclosure will be described. In the basic form of this embodiment, as Figure 17 shown, an insulating substrate 68 is arranged near the right side surface of the glass 21 instead of arranging the silicon chip 65 and the active element chip 40 to serve as a height correction object and a conductor, which is different from the second to fourth embodiments.

[0221] The insulating substrate 68 is, for example, an insulator formed of a thermosetting resin and is formed in a substantially square shape. However, the shape of the insulating substrate 68 is not limited to a substantially square shape and may also be a substantially cylindrical shape, and the horizontal cross-section may also be a substantially triangular shape, a substantially hexagonal shape, a substantially octagonal shape, a substantially elliptical shape, or the like.

[0222] Further, in the insulating substrate 68, a through electrode (TMV: Through Mold Via) 68a is provided penetrating through at a predetermined position where it is joined to the pad 10a of the imaging element 10. The through electrode 68a is a through hole that penetrates the upper and lower portions of the insulating substrate 68 and whose inner peripheral surface is covered with a conductive metal such as copper (Cu).

[0223] In the imaging element 10, its left end is arranged at the right end of the upper surface of the glass 21, and its right end is arranged at the upper surface of the insulating substrate 68. That is, the imaging element 10 is arranged on the right short side of the outer periphery of the upper surface of the glass 21 in such a manner as to just straddle the right end of the upper surface of the glass 21 and the upper surface of the insulating substrate 68 and closely contact the effective pixel area 24.

[0224] The upper end of the through electrode 68a is slightly protruded on the upper surface of the insulating substrate 68 to form a pad, and the lower end of the through electrode 68a is slightly protruded on the lower surface thereof to form a pad. In addition, pads 10a and 10a are formed at the left and right ends of the lower surface of the imaging element 10, respectively. The pad 10a at the left end of the imaging element 10 is soldered to the upper surface of the glass 21 by solder 70a, and the pad 10a at the right end is soldered to the upper end of the through electrode 68a by solder 70a.

[0225] Likewise, the lower end of the through-electrode 68 a of the insulating substrate 68 is soldered to the pad 30 a on the semiconductor substrate 30 by solder 70 b .

[0226] Therefore, the pad 30 a of the semiconductor substrate 30 is electrically connected to the pad 10 a of the imaging element 10 via the through-electrode 68 a .

[0227] In addition, the height (thickness) of the insulating substrate 68 can be set to any value. Therefore, for example, by forming the height (thickness) of the entire insulating substrate 68 to be the same as the height (thickness) of the glass 21, the welding height of the imaging element 10 can be aligned. As a result, welding can be performed accurately and easily.

[0228] Since the configuration other than the above is the same as that of the modification of the third embodiment or the fourth embodiment described above, the description thereof will be omitted.

[0229] In addition, the manufacturing process is basically the same as that of the second embodiment described above, and the description of the differences is omitted because they can be realized by adding appropriate changes.

[0230] <7. Sixth embodiment of the sight line detection device according to the present disclosure>

[0231] Next, a sixth embodiment of the sight line detection device 100 according to the present disclosure is described. In this embodiment, Figure 18 As shown in FIG. 1 , the driving substrate 51 is accommodated inside the insulating substrate 68 , which is different from the fifth embodiment.

[0232] The driving substrate 51 is, for example, a substrate of a driving circuit of the display device 20 that, in addition to driving an active element chip 40 such as a display driving IC for driving the aforementioned display device 20, also mounts an electronic circuit composed of passive elements such as resistors and capacitors. In addition, an interface circuit for connecting the imaging element 10 and an external image control unit 220 may be mounted on the driving substrate 51.

[0233] Regarding the configurations other than the above, since they are the same as those of the fifth embodiment described above, the description thereof is omitted.

[0234] In addition, regarding the manufacturing process, it is also basically the same as that of the second embodiment described above. Regarding the differences, since they can be achieved by making appropriate changes, the description thereof is omitted.

[0235] <8. Seventh Embodiment of the Gaze Detection Device According to the Present Disclosure>

[0236] Next, a seventh embodiment of the gaze detection device 100 according to the present disclosure will be described. In the present embodiment, as Figure 19 shown, the imaging element 10 is disposed in close contact with the right side surface of the glass 21 on the semiconductor substrate 30 of the display device 20.

[0237] Moreover, the pads 10a, 10a on the lower surface of the imaging element 10 are respectively soldered to the pads 30a, 30a on the upper surface of the semiconductor substrate 30 through solders 70a, 70a as conductors. Thus, the imaging element 10 is electrically connected to the semiconductor substrate 30.

[0238] Signals from the imaging element 10 and the display device 20 are connected to the external terminals 31 through a wiring pattern (not shown) formed on the semiconductor substrate 30. And via the external terminals 31, Figure 5 as shown, the connections to the image control unit 220 realized through the connection lines 93, 94 can be aggregated into one connection line. Therefore, the routing of the wiring becomes easy, and the man-hours for wiring can be reduced. In addition, since the wiring can be compactly aggregated, redundant connection lines can be reduced, and electromagnetic compatibility (EMC: Electromagnetic Compatibility) can be improved.

[0239] Regarding the configurations other than the above, since they are the same as those of the modified example of the third embodiment described above, the description thereof is omitted. In addition, regarding the manufacturing process, it is also basically the same as that of the second embodiment described above. Regarding the differences, since they can be achieved by making appropriate changes, the description thereof is omitted.

[0240] <9. Eighth Embodiment of the Gaze Detection Device According to the Present Disclosure>

[0241] Next, an eighth embodiment of the line-of-sight detection device 100 according to the present disclosure will be described. In the present embodiment, as Figure 20 shown, a display device 20 including a semiconductor substrate 30 is placed on a substrate 80 and joined by soldering. In addition, an imaging element 10 is placed on the substrate 80 close to the right side surface of the display device 20 so that the distance from the display element 25 is the shortest, and joined by soldering.

[0242] Specifically, pads 30b, 30b on the lower surface of the semiconductor substrate 30 are soldered to pads 80a, 80a on the upper surface of the substrate 80 through solders 70a, 70a as conductors, respectively. Thereby, the semiconductor substrate 30 is electrically connected to the substrate 80.

[0243] In addition, pads 10a, 10a on the lower surface of the imaging element 10 are soldered to pads 80a, 80a on the upper surface of the substrate 80 through solders 70a, 70a, respectively. Thereby, the imaging element 10 is electrically connected to the substrate 80.

[0244] Through the above electrical connection, signals from the imaging element 10 and the display device 20 are connected to the substrate 80. Thereby, for example, by arranging an image control unit 220 on the substrate 80 and forming a predetermined wiring pattern on the substrate 80, the imaging element 10 and the display device 20 can also be directly electrically connected to the image control unit 220.

[0245] In addition to the camera 300 built in the EVF, the line-of-sight detection device 100 according to the present embodiment can also be applied to many electronic devices, industrial devices, or devices of social infrastructure, etc.

[0246] Regarding the constitution other than the above, since it is the same as the constitution of the foregoing seventh embodiment, the description thereof is omitted. In addition, regarding the manufacturing process, it is also basically the same as the foregoing second embodiment, and regarding the differences, since they can be achieved by adding appropriate changes, the description thereof is omitted.

[0247] <10. Glasses terminal having the line-of-sight detection device according to the present disclosure>

[0248] As Figure 21 shown, in the glasses terminal 200 according to the present disclosure, a line-of-sight detection device 100 having an imaging element 10 and a display device 20 is arranged at a predetermined position of a frame 201. The shape of the frame 201 may be any shape as long as it can be mounted in front of the eyeball 400. Therefore, it is not limited to the shape of a head-mounted display used for head mounting.

[0249] In the glasses terminal 200 according to the present invention, a light source 90 is arranged at a predetermined position. Then, in a state where the user is observing an image displayed on the display device 20 through the lens 203, the light source light 91 of the light source 90 is irradiated onto the eyeball 400, the imaging element 10 images an image of the eyeball 400 including its reflected light 92, and eye tracking is performed based on the captured image.

[0250] As Figure 22 shown, the glasses terminal 200 according to the present disclosure includes a control unit 211, a storage unit 212, an input / output unit 213, an operation unit 217, a power supply unit 218, a light irradiation unit 215, an image control unit 220, and a line-of-sight detection device 100 connected to the image control unit 220.

[0251] The control unit 211, the storage unit 212, the input / output unit 213, the operation unit 217, the power supply unit 218, the light irradiation unit 215, and the image control unit 220 are interconnected via a bus line 219.

[0252] The control unit 211 includes a program control device such as a CPU and performs various information processes according to a program stored in the storage unit 212.

[0253] The storage unit 212 includes storage elements such as a RAM and a ROM, and stores programs and the like executed by the control unit 211. In addition, the storage unit 212 also functions as a working memory of the control unit 211.

[0254] The input / output unit 213 is, for example, an input / output interface such as an HDMI (registered trademark: High-Definition Multimedia Interface) port or a USB port for image signals and the like.

[0255] The operation unit 217 sets information required for starting the glasses terminal 200, such as passwords, menu selections, image quality, and sound quality, and the usage environment.

[0256] The power supply unit 218 creates a predetermined DC voltage from a commercial power supply, for example, and supplies power to each unit. Alternatively, it may be configured to have a built-in battery, create a predetermined DC voltage from the voltage of the battery, and supply power to each unit.

[0257] The light irradiation unit 215 incorporates a light source 90 such as an LED as an optical element, and controls the irradiation of light in a wavelength band other than the visible light band such as infrared light.

[0258] As described above, the imaging element 10 of the line-of-sight detection device 1000 images an image of the eyeball 400 including the reflected light 92 of the light source light 91 irradiated from the light source 90 onto the eyeball 400, and outputs the captured image to the image control unit 220.

[0259] The image control unit 220 performs predetermined image processing on the captured image to generate image data, and outputs the image data to the control unit 211.

[0260] In addition, the image control unit 220 performs control for displaying a predetermined image on a liquid crystal display, an organic EL display, or the like that constitutes the display device 20. A video or the like generated based on the video signal 96 is displayed on the display device 20. The video signal 96 is received, for example, from an entertainment device such as a home game console, a DVD player, or a Blu-ray (registered trademark) player that is connected via the input / output unit 213 and not shown. It should be noted that, in order to be closer to a real-life experience, the display device 20 may also be configured to be able to display a three-dimensional video.

[0261] The control unit 211 performs eye tracking based on the image data. Specifically, the control unit 211 detects the position and the line-of-sight direction of the eyeball 400 based on the image data including the user's eyeball 400 and its reflected light 92 obtained by capturing an image with the imaging element 10 of the image control unit 220. Here, it is assumed that the control unit 211 uses a known line-of-sight detection technique to detect the position and the line-of-sight direction of the user's eyeball 400.

[0262] It should be noted that the image control unit 220 may also be connected, for example, to a plurality of line-of-sight detection devices 100 that capture an image of the user's eyeball 400. In addition, the imaging element 10 may capture an image of visible light, and is not limited to infrared light.

[0263] Alternatively, it may be configured to transmit the image data including the user's eyeball 400 and its reflected light 92 obtained by capturing an image with the imaging element 10 to an external information processing device (not shown) via the input / output unit 213. In this case, it may also be assumed that the external information processing device uses the received image of the user's eyeball 400 to detect the line-of-sight direction of the user based on the position, the rotation angle, etc. of the user's eyeball 400.

[0264] As described above, according to the present disclosure, by using the line-of-sight detection device 100 related to the present disclosure, the glasses terminal 200 capable of performing highly accurate line-of-sight detection can be obtained at low cost.

[0265] <11. Camera having the line-of-sight detection device related to the present disclosure>

[0266] As Figure 23As shown, the camera 300 according to the present disclosure includes an EVF 330 having the gaze detection device 100 according to the present disclosure. The image displayed on the display device 20 of the EVF 330 is configured to be displayed on the EVF 330 and a display such as a liquid crystal panel (hereinafter referred to as "LCD") 333 arranged on the back of the camera body 301 after the imaging signal captured by the solid-state imaging device 322 built in the camera 300 is converted into image data by the image processing unit 30 and the like.

[0267] Since the camera 300 according to the present disclosure is configured in this way, an image in a state reflecting the set shooting conditions, the shooting conditions after setting change, etc. can be observed through the display device 20 of the EVF 330 and the LCD 333. Therefore, it is possible to confirm what kind of shooting method to perform before shooting.

[0268] It should be noted that the screen of the EVF 330 is easy to use when shooting a moving subject because it has a feeling of oneness with the line of sight. In addition, since the screen of the LCD 333 is close to the completed state of the photo, it is easy to determine the composition, etc., and is suitable for photographing landscapes, people, etc.

[0269] Since the camera 300 according to the present disclosure has both the EVF 330 and the LCD 333 screens, it can be used separately according to the purpose.

[0270] Regarding the external configuration of the camera 300 according to the present disclosure, as Figure 23 shown, a lens 302 is arranged at the front position (the left side of this figure) of the camera body 301, and the EVF 330 and the LCD 333 are arranged at the back position (the right side of this figure).

[0271] The lens 302 refracts and transmits the light from the subject, and through the transmitted light, the captured image of the subject is formed on the solid-state imaging device 322.

[0272] The EVF 330 performs predetermined image processing on the captured image of the solid-state imaging device 322 in the image processing unit 320 and the like, and displays it on the built-in display unit 20.

[0273] The LCD 333, like the EVF 330, performs predetermined image processing on the captured image of the solid-state imaging device 322 in the image processing unit 320 and the like and displays it.

[0274] In addition, in the camera body 301, various switches required for operation are arranged at positions accessible by fingers on its upper surface as an operation unit 317. The operation unit 317 includes a shutter button, a mode setting dial, a control value setting dial, a set button group, a cross key, a control button main switch, etc.

[0275] The power supply unit 318 is constituted by, for example, a constant voltage circuit or the like, and generates a DC voltage for driving various driving units such as each circuit including the control unit 311, the solid-state imaging device 322, the LCD 333, the display unit of the display device 20, and other lens control units 316, etc., of the entire camera 300, and supplies power. The power supply unit 318 incorporates a primary battery such as an alkaline dry battery, or a secondary battery such as a nickel-metal hydride rechargeable battery, and enables photography even in a state where it is not connected to a commercial power supply.

[0276] As Figure 23 As shown, the internal functional configuration of the camera 300 according to the present disclosure includes a control unit 311, a storage unit 312, a lens control unit 316, a shutter 321, a lens 302, a solid-state imaging device 322, an analog front end (AFE: Analog Front End, hereinafter referred to as "AFE".), 323, an image processing unit 320, an image memory 324, an LCD 333, incorporates a gaze detection device 100, an EVF 330 including a light irradiation unit 315 including a light source 90 and an eyepiece 303, the aforementioned operation unit 317, and the aforementioned power supply unit 318. They are interconnected via a bus line (not shown) or a predetermined signal line or the like.

[0277] The control unit 311 includes a program control device such as a CPU, and executes various information processes in accordance with a program stored in the storage unit 312.

[0278] The storage unit 312 includes storage elements such as a RAM and a ROM, and stores programs and the like executed by the control unit 311. In addition, the storage unit 312 also functions as a working memory of the control unit 311.

[0279] When the shutter button of the operation unit 317 is pressed (in a state where it is pressed halfway), the lens control unit 316 moves the lens 302 forward and backward to a predetermined focal length in combination with photographic conditions such as a photographic mode, and in combination with the position of the user's viewpoint.

[0280] The shutter 321 is operated based on photographic conditions such as a shutter speed set by the operation unit 317 by pressing the shutter button of the operation unit 317 (fully pressed), and irradiates the transmitted light from the lens 302 onto the solid-state imaging device 322. The solid-state imaging device 322 is, for example, a CMOS image sensor, and photoelectrically converts the transmitted light irradiated via the shutter 321.

[0281] The AFE323 is composed of, for example, a timing control section, a signal processing section, an A / D conversion section, etc. The AFE323 is a circuit for enabling digital signal processing of the electrical signals generated by photoelectric conversion in the solid-state imaging device 322 in the image processing section 320 and the control section 311. The captured image of the solid-state imaging device 322 is converted and processed into image data of digital signals by the A / D conversion section provided in the AFE323.

[0282] The image processing section 320 is composed of, for example, a black level correction section, a WB processing section, a γ correction processing section, etc. The image processing section 320 performs predetermined signal processing on the image data that has undergone signal processing in the AFE323 based on the shooting mode, aperture value, ISO sensitivity, etc. set by the operation section 317 to generate image data that is set as the captured image in reality.

[0283] The image memory 324 is a memory that temporarily stores the image data that has undergone predetermined signal processing in the image processing section 320 during shooting. In addition, the image memory 324 is a memory used as a work area for the control section 311 to perform predetermined processing on the stored image data. Also, the image memory 324 temporarily stores the image data read from a memory card (not shown) such as an SD card during reproduction.

[0284] The LCD 333 is composed of a color liquid crystal panel capable of image display, an organic EL display, etc., and performs display of the image of the subject captured by the solid-state imaging device 322, reproduction display of the recorded image, etc. In addition, the LCD 333 displays the setting screen and can select a suitable shooting mode from the shooting mode, aperture value, ISO sensitivity, etc. required for shooting in combination with the shooting conditions.

[0285] The EVF 330 includes a line-of-sight detection device 100 including a display device 20 and an imaging element 10, a light source 90, and an eyepiece 303. The display device 20 is composed of a color liquid crystal display or an organic EL display capable of image display and can display the image captured by the solid-state imaging device 322. The user visually confirms the image displayed on the display device 20 through the eyepiece 303. With such a configuration of the EVF 330, the user can visually confirm the image to be captured by the solid-state imaging device 322 or the reproduction display of the captured image, etc.

[0286] In the EVF 330 of the camera 300 according to the present disclosure, the imaging element 10 is disposed adjacent to the side surface of the display device 20, and the light source 90 is disposed adjacent to the side surface of the imaging element 10. Moreover, in a state where the user observes the image displayed on the display device 20 in the shooting mode, the light source light 91 of the light source 90 is irradiated onto the eyeball 400 of the user, and the imaging element 10 images the image of the eyeball 400 including the reflected light 92.

[0287] The control unit 311 inputs the captured image data of the imaging element 10 via the aforementioned AFE 323, image processing unit 320, and image memory 324, and performs arithmetic processing for eye tracking based on the captured image data. The control unit 311 infers the line of sight based on the result of the arithmetic processing of the eye tracking, and correspondingly, the lens control unit 316 moves the lens 302 back and forth to match the focal length with the viewpoint.

[0288] The control unit 311 infers the line of sight of the user based on the image data of the eyeball 400 of the user captured by the imaging element 10 as described above, and controls the lens 302, thereby enabling shooting of a subject that matches the viewpoint of the user.

[0289] Regarding the wavelength band of the light source 90 and the method for detecting the line of sight by the control unit 311 other than the above, since they are the same as those of the aforementioned glasses terminal 200, the description thereof is omitted. In addition, the flash function, moving image shooting function, recording function, etc. of the camera 300 are also omitted.

[0290] It should be noted that, in the present embodiment, an example in which the line of sight detection device 100 is provided in the EVF 330 has been described, but it may also be provided in the LCD 333. In addition, it may be provided in both the EVF 330 and the LCD 333. In addition, for example, the EVF 330 may also be connected to a plurality of line of sight detection devices 100 that image the eyeball 400 of the user. In addition, the imaging element 10 can also image visible light and is not limited to infrared light.

[0291] In summary, according to the present disclosure, by using the line of sight detection device 100 according to the present disclosure, a camera 300 capable of performing highly accurate line of sight detection can be obtained at low cost.

[0292] Finally, the description of each of the above embodiments is an example of the present disclosure, and the present disclosure is not limited to the above embodiments. Therefore, even if it is other than the above embodiments, as long as it is within the scope not departing from the technical idea of the present disclosure, various changes can of course be made according to the design, etc. In addition, the effects described in this specification are merely examples and are not limited, and there may be other effects.

[0293] It should be noted that the present technology can also adopt the following configuration. (1)

[0295] A line-of-sight detection device, comprising:

[0296] A display device for displaying an image; and

[0297] An imaging element for detecting the user's viewing point, which is disposed along the outer periphery of the effective pixel region of the display device and is adjacent to the effective pixel region. (2)

[0299] The line-of-sight detection device according to (1), wherein

[0300] The display device has:

[0301] A semiconductor substrate;

[0302] A display element disposed on the semiconductor substrate; and

[0303] A light guide disposed on the semiconductor substrate with the display element interposed therebetween, and sealing the display element between itself and the semiconductor substrate. (3)

[0305] The line-of-sight detection device according to (1), wherein the imaging element is disposed on the short side of the outer periphery of the effective pixel region. (4)

[0307] The line-of-sight detection device according to (1), wherein the imaging element is disposed on the long side of the outer periphery of the effective pixel region. (5)

[0309] The line-of-sight detection device according to (1), wherein the imaging element is disposed at a corner of the outer periphery of the effective pixel region. (6)

[0311] The line-of-sight detection device according to any one of (2) to (5), wherein one end of the imaging element is placed or joined on the upper surface of the light guide, and the other end of the imaging element is electrically connected to the semiconductor substrate via a conductive height corrector. (7)

[0313] The line-of-sight detection device according to any one of (2) to (6), wherein the joining of one end of the imaging element to the upper surface of the light guide is an adhesive bonding achieved by an adhesive, and the electrical connection of the other end of the imaging element to the semiconductor substrate via the height corrector is a welding. (8)

[0315] The line-of-sight detection device according to (6) or (7), wherein the height correction object is a conductive pillar. (9)

[0317] The line-of-sight detection device according to (6) or (7), wherein the height correction object is a silicon chip. (10)

[0319] The line-of-sight detection device according to (6) or (7), wherein the height correction object is an active element chip. (11)

[0321] The line-of-sight detection device according to (6) or (7), wherein the height correction object is an insulating substrate having TSVs. (12)

[0323] The line-of-sight detection device according to (6) or (7), wherein the height correction object is the insulating substrate having a driving substrate. (13)

[0325] A line-of-sight detection device, comprising:

[0326] A display device that seals a display element between a semiconductor substrate and a light guide and displays a formed image; and

[0327] An imaging element for detecting the user's viewing point, disposed on the semiconductor substrate in close contact with the light guide. (14)

[0329] A line-of-sight detection device, comprising:

[0330] A display device that displays an image;

[0331] An imaging element for detecting the user's viewing point; and

[0332] A substrate that disposes the display device and the imaging element in close contact with each other. (15)

[0334] The line-of-sight detection device according to any one of (1) to (14), wherein the imaging element is electrically connected to the display device via the conductor. (16)

[0336] The line-of-sight detection device according to any one of (1) to (15), wherein the electrical connection between the imaging element and the display device via the conductor is achieved by self-alignment of soldering. (17)

[0338] The line-of-sight detection device according to any one of (1) to (16), wherein the normal line of the imaging surface of the imaging element is arranged to face the same direction as the optical axis of the display device. (18)

[0340] A method for manufacturing a line-of-sight detection device, comprising:

[0341] A step of preparing a display device including a semiconductor substrate and a light guide;

[0342] A step of forming or bonding a height corrector having conductivity to the semiconductor substrate;

[0343] A step of forming protrusions on the height corrector;

[0344] A step of applying solder to the pads of the imaging element;

[0345] A step of respectively placing the pads of the imaging element on the upper surface of the light guide and the upper surface of the protrusions of the height corrector; and

[0346] A step of soldering the imaging element to the upper surface of the light guide and the upper surface of the protrusions by reflow soldering. (19)

[0348] A glasses terminal, comprising:

[0349] A line-of-sight detection device,

[0350] The line-of-sight detection device has:

[0351] A display device for displaying an image; and

[0352] An imaging element for detecting the user's viewing point, which is arranged close to the outer periphery of the effective pixel region of the display device and adjacent to the effective pixel region, or

[0353] The line-of-sight detection device has:

[0354] A display device for sealing a display element between a semiconductor substrate and a light guide and displaying the formed image; and

[0355] An imaging element for detecting the user's viewing point, which is arranged close to the light guide on the semiconductor substrate, or

[0356] The line-of-sight detection device has:

[0357] A display device for displaying an image;

[0358] An imaging element for detecting the user's viewing point; and

[0359] A substrate that closely arranges the display device and the imaging element. (20)

[0361] A camera having:

[0362] A line-of-sight detection device,

[0363] The line-of-sight detection device has:

[0364] A display device that displays an image; and

[0365] An imaging element for detecting the user's viewing point, which is arranged on the outer periphery of the effective pixel region of the display device in close contact with the effective pixel region of the display device, or

[0366] The line-of-sight detection device has:

[0367] A display device that seals a display element between a semiconductor substrate and a light guide and displays the formed image; and

[0368] An imaging element for detecting the user's viewing point, which is arranged on the semiconductor substrate in close contact with the light guide, or

[0369] The line-of-sight detection device has:

[0370] A display device that displays an image;

[0371] An imaging element for detecting the user's viewing point; and

[0372] A substrate that closely arranges the display device and the imaging element.

[0373] Explanation of reference numerals

[0374] 10: Image pickup element; 10a: Pad; 12: Color filter; 13: Lens array; 20: Display device; 21: Light guide (glass); 22: Sealant; 23: Sealed area; 24: Effective pixel area; 25: Display element; 30: Semiconductor substrate; 30a, 30b: Pads; 31: External terminal; 40: Active element chip; 40a, 40b: Pads; 51: Driving substrate; 60: Support pillar; 60a: Projection; 65: Silicon chip; 65a, 65b: Pads; 68: Insulating substrate; 68a: Through electrode; 70a, 70b: Solder; 71: Adhesive; 80: Substrate; 90: Light source; 91: Light source light; 92: Reflected light; 93: Connection line; 94: Connection line; 96: Video signal; 100: Line-of-sight detection device; 200: Glasses terminal; 201: Frame; 203: Lens; 211: Control unit; 215: Light irradiation unit; 220: Image control unit; 300: Camera; 301: Camera body; 302: Lens; 303: Eyepiece; 311: Control unit; 315: Light irradiation unit; 320: Image processing unit; 330: EVF; 333: LCD; Z: Position deviation.

Claims

1. A line-of-sight detection device, comprising: a display device that displays an image; and an imaging element for detecting the user's viewing point, which is disposed along the outer periphery of the effective pixel region of the display device in close contact with the effective pixel region.

2. The line-of-sight detection device according to claim 1, wherein the display device includes: a semiconductor substrate; a display element disposed on the semiconductor substrate; and a light guide body disposed on the semiconductor substrate with the display element interposed therebetween, and sealing the display element between the light guide body and the semiconductor substrate.

3. The line-of-sight detection device according to claim 1, wherein the imaging element is disposed on a short side of the outer periphery of the effective pixel region.

4. The line-of-sight detection device according to claim 1, wherein the imaging element is disposed on a long side of the outer periphery of the effective pixel region.

5. The line-of-sight detection device according to claim 1, wherein the imaging element is disposed at a corner of the outer periphery of the effective pixel region.

6. The line-of-sight detection device according to claim 2, wherein one end of the imaging element is placed or joined on the upper surface of the light guide body, and the other end of the imaging element is electrically connected to the semiconductor substrate via a conductive height corrector.

7. The line-of-sight detection device according to claim 2, wherein the joining of one end of the imaging element to the upper surface of the light guide body is an adhesion achieved by an adhesive, and the electrical connection of the other end of the imaging element to the semiconductor substrate via the height corrector is a soldering.

8. The line-of-sight detection device according to claim 6, wherein the height corrector is a conductive pillar.

9. The line-of-sight detection device according to claim 6, wherein the height corrector is a silicon chip.

10. The line-of-sight detection device according to claim 6, wherein the height corrector is an active element chip.

11. The line-of-sight detection device according to claim 6, wherein the height corrector is an insulating substrate having TSVs.

12. The line-of-sight detection device according to claim 6, wherein the height corrector is the insulating substrate having a driving substrate.

13. A line-of-sight detection device, comprising: a display device that seals a display element between a semiconductor substrate and a light guide body and displays the formed image; and an imaging element for detecting the user's viewing point, which is disposed on the semiconductor substrate in close contact with the light guide body.

14. A line-of-sight detection device, comprising: a display device that displays an image; an imaging element for detecting the user's viewing point; and a substrate that disposes the display device and the imaging element in close contact with each other.

15. The line-of-sight detection device according to claim 1, wherein the imaging element is electrically connected to the display device via the conductor.

16. The line-of-sight detection device according to claim 1, wherein the electrical connection between the imaging element and the display device via the conductor is achieved by self-alignment of soldering.

17. The line-of-sight detection device according to claim 1, wherein The normal line of the imaging surface of the imaging element is arranged to face the same direction as the optical axis of the display device.

18. A method for manufacturing a line-of-sight detection device, comprising: a step of preparing a display device including a semiconductor substrate and a light guide; a step of forming or bonding a height corrector having conductivity on the semiconductor substrate; a step of forming a protrusion on the height corrector; a step of applying solder to the pads of the imaging element; a step of respectively placing the pads of the imaging element on the upper surface of the light guide and the upper surface of the protrusion of the height corrector; and a step of soldering the imaging element to the upper surface of the light guide and the upper surface of the protrusion by reflow soldering.

19. A glasses terminal, comprising: a line-of-sight detection device, wherein the line-of-sight detection device includes: a display device that displays an image; and an imaging element for detecting the user's viewing point, which is arranged closely around the effective pixel area of the display device or wherein the line-of-sight detection device includes: A display device that seals a display element between a semiconductor substrate and a light guide and displays an image formed thereby; and an imaging element for detecting the user's viewing point, which is arranged closely on the semiconductor substrate with respect to the light guide, or wherein the line-of-sight detection device includes: a display device that displays an image; an imaging element for detecting the user's viewing point; and a substrate for arranging the display device and the imaging element closely.

20. A camera, comprising: a line-of-sight detection device, wherein the line-of-sight detection device includes: a display device that displays an image; and an imaging element for detecting the user's viewing point, which is arranged closely around the effective pixel area of the display device or wherein the line-of-sight detection device includes: A display device that seals a display element between a semiconductor substrate and a light guide and displays an image formed thereby; and an imaging element for detecting the user's viewing point, which is arranged closely on the semiconductor substrate with respect to the light guide, or wherein the line-of-sight detection device includes: a display device that displays an image; an imaging element for detecting the user's viewing point; and a substrate for arranging the display device and the imaging element closely.

Images